Since you mentioned Huddingface, What do you think of Pegasus for Abstractive Summarization?

Fine-tuning flan T5 xl or XXL can give you decent results. From my experience these are best open source models to fine-tune on . However they won't match results of larger models like gpt 3.5. But if you have millions of such reviews then chatgpt or gpt 3.5 may not be financially feasible.

If you have the budget, gpt3/gpt3.5 are pretty great at it.

If you want to do it on your hardware/cloud instances.

You can go for gpt-j or flan-T5, they do great with basic prompts.

Or you can try a finetuned model, you might be able to get a finetuned T5 or there's BART and pegasus as well.

For summarisation even smaller models are pretty good so don't just jump to the biggest models. Compare the performance.

No I need scalability, so consider millions.

How many of them are there? If you've got 100's of them, you could do it for a couple bucks or less with gpt-3 davinci without a ton of prompt engineering. If I had a ton of them I'd probably go with gpt-j and see if could do a serviceable job with few-shot learning.

You can use kaggle as well for training your DL models. Others platforms you need to pay for it.

Kaggle

Paperspace

Here's a good price comparison chart for all providers:

https://fullstackdeeplearning.com/cloud-gpus/

Use some spot instances on the cloud. It's a lot cheaper than getting a 3k rig unless if you train the model throughout the entire year. You can also connect to a VM on GCP through Colab so everything feels the same, just with a change.

Spot instances and Lambda cloud have super low margins and it isn't the big mark up in traditional products.

I think some models restrict training to gpu, but I may be wrong and its just configuration

You can use even a cpu to train. It doesn't matter, it's just slower.

Azure ML has compute instances.

I'm only familiar with Kaggle/Colab for free GPUs

As of right now, nothing is as fast as CUDA. You don't have to take my word for it. There are papers out on the topic.

GitHub has codespaces for machine learning.

Thank you.

I agree, now trying to get the company pay for it.

I'm learning the basics of what is used, for example from this video

There are some AMD GPU chips that might work. But Nvidia is just so much better integrated with TensorFlow and Pytorch and thus the much better hardware.

I equate machine learning to art. You may have great ideas and know-how but at some point you have to buy the material (e.g., good paint, canvas, the better brushes, clay, etc.) to bring your ideas to life.

I know it's frustrating having slow machines when testing models... It really kills the enthusiasm.

I suppose the best thing to do is get into or involved with or work for groups that have the hardware and go from there, or buy new good hardware, or build your own for half price.

I see.

But is nvidia gpus the only popular way to compute fast to your knowledge? I cant believe that cant be achieved w a Radeon card.

Great info, thank you. I am not against paying but it is a shame if you are just started and have skills to write models but cant train them properly.

My laptop is fairly old Lenovo, has no NVIDIA but Radeon graphics card. So it cant use cuda if I am correct.

I wonder how difficult it to set it up for calculations. That card is useful for "gpu calculations with cuda" correct? That is so much faster afaik.

You can get around this with "sweat equity". But money is more effective.

I've used Ray.io in the past to connect old laptops with Nvidia GPUs together on a local network to get the job done.

Bottom line, in this field you will have to pay. Equipment, the tech, matters.

For most models I use my PC or laptop both of which have decent Nvidia cards, good CPUs, and ample RAM. I develop my models off line and when ready (if need be) for the final large scale training I use my AWS/Google accounts.

For super large models I go straight to AWS or Google.

I just recently build a PC and set it up with Ubuntu server. I thought it was going to be a hell to set up, but it was really fun and only took a couple of days. I made a guide on the hardware and everything I did to run Fastai on it link to the guide it should be the most recent reply on that thread.

Anyway, Linux is really really useful for this line of work, I'd suggest you learn it :)

Edit: spelling

Why exactly do they need to be brand new? I've heard about those used for mining but other than that anything else? Thx