Get a workstation. We used GCP/Vertex to do batch prediction on a computer vision model, but for larger videos it inexplicably fails. Google has spent 6 weeks now trying to figure out why it doesn’t work (everyone, including Google engineers, are in agreement that the model container is not the problem). They still don’t have an answer.

We ended up investing in building our own multi-GPU server and not only are our prediction times better, but we can instantly see and diagnose issues that arise.

One of the often overlooked aspects of using public clouds is that there are several layers of abstraction that remove you from what’s happening under the hood. If something happens behind the scenes that you can’t readily diagnose and fix yourself, you’re basically at the mercy of AWS et al to provide you with an answer.

For 10-12k, you can get a handful of high end consumer cards and a boatload of memory, and you have full control of the system.

A bare 128gb 16cpu, 2gpu instance, using reserve instances would be $800/month and that's without any bells and whistles (like an EBS volume or data transfer) so I think you would last less than a year.

Given that hardware should last you a couple of years (hopefully), it's a better way to spend your money than EC2.

Honestly lookup Paperspace Gradient and consider their monthly service. They have a tier where you can quite routinely get decent free GPUs, which honestly when you're just working up code and refactoring and making sure a training run is actually going to run then it's perfect for that. Then when you're ready to let something run overnight then you select an or whatever A6000 and it's reasonably priced.

Those $10,000 won't last long on AWS. There's also LambdaLabs, their cloud prices are a lot more affordable. They also make dedicated servers for machine learning.

Setting up a machine with the right GPU and drivers and CPU and cooling can be a PIA, and takes time!

Defaulted to one day.

I will be messaging you on 2022-11-28 15:49:18 UTC to remind you of this link

2 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^(Parent commenter can ) ^(delete this message to hide from others.)

!remindme

https://fleuret.org/dlc/

TIL thanks! Will have to check it out.

Not exactly answering your question, but a 4090 is probably quite a bit of overkill for personal ml projects. On the rare occasion you will be training models that require those resources, you can probably leverage cloud based resources and it will be cheaper in the long run.

With that said you mentioned gaming being a primary use case of your system. Depending on what you play I assume you will want a windows os, and if that is the case I will echo what others have said and say wsl2 is great.

I personally used a 2070 with wsl 2 for graduate work In computer vision and deep learning.

I think it’s great from a local workstation perspective and see it no more fragile than a standard windows, Mac, or Linux work station environment.

I do agree it doesn’t compare to a professional server cluster. I use wsl at work for a local workstation (small builds/jobs and debugging) but farm out big jobs to a server cluster.

Based on the described use case I think wsl would be a great option. I use it on my personal machine and transitioning from a code/ml project to playing a game of league is so easy compared to switching to my dual boot partition. Running wsl does share system resources with a second os, but the over head of windows is not that large. I personally never run into the case where native Linux provides a Benifit over wsl for jobs small enough that I wouldn’t just farm out to a cloud based server (very rare with my personal projects).

Message me personally i have free vedios of a 100$ course

3blue1brown has the best high level explanation/intuition: https://m.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

Andrej Karphaty has some amazing videos where he implements some things: https://m.youtube.com/c/AndrejKarpathy/

There are full courses on Youtube from Stanford and MIT which are excellent.

Same kinda, could check out dan bourke, he has some basic tf / pytorch beginner videos

Lambda Stack ftw

Check out deeplizard

Following

Oh shit - this changes everything.... might need to revisit WSL.

Yeah it seems to have come a long way. I was prepared for an awful experience based on what others had said, but I found it to be pretty smooth.

Sorry but I also mention arch packages are for bleeding edge tech and garuda isn't for expert Linux users. Better this than Debian distributions even in stability

Garuda kde dragonized btw

Yes. Id say go for an arch based distribution like garuda Linux (perfect for gaming and regular Linux use). Remember never get noveau drivers for your gpu. You can qemu-kvm a windows 10/11 slim etc ISO or just dual boot but it's a Linux machine better just kvm windows.

Even more recently MS have provided their own solution for Windows 10 (provided you’re running a sufficiently updated version) and 11, which supports both X11 and Wayland.

See here albeit this document is slightly out of date because, as of just a few days ago, they now support Windows 10 from 19.044 (21H2) onwards iirc. Edit: as per here.

Is it ok to use windows now? I remember 5 years ago I have to install numpy in windows from unofficial repo

Yes. You can go with Windows but if you don't want to be bothered by your environment and things fo go smoothly take a mainstream Linux distro or something based upon it.