If you have to ask this, most likely a 3080ti is way overkill for your needs.

If you have a huge budget to burn for a beginner, buy either one. Nevertheless, I suggest you start with Google Colab or other free services. Then you can get some feel for the kind of hardware you need for the work you want to do.

It will take you a while to get to a point where Colab is not enough. By that time you won't be a beginner anymore and you will be able to wisely choose a suitable gpu (which may well be no gpu at all, maybe a cloud service fits your use case better?)

You don't even need a computer to learn about deep learning, let alone a high-end graphics card like an RTX 3080ti.

Go for the 3090 unless they're the same price. The 3090Ti is slightly more performant, from our benchmarks up to 10-15% in training, but it generates much more heat and noise, and consumes way more power.

No particular reason. I'm going to try different patch sizes. Regarding the batch size, right now I'm using 32. Discarding chunks without mass, I get a dice score of 0.8 approx. which isn't really great, but I still have many things to try. And I'm checking recall and precision (and specificity) at the end of each epoch.

I'm going to try a bigger patch size and change the data loader as you suggested. Thanks for your help!

Without a doubt. You get more than double the VRAM (11GB -> 24GB), and you get tensor cores which are significantly faster, and also half-precision tricks give you effectively double VRAM again compared to FP32. A 3090 (possibly Ti) can train the kinds of models you used to train on 4x1080Ti.

3090/ 3090ti Definitely worth the update

Imo 3090 is more than enough for personal use. For dense DL research, one 3090ti card won’t have visible difference than one 3090. Also 3090ti requires much higher total power supply(~1000 vs 3090: ~800).

Is there a particular reason why you chose a patch size of 32x32? You have a rather shallow U-Net with this small patch size and not that much information in your patches but look rather uniform. I would try to go for maybe 128x128 or even 256x256. For 2d segmentation that should still not be too memory intensive. What's the batch size you are using? If you use for example a batch size of 32 you could force in the data loader that at least 8 of the scans should have some mass in them. Just play around a bit with this number to see how it works. Keep an eye on recall and precision in addition to dice to see if your false positives will rise.

Our use case was in biometrics, where the test sample would usually range in millions of images which needed to be matched simultaneously. Over here even accumulating 2-3ms over each batch or batch would lead to huge delay.

The images are grayscale x-ray images that have masses in them. For example, one of the images may have a shape of (2000, 1500, 1). I extract chunks of (32, 32, 1) and use those chunks to train the segmentation network. The dice score and loss is calculated on the foreground, and there is only one class (mass). I'm going to change the data loader to use only chunks with at least some mass in them, although my guess is that then the trained model is going to find masses in empty chunks too. Thanks for your suggestions!

woah that is a cool project.

okay I could see how I was thinking about it kind of wrong. Thanks for the reply

that could definitely be one solution

For live video at 30fps you get 33ms to read the frame, run inference on the frame and draw detection boxes + overhead.

Also because it's super user friendly and easy to implement, have a look at pytorch lightning. They make distributing and such very easy

This is called model parallelism. Think of this as having model.conv1 on gpu1 and model.conv2 on gpu2. This is actually not too hard to do as you just need to manually specify your model components with statements like .to(“cuda:”). Start with this.

A more advanced model is model parallelism + data parallelism where you can benefit from having both gpus split the dataset to accelerate the training. Typically this is not possible with simple model parallelism, but an advanced model like fairseq can do it for you.

This kind of thing is likely applicable to digital twins of many fields. The idea is to create a digital representation of whatever you are trying to model and run it alongside the real thing. It has applications in control engineering and predictive / prescriptive analytics. Depending on the application this could be done many ways (not necessarily using neural nets at all) and be fast or slow to run.

They care about cost for certain. Speed and hardware may relate to that.

Of course!

www.sasdghub.up.ac.za

The semantic search as well as a few other key features haven't made it up yet. We're aiming to have them up end of November, mid December.

We've got a two server setup with the second being our "Work-horse" intended for GPU related jobs. It's an RTX 3090 with 32GB VRAM, 64GB DDR4 RAM and a 8 core CPU (I forget it's exact setup)

Well then it's a matter of trust - every serious cloud provider has a privacy policy that claims nothing is logged. Of course, you don't have to trust this, but this is a liability for the cloud provider, so you get to shift the blame if something goes wrong. And I'd argue that for most companies the word of a cloud peovider means more than your word, since they've got much to lose.

It's also standard practice to use end-to-end encryption, with some using end-to-end encrypted models. I don't really see a way how our company would handle personal data and retain samples in a GDPR compliant way without proprietary models in the cloud.

That's right but sometimes data sensitivity prevents the use of cloud.

For which use case if you don't mind me asking? And are you referring to inference or training?

The perspective of your use case makes so much sense. I appreciate you sharing that info!

Mind sharing which use case that would be? I'm also trying to pin point which industries care about model speed.

Really interesting. And that's kind of where my mind was leaning towards.

Faster training usually means more training from a cost perspective.

Thanks for sharing!

OK right. That's also a project with immense scale.

I guess the bigger the project, the more inference speed is required. But I've never heard about caring deeply about the ms in training. Mind sharing why that was important in that use case?