Yeah I’m not following either. What do you mean by yolov7 algorithm? Yolov7 is a model architecture, maybe they are talking about nms and iou?

I have trained locally and in the cloud on a variety of cards and server arch’s, depending on what model you are training it could be for a huge variety of reasons, but if you can fit the model on a 3080 you really aren’t going to be taking advantage of the A100s huge memory, the higher clock speed of the 3080 might simply suit this model and parameter set better.

What kind of condition. Can it be a post processing thing that happens at inference?

What do you mean least problems and why is one worst case?

I actually assumed you will be having 2 PSUs. For least problems, buy 2xAX1600i, for cheaper option buy 2xAX1200i. One PSU is actually the worst case, but yeah you can try with a single SFL 2000.

It's a budgeting issue, if I could do 4 4090s, I would.

For sure that's the upgrade path in the future, but rn my electricity is free so it's not too much of an issue.

3090 is not a good card, running at high temperatures, high noise, excessively high VRAM temperatures.

You save a lot electricity cost. Much beter value if you plan to use it a lot. It"s much easier to manage 2 cards than 4 cards.

Performance is only 1.9 times the 3090 for deep learning and price is more than double, just bad value rn.

Buy 2× 4090

[deleted]

I haven't looked into that. I would guess it wouldn't matter in my case, but I might be wrong.

I see, thanks! In that case, I might be asking the vendor more questions.

Then I agree. If you are doing ResNet inference on 8K images, then it will probably be quite slow. However 8K segmentation will probably be even slower (the point of comparison that I was thinking of).
Also, when you get to large images, I suspect the PCIe will become a bottleneck (sending data to the GPUs), which will not be helped by the setup described by the OP.

There would be a 5-8% overhead for the same gpu in a bare vm vs physical comparison. A100 is significantly faster for ML workloads than a 3090 iirc. So it’s probably something related to how it’s setup in your case. Also try using a single gpu instead of distributed learning if you are. MPI might be leading to more overhead in your compute node.

i’m not into deep learning so fast as too using cloud gpu and local. I have benchmarked one locally and not sure if i did it properly so i really don’t have any advice. my question is if ECC buffering plays a factor in how the data is processed and stored? I thought that was one advantages of using a workstation gpu but i could be way off.

I'm seeing the argument for 2 1600 PSUs. It's fine for the mining rig case frame, but it's baiscly confirming to me that this is never going to fit in a case lol.

I suppose one PSU will take care of motherboard + CPU + some GPUs and the other one will take care of remaining GPUs.

So if you get 4x 3090, that's 350W x4 = 1400W just for GPUs, +300 watts for CPU, +powering the rest of the components, drives, etc... So let's say we round that up to 2000W, then add at least 10% margin, that's 2200W total.

So maybe 1600W PSU for mobo and some GPUs, and another 1000W or more for the remaining GPUs. Note, if you go with 3090TI, it's more like 450-500W per card, so you have to do the maths.

Or if you want to go future proof, just put two 1600W PSUs, and then you can just swap your 3090 with 4090 in the future and not worry about upgrading PSUs.

Do you have some numbers or a link because all benchmarks I've seen point to the contrary? I'm happy to update my opinion if things have changed and there's data to support it.

Hmm, the Axi1600 might not be enough for me. This is my new build: https://ca.pcpartpicker.com/user/sixartsdragon/saved/DCh6Q7 and I'm looking at 1821W, so realistically I'm looking for a 2K PSU. I've chosen a Super Flower Leadex 2000 for now? What do you think?

this is likely the problem

Sorry, I referred explicitly to the the last paragraph of yours (that it's quick for small models)

The punchline is, if you need solid results, you can't generally risk the newer frameworks as you end-up debugging and/or not trusting the results (so you validate on other stacks....).

It's sort of close to a meritocracy + most people aren't using the capabilities of the existing stuff...

I also do this. Always verify that the model is actually able to learn “something”. After that you can go ahead and solve the task at hand. That way, you can be (more) sure about your model working correctly.