CUDA was a great technology before 2018.

Then in 2018, they prohibited usage of GeForce GPUs in data centers: https://www.datacenterdynamics.com/en/news/nvidia-updates-ge... And during 5 years which followed, they have reached the point where for many use cases nVidia is dramatically worse value compared to competitors.

On my day job I do CAM/CAE, we compute a lot of FP64 numbers. Because we use D3D tech for GPGPU, we recently bought some computers with AMD 7900 XTX. These GPUs were sold for about $1000. An nVidia equivalent is L40: the AMD is 1.459 TFlops and 0.96 TB/sec memory bandwidth, the nVidia is 1.414 TFlops and 0.864 TB/sec memory bandwidth, but the nVidia is sold for about $9000. That’s an order of magnitude difference in cost efficiency.

P_I_Staker

I agree the cost difference being substantial, but something seems off here. I think you can buy nVidia cores that can beat anything AMD has for ~$1500-2000. That is gaming focused, but it makes me extremely skeptical of the overall numbers.

If the AMD chip really is better for your job, that's not that crazy a claim, but it make no sense that nVidia wouldn't have something for 1.5-2x as much. This seems to be the going rate, currently.

If you're getting comparable speed for 1/9 the cost, I don't think that product would exist for very long. I've been looking at graphics cards and nVidia clearly beats AMD in every category with around a 50-100% markup.

Const-me

Gamers don’t care about FP64 performance, and it seems nVidia is using that for market segmentation. The FP64 performance for RTX 4090 is 1.142 TFlops, for RTX 3090 Ti 0.524 TFlops. AMD doesn’t do that, FP64 performance is consistently better there, and have been this way for quite a few years. For example, the figure for 3090 Ti (a $2000 card from 2022) is similar to Radeon Vega 56, a $400 card from 2017 which can do 0.518 TFlops.

And another thing: nVidia forbids usage of GeForce cards in data centers, while AMD allows that. I don’t know how specifically they define datacenter, whether it’s enforceable, or whether it’s tested in courts of various jurisdictions. I just don’t want to find out answers to these questions at the legal expenses of my employer. I believe they would prefer to not cut corners like that.

I think nVidia only beats AMD due to the ecosystem: for GPGPU that’s CUDA (and especially the included first-party libraries like BLAS, FFT, DNN and others), also due to the support in popular libraries like TensorFlow. However, it’s not that hard to ignore the ecosystem, and instead write some compute shaders in HLSL. Here’s a non-trivial open-source project unrelated to CAE, where I managed to do just that with decent results: https://github.com/Const-me/Whisper That software even works on Linux, probably due to Valve’s work on DXVK 2.0 (a compatibility layer which implements D3D11 on top of Vulkan).