NVIDIA V100 versus NVIDIA A40

Systems

NVIDIA DGX 1
8x NVIDIA V100 (16 GB)
2x Intel Xeon E5 2698 v4
512 GB RAM
DGX OS 5.0.5
NVIDIA Driver 470.82.01
Supermicro AS 4124GS TNR
8x NVIDIA A40 (40 GB)
2x AMD EPYC 7302
1 TB RAM
RHEL 7.9
NVIDIA Driver 460.73.01
nanoFluidX Software Stack
nanoFluidX 2022.0 with single precision floating point arithmetic
CUDA 11.1.1
Open MPI 4.0.6 (with CUDA support)

Results

Minimal Cube
Simple cube of static fluid particles in rest
Minimal case to estimate raw performance of solver core
Two sizes:
  • Small: ~7m fluid particles (slightly smaller than what is recommended for four GPU's)
  • Big: ~57m fluid particles (size of a bigger production)
All runs cover 1,000 timesteps


Figure 1.
Dambreak
Collapsing water column under gravity in domain (indicated by lines)
Good case to evaluate performance for tricky particle distributions and indicates whether load balancing works
Two sizes:
  • Small: ~7m fluid particles and ~9m total (slightly smaller than what is recommended for four GPU's)
  • Big: ~54m fluid particles and ~64m total (size of a bigger production case)
All runs cover 10,000 timesteps


Figure 2.
Altair E-Gearbox
Showcase by Altair for E-Mobility application
Size: ~6.5m fluid particles and ~12m total (slightly smaller than what is recommended for four GPU's)
Run covers 10,000 timesteps


Figure 3.
Aerospace Gearbox
Another showcase for aerospace gearbox applications
Chosen to have another slightly bigger case than previous one to fully benefit from four or more GPU's
Size: ~21m fluid particles and ~26.7m total
Run covers 10,000 timesteps


Figure 4.

Additional Notes

  • Performance data in the graphs is always relative to one V100 on the DGX-1.
  • All cases were run with the WEIGHTED particle interaction scheme.
  • All case were run in single precision.
  • All solver output has been deactivated to focus on solver performance, but generally this does not change the results significantly.
  • Scalability between one and two GPU's is usually slightly impaired because some parts related to multi-GPU may be skipped entirely in single GPU runs.