The NCF model focuses on providing recommendations. This is a modified implementation with improved overfitting and better accuracy.
The performance measurements in this document were conducted at the time of publication and may not reflect the performance achieved from NVIDIA's latest software release. For the most up-to-date performance measurements, go to NVIDIA Data Center Deep Learning Product Performance.
Benchmarking
The following section shows how to run benchmarks measuring the model performance in training and inference modes.
Performance Benchmark
To benchmark the training and inference performance, run:
mpirun -np 1 --allow-run-as-root python ncf.py --data /data/cache/ml-20m
By default, the ncf.py script outputs metrics describing the following:
- Training speed and throughput
- Evaluation speed and throughput
Results
The following sections provide details on how we achieved our performance and accuracy in training and inference.
All throughput numbers are reported in millions of samples per second while time-to-train numbers are in seconds.
Training accuracy results
For all the sections below, our results were obtained by running:
mpirun -np <number_of_GPUs> --allow-run-as-root python ncf.py [--amp] --data /data/cache/ml-20m
in the TensorFlow-1 20.07 NGC container.
Training accuracy: NVIDIA DGX A100 (8x A100 40GB)
| GPUs | Batch size / GPU | Accuracy - TF32 | Accuracy - mixed precision | Time to train - TF32 [s] | Time to train - mixed precision [s] | Time to train speedup (TF32 to mixed precision) |
|---|---|---|---|---|---|---|
| 1 | 1,048,576 | 0.9588 | 0.9589 | 59.4 | 53.1 | 1.12 |
| 4 | 262,144 | 0.9588 | 0.9590 | 22.8 | 21.5 | 1.06 |
| 8 | 131,072 | 0.9587 | 0.9589 | 19.8 | 20.2 | 0.98 |
Training accuracy: NVIDIA DGX-1 (8x V100 32GB)
| GPUs | Batch size / GPU | Accuracy - FP32 | Accuracy - mixed precision | Time to train - FP32 | Time to train - mixed precision | Time to train speedup (FP32 to mixed precision) |
|---|---|---|---|---|---|---|
| 1 | 1,048,576 | 0.9583 | 0.9589 | 120.9 | 91.6 | 1.32 |
| 4 | 262,144 | 0.9589 | 0.9583 | 43.7 | 31.8 | 1.37 |
| 8 | 131,072 | 0.9590 | 0.9588 | 26.2 | 21.9 | 1.20 |
Training Performance Results
For all the sections below, our results were obtained by running:
mpirun -np <number_of_GPUs> --allow-run-as-root python ncf.py [--amp] --data /data/cache/ml-20m
in the TensorFlow-1 20.07 NGC container.
Training performance: NVIDIA DGX A100 (8x A100 40GB)
| GPUs | Batch size / GPU | Throughput - TF32 | Throughput - mixed precision | Throughput speedup (TF32 - mixed precision) | Strong scaling - TF32 | Strong scaling - mixed precision |
|---|---|---|---|---|---|---|
| 1 | 1,048,576 | 20.18 | 22.84 | 1.132 | 1 | 1 |
| 4 | 262,144 | 60.34 | 62.70 | 1.039 | 2.99 | 2.75 |
| 8 | 131,072 | 89.88 | 80.86 | 0.900 | 4.45 | 3.54 |
Training performance: NVIDIA DGX-1 (8x V100 32GB)
| GPUs | Batch size / GPU | Throughput - FP32 | Throughput - mixed precision | Throughput speedup (FP32 - mixed precision) | Strong scaling - FP32 | Strong scaling - mixed precision |
|---|---|---|---|---|---|---|
| 1 | 1,048,576 | 9.73 | 15.21 | 1.563 | 1 | 1 |
| 4 | 262,144 | 30.31 | 39.47 | 1.302 | 3.11 | 2.60 |
| 8 | 131,072 | 50.91 | 59.13 | 1.161 | 5.23 | 3.89 |
Inference Performance Results
Our results were obtained by running the inference.py script in the PyTorch 20.07 NGC container.
Throughput is reported in millions of samples per second while latency is reported in seconds.
Inference performance: NVIDIA DGX A100 (1x A100 40GB)
TF32
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1,024 | 1.67 | 0.0006 | 0.0006 | 0.0007 | 0.0007 |
| 4,096 | 6.02 | 0.0007 | 0.0007 | 0.0007 | 0.0007 |
| 16,384 | 19.01 | 0.0009 | 0.0009 | 0.0009 | 0.0009 |
| 65,536 | 34.91 | 0.0019 | 0.0019 | 0.0019 | 0.0019 |
| 262,144 | 44.72 | 0.0059 | 0.0063 | 0.0063 | 0.0066 |
| 1,048,576 | 47.22 | 0.0222 | 0.0230 | 0.0232 | 0.0237 |
FP16
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1,024 | 1.34 | 0.0008 | 0.0008 | 0.0008 | 0.0008 |
| 4,096 | 5.23 | 0.0008 | 0.0008 | 0.0008 | 0.0008 |
| 16,384 | 17.61 | 0.0009 | 0.0009 | 0.0010 | 0.0010 |
| 65,536 | 38.63 | 0.0017 | 0.0017 | 0.0018 | 0.0018 |
| 262,144 | 55.36 | 0.0047 | 0.0049 | 0.0050 | 0.0051 |
| 1,048,576 | 59.48 | 0.0176 | 0.0178 | 0.0179 | 0.0184 |
Inference performance: NVIDIA DGX-1 (8x V100 32GB)
FP32
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1,024 | 0.79 | 0.0013 | 0.0015 | 0.0015 | 0.0016 |
| 4,096 | 2.88 | 0.0014 | 0.0016 | 0.0016 | 0.0017 |
| 16,384 | 8.38 | 0.0020 | 0.0021 | 0.0021 | 0.0024 |
| 65,536 | 16.77 | 0.0039 | 0.0041 | 0.0041 | 0.0041 |
| 262,144 | 22.53 | 0.0116 | 0.0118 | 0.0119 | 0.0122 |
| 1,048,576 | 25.14 | 0.0417 | 0.0425 | 0.0431 | 0.0440 |
FP16
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1,024 | 0.69 | 0.0015 | 0.0017 | 0.0017 | 0.0018 |
| 4,096 | 2.64 | 0.0016 | 0.0017 | 0.0017 | 0.0018 |
| 16,384 | 8.84 | 0.0019 | 0.0020 | 0.0020 | 0.0021 |
| 65,536 | 21.43 | 0.0031 | 0.0032 | 0.0032 | 0.0032 |
| 262,144 | 33.61 | 0.0078 | 0.0080 | 0.0081 | 0.0083 |
| 1,048,576 | 38.83 | 0.0270 | 0.0276 | 0.0277 | 0.0286 |