The Variational Autoencoder for collaborative filtering focuses on providing recommendations. This is an optimized implementation.
The performance measurements in this document were conducted at the time of publication and may not reflect the performance achieved from NVIDIA's latest software release. For the most up-to-date performance measurements, go to NVIDIA Data Center Deep Learning Product Performance.
Benchmarking
The following section shows how to run benchmarks measuring the model performance in training and inference modes.
Training performance benchmark
To benchmark the training performance, run:
mpirun --bind-to numa --allow-run-as-root -np 8 -H localhost:8 python main.py --train [--amp]
Inference performance benchmark
To benchmark the inference performance, run:
python main.py --inference_benchmark [--amp]
Results
The following sections provide details on how we achieved our performance and accuracy in training and inference.
Training accuracy results
All training performance results were obtained by running:
mpirun --bind-to numa --allow-run-as-root -np <gpus> -H localhost:8 python main.py --train [--amp]
in the TensorFlow 20.06 NGC container.
Training accuracy: NVIDIA DGX A100 (8x A100 40GB)
| GPUs | Batch size / GPU | Accuracy - TF32 | Accuracy - mixed precision | Time to train - TF32 [s] | Time to train - mixed precision [s] | Time to train speedup (TF32 to mixed precision) |
|---|---|---|---|---|---|---|
| 1 | 24,576 | 0.430298 | 0.430398 | 112.8 | 109.4 | 1.03 |
| 8 | 3,072 | 0.430897 | 0.430353 | 25.9 | 30.4 | 0.85 |
Training accuracy: NVIDIA DGX-1 (8x V100 32GB)
| GPUs | Batch size / GPU | Accuracy - FP32 | Accuracy - mixed precision | Time to train - FP32 [s] | Time to train - mixed precision [s] | Time to train speedup (FP32 to mixed precision) |
|---|---|---|---|---|---|---|
| 1 | 24,576 | 0.430592 | 0.430525 | 346.5 | 186.5 | 1.86 |
| 8 | 3,072 | 0.430753 | 0.431202 | 59.1 | 42.2 | 1.40 |
Training performance results
Performance numbers below show throughput in users processed per second. They were averaged over an entire training run.
Training performance: NVIDIA DGX A100 (8x A100 40GB)
| GPUs | Batch size / GPU | Throughput - TF32 | Throughput - mixed precision | Throughput speedup (TF32 - mixed precision) | Strong scaling - TF32 | Strong scaling - mixed precision |
|---|---|---|---|---|---|---|
| 1 | 24,576 | 354,032 | 365,474 | 1.03 | 1 | 1 |
| 8 | 3,072 | 1,660,700 | 1,409,770 | 0.85 | 4.69 | 3.86 |
Training performance: NVIDIA DGX-1 (8x V100 32GB)
| GPUs | Batch size / GPU | Throughput - FP32 | Throughput - mixed precision | Throughput speedup (FP32 - mixed precision) | Strong scaling - FP32 | Strong scaling - mixed precision |
|---|---|---|---|---|---|---|
| 1 | 24,576 | 114,125 | 213,283 | 1.87 | 1 | 1 |
| 8 | 3,072 | 697,628 | 1,001,210 | 1.44 | 6.11 | 4.69 |
Inference performance results
Our results were obtained by running:
python main.py --inference_benchmark [--amp]
in the TensorFlow 20.06 NGC container.
We use users processed per second as a throughput metric for measuring inference performance. All latency numbers are in seconds.
Inference performance: NVIDIA DGX A100 (1x A100 40GB)
TF32
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1 | 1181 | 0.000847 | 0.000863 | 0.000871 | 0.000901 |
FP16
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1 | 1215 | 0.000823 | 0.000858 | 0.000864 | 0.000877 |
Inference performance: NVIDIA DGX-1 (1x V100 16GB)
FP32
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1 | 718 | 0.001392 | 0.001443 | 0.001458 | 0.001499 |
FP16
| Batch size | Throughput Avg | Latency Avg | Latency 90% | Latency 95% | Latency 99% |
|---|---|---|---|---|---|
| 1 | 707 | 0.001413 | 0.001511 | 0.001543 | 0.001622 |