NCF for TensorFlow1 | NVIDIA NGC

NVIDIA Deep Learning Examples

NCF for TensorFlow1

Resource

NVIDIA Deep Learning Examples

NCF for TensorFlow1

The NCF model focuses on providing recommendations. This is a modified implementation with improved overfitting and better accuracy.

The performance measurements in this document were conducted at the time of publication and may not reflect the performance achieved from NVIDIA's latest software release. For the most up-to-date performance measurements, go to NVIDIA Data Center Deep Learning Product Performance.

Benchmarking

The following section shows how to run benchmarks measuring the model performance in training and inference modes.

Performance Benchmark

To benchmark the training and inference performance, run:

mpirun -np 1 --allow-run-as-root python ncf.py --data /data/cache/ml-20m

By default, the ncf.py script outputs metrics describing the following:

Training speed and throughput
Evaluation speed and throughput

Results

The following sections provide details on how we achieved our performance and accuracy in training and inference.

All throughput numbers are reported in millions of samples per second while time-to-train numbers are in seconds.

Training accuracy results

For all the sections below, our results were obtained by running:

mpirun -np <number_of_GPUs> --allow-run-as-root python ncf.py [--amp] --data /data/cache/ml-20m

in the TensorFlow-1 20.07 NGC container.

Training accuracy: NVIDIA DGX A100 (8x A100 40GB)

GPUs	Batch size / GPU	Accuracy - TF32	Accuracy - mixed precision	Time to train - TF32 [s]	Time to train - mixed precision [s]	Time to train speedup (TF32 to mixed precision)
1	1,048,576	0.9588	0.9589	59.4	53.1	1.12
4	262,144	0.9588	0.9590	22.8	21.5	1.06
8	131,072	0.9587	0.9589	19.8	20.2	0.98

Training accuracy: NVIDIA DGX-1 (8x V100 32GB)

GPUs	Batch size / GPU	Accuracy - FP32	Accuracy - mixed precision	Time to train - FP32	Time to train - mixed precision	Time to train speedup (FP32 to mixed precision)
1	1,048,576	0.9583	0.9589	120.9	91.6	1.32
4	262,144	0.9589	0.9583	43.7	31.8	1.37
8	131,072	0.9590	0.9588	26.2	21.9	1.20

Training Performance Results

For all the sections below, our results were obtained by running:

mpirun -np <number_of_GPUs> --allow-run-as-root python ncf.py [--amp] --data /data/cache/ml-20m

in the TensorFlow-1 20.07 NGC container.

Training performance: NVIDIA DGX A100 (8x A100 40GB)

GPUs	Batch size / GPU	Throughput - TF32	Throughput - mixed precision	Throughput speedup (TF32 - mixed precision)	Strong scaling - TF32	Strong scaling - mixed precision
1	1,048,576	20.18	22.84	1.132	1	1
4	262,144	60.34	62.70	1.039	2.99	2.75
8	131,072	89.88	80.86	0.900	4.45	3.54

Training performance: NVIDIA DGX-1 (8x V100 32GB)

GPUs	Batch size / GPU	Throughput - FP32	Throughput - mixed precision	Throughput speedup (FP32 - mixed precision)	Strong scaling - FP32	Strong scaling - mixed precision
1	1,048,576	9.73	15.21	1.563	1	1
4	262,144	30.31	39.47	1.302	3.11	2.60
8	131,072	50.91	59.13	1.161	5.23	3.89

Inference Performance Results

Our results were obtained by running the inference.py script in the PyTorch 20.07 NGC container.

Throughput is reported in millions of samples per second while latency is reported in seconds.

Inference performance: NVIDIA DGX A100 (1x A100 40GB)

TF32

Batch size	Throughput Avg	Latency Avg	Latency 90%	Latency 95%	Latency 99%
1,024	1.67	0.0006	0.0006	0.0007	0.0007
4,096	6.02	0.0007	0.0007	0.0007	0.0007
16,384	19.01	0.0009	0.0009	0.0009	0.0009
65,536	34.91	0.0019	0.0019	0.0019	0.0019
262,144	44.72	0.0059	0.0063	0.0063	0.0066
1,048,576	47.22	0.0222	0.0230	0.0232	0.0237

FP16

Batch size	Throughput Avg	Latency Avg	Latency 90%	Latency 95%	Latency 99%
1,024	1.34	0.0008	0.0008	0.0008	0.0008
4,096	5.23	0.0008	0.0008	0.0008	0.0008
16,384	17.61	0.0009	0.0009	0.0010	0.0010
65,536	38.63	0.0017	0.0017	0.0018	0.0018
262,144	55.36	0.0047	0.0049	0.0050	0.0051
1,048,576	59.48	0.0176	0.0178	0.0179	0.0184

Inference performance: NVIDIA DGX-1 (8x V100 32GB)

FP32

Batch size	Throughput Avg	Latency Avg	Latency 90%	Latency 95%	Latency 99%
1,024	0.79	0.0013	0.0015	0.0015	0.0016
4,096	2.88	0.0014	0.0016	0.0016	0.0017
16,384	8.38	0.0020	0.0021	0.0021	0.0024
65,536	16.77	0.0039	0.0041	0.0041	0.0041
262,144	22.53	0.0116	0.0118	0.0119	0.0122
1,048,576	25.14	0.0417	0.0425	0.0431	0.0440

FP16

Batch size	Throughput Avg	Latency Avg	Latency 90%	Latency 95%	Latency 99%
1,024	0.69	0.0015	0.0017	0.0017	0.0018
4,096	2.64	0.0016	0.0017	0.0017	0.0018
16,384	8.84	0.0019	0.0020	0.0020	0.0021
65,536	21.43	0.0031	0.0032	0.0032	0.0032
262,144	33.61	0.0078	0.0080	0.0081	0.0083
1,048,576	38.83	0.0270	0.0276	0.0277	0.0286