Performance

Benchmarking

The following section shows how to run benchmarks measuring the model performance in training and inference modes.

Training performance benchmark

Training benchmark was run in various scenarios on V100 16G GPU. For each scenario, batch size was set to 32. The benchmark does not require a checkpoint from a fully trained model.

To benchmark training, run:

python -m torch.distributed.launch --nproc_per_node={NGPU} \
       main.py --batch-size {bs} \
               --mode benchmark-training \
               --benchmark-warmup 100 \
               --benchmark-iterations 200 \
               {fp16} \
               --data {data}

Where the {NGPU} selects number of GPUs used in benchmark, the {bs} is the desired batch size, the {fp16} is set to --fp16 if you want to benchmark training with tensor cores, and the {data} is the location of the COCO 2017 dataset.

Benchmark warmup is specified to omit first iterations of first epoch. Benchmark iterations is number of iterations used to measure performance.

Inference performance benchmark

Inference benchmark was run on 1x V100 16G GPU. To benchmark inference, run:

python main.py --eval-batch-size {bs} \
               --mode benchmark-inference \
               --benchmark-warmup 100 \
               --benchmark-iterations 200 \
               {fp16} \
               --data {data}

Where the {bs} is the desired batch size, the {fp16} is set to --fp16 if you want to benchmark inference with Tensor Cores, and the {data} is the location of the COCO 2017 dataset.

Benchmark warmup is specified to omit first iterations of first epoch. Benchmark iterations is number of iterations used to measure performance.

Results

The following sections provide details on how we achieved our performance and accuracy in training and inference.

Training accuracy results

Our results were obtained by running the ./examples/SSD300_FP{16,32}_{1,4,8}GPU.sh script in the pytorch-19.03-py3 NGC container on NVIDIA DGX-1 with 8x V100 16G GPUs. Batch was set to size best utilizing GPU memory. For FP32 precision, batch size is 32, for mixed precision batch size is 64

Here are example graphs of FP32 and FP16 training on 8 GPU configuration:

Training performance results

Our results were obtained by running the main.py script with the --mode benchmark-training flag in the pytorch-19.03-py3 NGC container on NVIDIA DGX-1 with V100 16G GPUs.

To achieve same results, follow the Quick start guide outlined above.

Inference performance results

Our results were obtained by running the main.py script with --mode benchmark-inference flag in the pytorch-19.03-py3 NGC container on NVIDIA DGX-1 with 1x V100 16G GPUs.

To achieve same results, follow the Quick start guide outlined above.