Performance

Results

Training Accuracy Results

Results were obtained by running the train.py script with the default batch size = 128 per GPU in the pytorch-19.01-py3 Docker container.

NVIDIA DGX-1 (8x Tesla V100 16G)

Command used to launch the training:

python3 -m launch train.py --seed 2 --train-global-batch-size 1024

NVIDIA DGX-2 (16x Tesla V100 32G)

Commands used to launch the training:

for 1,4,8 GPUs:
python3 -m launch train.py --seed 2 --train-global-batch-size 1024
for 16 GPUs:
python3 -m launch train.py --seed 2 --train-global-batch-size 2048

Training Stability Test

The GNMT v2 model was trained for 6 epochs, starting from 50 different initial random seeds. After each training epoch the model was evaluated on the test dataset and the BLEU score was recorded. The training was performed in the pytorch-19.01-py3 Docker container on NVIDIA DGX-1 with 8 Tesla V100 16G GPUs. The following table summarizes results of the stability test.

BLEU scores after each training epoch for different initial random seeds

Training Performance Results

All results were obtained by running the train.py training script in the pytorch-19.01-py3 Docker container. Performance numbers (in tokens per second) were averaged over an entire training epoch.

NVIDIA DGX-1 (8x Tesla V100 16G)

NVIDIA DGX-2 (16x Tesla V100 32G)

Inference Performance Results

All results were obtained by running the translate.py script in the pytorch-19.01-py3 Docker container on NVIDIA DGX-1. Inference benchmark was run on a single Tesla V100 16G GPU. The benchmark requires a checkpoint from a fully trained model.

Command to launch the inference benchmark:

python3 translate.py --input data/wmt16_de_en/newstest2014.tok.bpe.32000.en \
  --reference data/wmt16_de_en/newstest2014.de --output /tmp/output \
  --model results/gnmt/model_best.pth --batch-size 32 128 512 \
  --beam-size 1 2 5 10 --math fp16 fp32