This model is a convolutional neural network for 2D image segmentation tuned to avoid overfitting.
The performance measurements in this document were conducted at the time of publication and may not reflect the performance achieved from NVIDIA's latest software release. For the most up-to-date performance measurements, go to NVIDIA Data Center Deep Learning Product Performance.
Benchmarking
The following sections shows how to run benchmarks measuring the model performance in training and inference modes.
Training performance benchmark
To benchmark the inference performance, you can run one of the scripts in the ./scripts/benchmarking/ directory
called ./scripts/benchmarking/UNet_trainbench{_AMP}_{1, 4, 8}GPU.sh.
Each of the scripts requires three parameters:
- path to the dataset as the first argument
- class ID from DAGM used (between 1-10)
For example:
cd scripts/benchmarking/
./UNet_trainbench_1GPU.sh /data 1
Inference performance benchmark
To benchmark the training performance, you can run one of the scripts in the ./scripts/benchmarking/ directory
called ./scripts/benchmarking/UNet_evalbench{_AMP}.sh.
Each of the scripts requires three parameters:
- path to the dataset as the first argument
- class ID from DAGM used (between 1-10)
For example:
cd scripts/benchmarking/
./UNet_evalbench_AMP.sh /data 1
Results
The following sections provide details on the achieved results in training accuracy, performance and inference performance.
Training accuracy results
Training accuracy: NVIDIA DGX A100 (8x A100 40GB)
Our results were obtained by running the ./scripts/UNet{_AMP}_{1, 8}GPU.sh training
script in the tensorflow:20.06-tf1-py3 NGC container on NVIDIA DGX A100 (8x A100 40GB) GPUs.
| GPUs | Batch size / GPU | Accuracy - TF32 | Accuracy - mixed precision | Time to train - TF32 [min] | Time to train - mixed precision [min] | Time to train speedup (TF32 to mixed precision) |
|---|---|---|---|---|---|---|
| 1 | 16 | 0.9717 | 0.9726 | 3.6 | 2.3 | 1.57 |
| 8 | 2 | 0.9733 | 0.9683 | 4.3 | 3.5 | 1.23 |
Training accuracy: NVIDIA DGX-1 (8x V100 16GB)
Our results were obtained by running the ./scripts/UNet{_AMP}_{1, 8}GPU.sh training
script in the tensorflow:20.06-tf1-py3 NGC container on NVIDIA DGX-1 (8x V100 16GB) GPUs.
| GPUs | Batch size / GPU | Accuracy - FP32 | Accuracy - mixed precision | Time to train - FP32 [min] | Time to train - mixed precision [min] | Time to train speedup (FP32 to mixed precision) |
|---|---|---|---|---|---|---|
| 1 | 16 | 0.9643 | 0.9653 | 10 | 8 | 1.25 |
| 8 | 2 | 0.9637 | 0.9655 | 2.5 | 2.5 | 1.00 |
Training performance results
Training performance: NVIDIA DGX A100 (8x A100 40GB)
Our results were obtained by running the scripts
./scripts/benchmarking/UNet_trainbench{_AMP}_{1, 4, 8}GPU.sh training script in the
TensorFlow 20.06-tf1-py3 NGC container on NVIDIA DGX A100 (8x A100 40GB) GPUs.
| GPUs | Batch size / GPU | Throughput - TF32 [img/s] | Throughput - mixed precision [img/s] | Throughput speedup (TF32 - mixed precision) | Strong scaling - TF32 | Strong scaling - mixed precision |
|---|---|---|---|---|---|---|
| 1 | 16 | 135.95 | 255.26 | 1.88 | - | - |
| 4 | 4 | 420.2 | 691.19 | 1.64 | 3.09 | 2.71 |
| 8 | 2 | 655.05 | 665.66 | 1.02 | 4.82 | 2.61 |
Training performance: NVIDIA DGX-1 (8x V100 16GB)
Our results were obtained by running the scripts
./scripts/benchmarking/UNet_trainbench{_AMP}_{1, 4, 8}GPU.sh training script in the
TensorFlow 20.06-tf1-py3 NGC container on an NVIDIA DGX-1 (8 V100 16GB) GPUs.
| GPUs | Batch size / GPU | Throughput - FP32 [img/s] | Throughput - mixed precision [img/s] | Throughput speedup (FP32 - mixed precision) | Strong scaling - FP32 | Strong scaling - mixed precision |
|---|---|---|---|---|---|---|
| 1 | 16 | 86.95 | 168.54 | 1.94 | - | - |
| 4 | 4 | 287.01 | 459.07 | 1.60 | 3.30 | 2.72 |
| 8 | 2 | 474.77 | 444.13 | 0.94 | 5.46 | 2.64 |
To achieve these same results, follow the Quick Start Guide outlined above.
Inference performance results
Inference performance results
Inference performance: NVIDIA DGX A100 (1x A100 40GB)
Our results were obtained by running the scripts ./scripts/benchmarking/UNet_evalbench{_AMP}.sh
evaluation script in the 20.06-tf1-py3 NGC container on NVIDIA DGX A100 (1x A100 40GB) GPUs.
FP16
| Batch size | Resolution | Throughput Avg [img/s] |
|---|---|---|
| 1 | 512x512x1 | 247.83 |
| 8 | 512x512x1 | 761.41 |
| 16 | 512x512x1 | 823.46 |
TF32
| Batch size | Resolution | Throughput Avg [img/s] |
|---|---|---|
| 1 | 512x512x1 | 227.97 |
| 8 | 512x512x1 | 419.70 |
| 16 | 512x512x1 | 424.57 |
To achieve these same results, follow the steps in the Quick Start Guide.
Inference performance: NVIDIA DGX-1 (1x V100 16GB)
Our results were obtained by running the scripts ./scripts/benchmarking/UNet_evalbench{_AMP}.sh
evaluation script in the 20.06-tf1-py3 NGC container on NVIDIA DGX-1 (1x V100 16GB) GPUs.
FP16
| Batch size | Resolution | Throughput Avg [img/s] |
|---|---|---|
| 1 | 512x512x1 | 157.91 |
| 8 | 512x512x1 | 438.00 |
| 16 | 512x512x1 | 469.27 |
FP32
| Batch size | Resolution | Throughput Avg [img/s] |
|---|---|---|
| 1 | 512x512x1 | 159.65 |
| 8 | 512x512x1 | 243.99 |
| 16 | 512x512x1 | 250.23 |
To achieve these same results, follow the Quick Start Guide outlined above.