NVIDIA Deep Learning Examples
Use the NGC CLI to download:
Copied!
Model
NVIDIA Deep Learning Examples
BERT PyTorch checkpoint (Dist-6L-768D, SQuAD1, AMP)BERT Distilled 6L-768D PyTorch checkpoint distilled on SQuAD v1.1 dataset using AMP
Use the NGC CLI to download:
Copied!
1 Version
20.12.0_ampSelected07/07/2022 2:27 PM UTC253.45 MBAccuracy: 0.88490 EpochsBatch Size: 0GPU: V100Copied!
20.12.0_ampSelected
07/07/2022 2:27 PM UTC253.45 MBAccuracy: 0.88490 EpochsBatch Size: 0GPU: V100
Copied!
architecture
| Key | Value |
|---|---|
| attn_heads | 12 |
| task | SQuAD v1.1 |
| hidden_size | 768 |
| di | 3072 |
| layers | 6 |
| type | Dist-6L-768D |
performance
| Key | Value |
|---|---|
| accuracy | 0.8849 |
training
| Key | Value |
|---|---|
| training_precision | AMP |
| dataset | SQuAD v1.1 |