ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
BERT PyTorch checkpoint (Large, Pretraining, AMP, LAMB)
Model
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
BERT PyTorch checkpoint (Large, Pretraining, AMP, LAMB)

BERT-Large PyTorch checkpoint pretrained with LAMB using AMP

1 Version
20.03.0_amp_optim-lambSelected
09/22/2022 5:36 PM UTC5.13 GBAccuracy: 00 EpochsBatch Size: 0GPU: V100
architecture
KeyValue
typeLarge
performance
KeyValue
training_loss1.33
training
KeyValue
global_batch_size_phase232768
global_batch_size_phase165536
iterations_phase17038
LR_phase20.004
LR_phase10.006
iterations_phase21563
training_precisionAMP
bs_phase28
warmup_proportion_phase20.128
bs_phase164
warmup_proportion_phase10.2843
iterations8601