ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
BERT PyTorch checkpoint (Base, Pretraining, AMP, LAMB)
Model
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
BERT PyTorch checkpoint (Base, Pretraining, AMP, LAMB)

BERT Base PyTorch checkpoint pretrained with LAMB optimizer using AMP

1 Version
19.09.0_ampSelected
09/22/2022 5:33 PM UTC1.73 GBAccuracy: 00 EpochsBatch Size: 0GPU: V100
architecture
KeyValue
taskPretraining
typeBase
performance
KeyValue
training_loss1.39
training
KeyValue
global_batch_size_phase232768
global_batch_size_phase165536
iterations_phase17038
LR_phase20.004
LR_phase10.006
iterations_phase21563
training_precisionAMP
bs_phase28
warmup_proportion_phase20.128
bs_phase164
warmup_proportion_phase10.2843
iterations8601