ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
Bert Large checkpoint (TensorFlow2, Pretraining AMP, LAMB)
Model
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
Bert Large checkpoint (TensorFlow2, Pretraining AMP, LAMB)

Bert Large TensorFlow2 checkpoint pretrained using AMP and LAMB optimizer

1 Version
21.02.0_amp_optim-lambSelected
09/22/2022 5:34 PM UTC1.25 GBAccuracy: 00 EpochsBatch Size: 0GPU: A100
architecture
KeyValue
typelarge
performance
KeyValue
training_loss1.656
training
KeyValue
ITERATIONS_PHASE15774
GLOBAL_BATCH_SIZE_PHASE179872
LR_PHASE10.0065
GLOBAL_BATCH_SIZE_PHASE230720
training_precisionAMP
LR_PHASE20.004
BS_PHASE240
BS_PHASE1312
WARMUP_PHASE2200
datasetWikipedia;BookCorpus
ITERATIONS_PHASE21666
WARMUP_PHASE12000