ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
  • NVIDIA Deep Learning Examples
    BERT PyTorch checkpoint (Large, Pretraining, AMP, LAMB)
    Model
    NVIDIA Deep Learning Examples
    BERT PyTorch checkpoint (Large, Pretraining, AMP, LAMB)

    BERT-Large PyTorch checkpoint pretrained with LAMB using AMP

  • 1 Version
    20.03.0_amp_optim-lambSelected
    09/22/2022 5:36 PM UTC5.13 GBAccuracy: 00 EpochsBatch Size: 0GPU: V100
  • architecture
    KeyValue
    typeLarge
    performance
    KeyValue
    training_loss1.33
    training
    KeyValue
    global_batch_size_phase232768
    global_batch_size_phase165536
    iterations_phase17038
    LR_phase20.004
    LR_phase10.006
    iterations_phase21563
    training_precisionAMP
    bs_phase28
    warmup_proportion_phase20.128
    bs_phase164
    warmup_proportion_phase10.2843
    iterations8601