ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
  • NVIDIA Deep Learning Examples
    BERT for PyTorch
    Resource
    NVIDIA Deep Learning Examples
    BERT for PyTorch

    BERT is a method of pre-training language representations which obtains state-of-the-art results on a wide array of NLP tasks.

  • Changelog

    January 2022

    • Knowledge Distillation support
    • Pre-training with native AMP, native DDP, and TorchScript with NVFuser backend
    • Pre-training using Language Datasets and Data Loaders (LDDL)
    • Binned pretraining for phase2 with LDDL using a bin size of 64

    July 2020

    • Updated accuracy and performance tables to include A100 results
    • Fine-tuning with the MRPC and SST-2 datasets.

    March 2020

    • TRITON Inference Server support.

    February 2020

    • Integrate DLLogger.

    November 2019

    • Use LAMB from APEX.
    • Code cleanup.
    • Bug fix in BertAdam optimizer.

    September 2019

    • Scripts to support a multi-node launch.
    • Update pre-training loss results based on the latest data preparation scripts.

    August 2019

    • Pre-training support with LAMB optimizer.
    • Updated Data download and Preprocessing.

    July 2019

    • Initial release.

    Known issues

    There are no known issues with this model.