ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
BERT for PyTorch
Resource
NVIDIA Deep Learning Examples
NVIDIA Deep Learning Examples
BERT for PyTorch

BERT is a method of pre-training language representations which obtains state-of-the-art results on a wide array of NLP tasks.

Changelog

January 2022

  • Knowledge Distillation support
  • Pre-training with native AMP, native DDP, and TorchScript with NVFuser backend
  • Pre-training using Language Datasets and Data Loaders (LDDL)
  • Binned pretraining for phase2 with LDDL using a bin size of 64

July 2020

  • Updated accuracy and performance tables to include A100 results
  • Fine-tuning with the MRPC and SST-2 datasets.

March 2020

  • TRITON Inference Server support.

February 2020

  • Integrate DLLogger.

November 2019

  • Use LAMB from APEX.
  • Code cleanup.
  • Bug fix in BertAdam optimizer.

September 2019

  • Scripts to support a multi-node launch.
  • Update pre-training loss results based on the latest data preparation scripts.

August 2019

  • Pre-training support with LAMB optimizer.
  • Updated Data download and Preprocessing.

July 2019

  • Initial release.

Known issues

There are no known issues with this model.