ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
Transformer for PyTorch
Resource
NVIDIA
NVIDIA
Transformer for PyTorch

This implementation of Transformer model architecture is based on the optimized implementation in Fairseq NLP toolkit.

Changelog

January 2019

  • initial commit, forked from fairseq

May 2019:

  • add mid-training SacreBLEU evaluation. Better handling of OOMs.

June 2019

  • new README

July 2019

  • replace custom fused operators with jit functions

August 2019

  • add basic AMP support

Known issues

  • Course of a training heavily depends on a random seed. There is high variance in the time required to reach a certain BLEU score. Also the highest BLEU score value observed vary between runs with different seeds.
  • Translations produced by training script during online evaluation may differ from those produced by generate.py script. It is probably a format conversion issue.