Skip to main content
ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
  • NVIDIA
    Transformer for PyTorch
    Resource
    NVIDIA
    Transformer for PyTorch

    This implementation of Transformer model architecture is based on the optimized implementation in Fairseq NLP toolkit.

  • Changelog

    June 2020

    • add TorchScript support
    • Ampere support

    March 2020

    • remove language modeling from the repository
    • one inference script for large chunks of data as well as for interactive demo
    • change custom distributed strategy to APEX's DDP
    • replace custom fp16 training with AMP
    • major refactoring of the codebase

    December 2019

    • Change evaluation metric

    August 2019

    • add basic AMP support

    July 2019

    • Replace custom fused operators with jit functions

    June 2019

    • New README

    March 2019

    • Add mid-training SacreBLEU evaluation. Better handling of OOMs.

    Initial commit, forked from fairseq