The GNMT v2 model is an improved version of the first Google's Neural Machine Translation System with a modified attention mechanism.
Setup
The following section list the requirements in order to start training the GNMT model.
Requirements
This repository contains Dockerfile which extends the PyTorch NGC container
and encapsulates all dependencies.
For more information about how to get started with NGC containers, see the following sections from the NVIDIA GPU Cloud Documentation and the Deep Learning DGX Documentation: Getting Started Using NVIDIA GPU Cloud, Accessing And Pulling From The NGC container registry and Running PyTorch.
Training using mixed precision with Tensor Cores
Before you can train using mixed precision with Tensor Cores, ensure that you have a NVIDIA Volta based GPU. Other platforms might likely work but aren't officially supported. For information about how to train using mixed precision, see the Mixed Precision Training paper and Training With Mixed Precision documentation.
Another option for adding mixed-precision support is available from NVIDIA's APEX, A PyTorch Extension, that contains utility libraries, such as AMP, which require minimal network code changes to leverage Tensor Core performance.