This model can be used for translating text in source language (En) to a text in target language (De/Es/Fr).
The model is based on Transformer "Big" architecture originally presented in "Attention Is All You Need" paper [1]. It is using SentencePiece tokenizer [2].
These models were trained on a collection of many publicly available datasets comprising of millions of parallel sentences. The NeMo toolkit [5] was used for training this model over roughly 700k steps.
While training this model, we used the following datasets:
German
Spanish
French
We used the SentencePiece tokenizer [2] with shared encoder and decoder BPE tokenizers.
The accuracy of translation models are often measured using BLEU scores [3]. The model achieves the following sacreBLEU [4] scores on WMT test sets
De
WMT13 - 28.4
WMT14 - 29.7
Es
WMT12 - 39.8
WMT13 - 35.8
Fr
WMT13 - 34.7
WMT14 - 40.8
The model is available for use in the NeMo toolkit [5], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.
import nemo
import nemo.collections.nlp as nemo_nlp
nmt_model = nemo_nlp.models.machine_translation.MTEncDecModel.from_pretrained(model_name="mnmt_en_deesfr_transformer6x6")
python [NEMO_GIT_FOLDER]/examples/nlp/machine_translation/nmt_transformer_infer.py --model=mnmt_en_deesfr_transformer6x6.nemo --srctext=[TEXT_IN_SRC_LANGUAGE] --tgtout=[WHERE_TO_SAVE_TRANSLATIONS] --target_lang [TARGET_LANGUAGE] --source_lang en
where [TARGET_LANGUAGE] can be 'de' or 'es' or 'fr'
This translate method of the NMT model accepts a list of de-tokenized strings.
The translate method outputs a list of de-tokenized strings in the target language.
No known limitations at this time.
[1] Vaswani, Ashish, et al. "Attention is all you need." arXiv preprint arXiv:1706.03762 (2017).
[2] https://github.com/google/sentencepiece
[3] https://en.wikipedia.org/wiki/BLEU
[4] https://github.com/mjpost/sacreBLEU
License to use this model is covered by the NGC TERMS OF USE unless another License/Terms Of Use/EULA is clearly specified. By downloading the public and release version of the model, you accept the terms and conditions of the NGC TERMS OF USE.