QuartzNet for PyTorch

NVIDIA Deep Learning Examples

Resource

NVIDIA Deep Learning Examples

QuartzNet for PyTorch

End-to-end neural acoustic model for automatic speech recognition providing high accuracy at a low memory footprint.

To train your model using mixed or TF32 precision with Tensor Cores or using FP32, perform the following steps using the default parameters of the QuartzNet model on the LibriSpeech dataset. For the specifics concerning training and inference, see the Advanced section.

Clone the repository.

git clone https://github.com/NVIDIA/DeepLearningExamples
cd DeepLearningExamples/PyTorch/SpeechRecognition/QuartzNet

Build the QuartzNet PyTorch NGC container.
```
bash scripts/docker/build.sh
```
Start an interactive session in the NGC container to prepare the dataset, or run training/inference. Specify a local mountpoint for the dataset with the DATA_DIR variable:
```
DATA_DIR=<path_on_the_host> bash scripts/docker/launch.sh
```
Download and preprocess the dataset. No GPU is required for data download and preprocessing. It can take several hours to complete, and requires over 250GB of free disk space.

This repository provides scripts to download and extract LibriSpeech http://www.openslr.org/12. The dataset contains 1000 hours of 16kHz read English speech derived from public domain audiobooks from the LibriVox project and has been carefully segmented and aligned. For more information, see the LIBRISPEECH: AN ASR CORPUS BASED ON PUBLIC DOMAIN AUDIO BOOKS paper.

Inside the container, download and extract the datasets into the required format for later training and inference:
```
bash scripts/download_librispeech.sh
```
After the data download is complete, the following folders should exist:
```
datasets/LibriSpeech/
|-- dev-clean
|-- dev-other
|-- test-clean
|-- test-other
|-- train-clean-100
|-- train-clean-360
|-- train-other-500
```
Since /datasets/ is mounted to DATA_DIR on the host, after the dataset is downloaded it will be accessible from outside of the container at $DATA_DIR/LibriSpeech.

Next, convert the data into WAV files:
```
bash scripts/preprocess_librispeech.sh
```
After the data is converted, the following additional files and folders should exist:
```
datasets/LibriSpeech/
|-- dev-clean-wav
|-- dev-other-wav
|-- librispeech-train-clean-100-wav.json
|-- librispeech-train-clean-360-wav.json
|-- librispeech-train-other-500-wav.json
|-- librispeech-dev-clean-wav.json
|-- librispeech-dev-other-wav.json
|-- librispeech-test-clean-wav.json
|-- librispeech-test-other-wav.json
|-- test-clean-wav
|-- test-other-wav
|-- train-clean-100-wav
|-- train-clean-360-wav
|-- train-other-500-wav
```
Start training. Inside the container, use the following script to start training. Make sure the downloaded and preprocessed dataset is located at $DATA_DIR/LibriSpeech on the host, which is mounted as /datasets/LibriSpeech inside the container.
```
[OPTION1=value1 OPTION2=value2 ...] bash scripts/train.sh
```
By default, automatic precision is disabled, batch size is 144 over two gradient accumulation steps, and the recipe is run on a total of 8 GPUs. The hyperparameters are tuned for a GPU with at least 32GB of memory and will require adjustment for different configurations (for example, by lowering the batch size and using more gradient accumulation steps).

Options are being passed as environment variables. More details on the available options can be found in the Parameters and Training process sections.
Start validation/evaluation. Inside the container, use the following script to run evaluation. Make sure the downloaded and preprocessed dataset is located at $DATA_DIR/LibriSpeech on the host, which is mounted as /datasets/LibriSpeech inside the container.
```
[OPTION1=value1 OPTION2=value2 ...] bash scripts/evaluation.sh [OPTIONS]
```
By default, this will use full precision, a batch size of 64, and run on a single GPU.

Options are being passed as environment variables. More details on the available options can be found in the Parameters and Evaluation process sections.
Start inference/predictions. Inside the container, use the following script to run inference. Make sure the downloaded and preprocessed dataset is located at $DATA_DIR/LibriSpeech on the host, which is mounted as /datasets/LibriSpeech inside the container. A pretrained model checkpoint can be downloaded from NGC model repository, manually or automatically using scripts/download_quartznet.sh.
```
[OPTION1=value1 OPTION2=value2 ...] bash scripts/inference.sh
```
By default, this will use single precision, a batch size of 64, and run on a single GPU.

Options are being passed as environment variables. More details on the available options can be found in the Parameters and Inference process sections.

Now that you have your model trained and evaluated, you can choose to compare your training results with our Training accuracy results. You can also choose to benchmark your performance to Training performance benchmark, or Inference performance benchmark. Following the steps in these sections will ensure that you achieve the same accuracy and performance results as stated in the Results section.