Quick Start Guide for Data Center Platforms

NVIDIA Riva supports two architectures, Linux x86_64 and Linux ARM64. These are referred to as data center (x86_64) and embedded (ARM64). These instructions are applicable to data center users.

For more information and questions, visit the NVIDIA Riva Developer Forum.

Prerequisites

Before using Riva Speech AI, ensure you meet the following prerequisites:

You have access and are logged into NVIDIA NGC. For step-by-step instructions, refer to the NGC Getting Started Guide.
You have access to an NVIDIA Volta, NVIDIA Turing, or an NVIDIA Ampere architecture-based A100 GPU. For more information, refer to the Support Matrix.
You have Docker installed with support for NVIDIA GPUs. For more information, refer to the Support Matrix.

Models Available for Deployment

There are two push-button deployment options to deploy Riva Speech AI, which use pretrained models available from the NGC catalog:

Local Docker: You can use the Quick Start scripts to set up a local workstation and deploy the Riva services using Docker. Continue with this guide to use the Quick Start scripts.

Kubernetes: The Riva Helm Chart is designed to automate the steps for push-button deployment to a Kubernetes cluster. For more information, refer to Kubernetes deployment.

In addition to using pretrained models, Riva Speech AI can run with fine-tune custom models using NVIDIA NeMo. Refer to the {ref}nemo-development section for details regarding the advanced option to create a model repository with NVIDIA NeMo.

Getting Started with Riva for Data Center Platforms

Riva includes Quick Start scripts to help you get started with Riva Speech AI Skills. These scripts are meant for deploying the services locally, for testing, and running the example applications.

Download the Riva Quick Start scripts. You can either use the command-line interface or you can download the scripts directly from your browser. Click the Download drop-down button in the upper right corner and select:
- CLI - the download command is copied. Ensure you have the NGC CLI tool installed. Once installed, open the command prompt and paste the copied command to start your download.
- Browser (Direct Download) - the download begins in a location of your choosing.
Initialize and start Riva. The initialization step downloads and prepares Docker images and models. The start script launches the server.

:::{note} This process can take up to an hour on an average internet connection. On the data center, each model is individually optimized for the target GPU after download. :::

Optional: Modify the config.sh file within the quickstart directory with your preferred configuration. Options include:
- which services to enable
- which models to retrieve from NGC
- where to store them
- which GPU to use if more than one is installed on your system (refer to Local (Docker) for more details)
- locations of the SSL/TLS certificate
- key files if using a secure connection
In the example below, NMT and ASR are true, which enables the translation and ASR services. The TTS and NLP services are false, which disables these services.
```
# Enable or Disable Riva Services
service_enabled_tts=false
service_enabled_asr=true
service_enabled_nmt=true
service_enabled_nlp=false
```
The models are installed and configured if they are uncommented in config.sh and the corresponding service is enabled.

Continuing from the above example, in the NMT models section, the model for the English -> German language pair is uncommented, and is made available.
```
  "${riva_ngc_org}/${riva_ngc_team}/rmir_nmt_en_de_24x6:${riva_ngc_model_version}"
  #"${riva_ngc_org}/${riva_ngc_team}/rmir_nmt_en_es_24x6:${riva_ngc_model_version}"
  #"${riva_ngc_org}/${riva_ngc_team}/rmir_nmt_en_zh_24x6:${riva_ngc_model_version}"
  #"${riva_ngc_org}/${riva_ngc_team}/rmir_nmt_en_ru_24x6:${riva_ngc_model_version}"
```
```
cd riva_quickstart_v2.11.0
```
Initialize and start Riva
```
bash riva_init.sh
bash riva_start.sh
```
Try walking through the different tutorials on GitHub. If running the Riva Quick Start scripts on a cloud service provider (such as AWS or GCP), ensure that your compute instance has an externally visible IP address. To run the tutorials, connect a browser window to the correct port (8888 by default) of that external IP address.
Shut down the server when finished. After you've completed these steps and experimented with inferencing, run the riva_stop.sh script to stop the server.

For more information on how to customize a local deployment, refer to the Local (Docker) section.

Translate Text with Riva

From within the Riva container run the following commands:

Retrieve the available models and language pairs:

riva_nmt_client --list_models
languages {
  key: "en_de_24x6"
  value {
    src_lang: "en"
    tgt_lang: "de"
  }
}

Perform a translation from English to German, using the parameters from the list models RPC:

riva_nmt_client --model_name=en_de_24x6 --src_language="en" --tgt_language="de" --text="This will become German words."

Translate Speech-to-Speech with Riva (S2S)

From within the client container, run the following commands to perform an audio translation from Spanish audio to English audio, using the following parameters:

riva_streaming_s2s_client --audio_file=/opt/riva/wav/es-US_sample.wav --source_language_code="es-US" --target_language_code="en-US"

Translate Speech-to-Text with Riva (S2T)

From within the client container, run the following commands to perform an audio translation from English audio to German text, using the following parameters:

riva_streaming_speech_translate_client --audio_file=/opt/riva/wav/en-US_sample.wav --source_language_code="en-US" --target_language_code="de-DE"

Transcribe Audio Files with Riva

For Automatic Speech Recognition (ASR), run the following commands from inside the Riva client container to perform streaming and offline transcription of audio files. If using SSL/TLS, ensure to include the --ssl_server_cert /ssl/server.crt option.

Issue the riva_start_client.sh script to start the client container with sample clients for each service. The script is located in the Quick Start folder.
```
bash riva_start_client.sh
```

For offline recognition, run:

riva_asr_client --audio_file=/opt/riva/wav/en-US_sample.wav

For streaming recognition, run:

riva_streaming_asr_client --audio_file=/opt/riva/wav/en-US_sample.wav

Synthesize Speech with Riva

From within the Riva client container, run the following command to synthesize the audio files.

riva_tts_client --voice_name=English-US.Female-1 \
                --text="Hello, this is a speech synthesizer." \
                --audio_file=/opt/riva/wav/output.wav

The audio files are stored in the /opt/riva/wav directory.

The streaming API can be tested by using the command-line option --online=true. However, there is no difference between both options with the command-line client since it saves the entire audio to a .wav file.

Riva Collections

The Riva Collection contains the Riva Speech AI server, the Riva Speech AI client containers, the Riva Quick Start scripts resources, and the Riva Speech AI Skills Helm chart.

Additional Resources

For organizations looking to deploy Riva-based applications in production to unlimited workloads and get full NVIDIA support with direct access to NVIDIA AI experts globally, explore NVIDIA Riva or try NVIDIA Riva for free on NVIDIA LaunchPad.

License

By downloading and using Riva software, you accept the terms and conditions of this license.

Riva Skills Quick Start

Description

Publisher

Latest Version

Modified

Compressed Size