# Quick Start Guide for Embedded Platforms

NVIDIA Riva Speech AI Skills supports two architectures, Linux x86_64 and Linux ARM64. These are referred to as **data center** (x86_64) and **embedded** (ARM64). These instructions are applicable to embedded users.

For more information and questions, visit the [NVIDIA Riva Developer Forum](https://forums.developer.nvidia.com/c/ai-data-science/deep-learning/riva/475).

:::{note}
Riva embedded (ARM64) is in [public beta](https://developer.nvidia.com/riva/license).
:::


## Prerequisites

Before using Riva Speech AI, ensure you meet the following prerequisites:

1. You have access and are [logged into](https://ngc.nvidia.com/signin) NVIDIA NGC. For step-by-step instructions, refer to the [NGC Getting Started Guide](https://docs.nvidia.com/ngc/ngc-overview/index.html#registering-activating-ngc-account).

2. You have access to an NVIDIA Jetson Orin, NVIDIA Jetson AGX Xavier, or NVIDIA Jetson NX Xavier. For more information, refer to the [Support Matrix](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/support-matrix.html).

3. You have installed NVIDIA JetPack version 5.1 or 5.1.1 on the Jetson platform. For more information, refer to the [Support Matrix](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/support-matrix.html).

4. You have ~15 GB free disk space on Jetson as required by the default containers and models. If you are deploying your custom Riva model intermediate representation (RMIR) models, the additional disk space required is ~14 GB plus the size of custom RMIR models.

5. You have enabled the following power modes on the Jetson platform. These modes activate all CPU cores and clock the CPU/GPU at maximum frequency for achieving best performance.

    ```bash
    sudo nvpmodel -m 0 (Jetson Orin AGX, mode MAXN)
    sudo nvpmodel -m 0 (Jetson Xavier AGX, mode MAXN)
    sudo nvpmodel -m 2 (Jetson Xavier NX, mode MODE_15W_6CORE)
    ```

6. You have set the default runtime to `nvidia` on the Jetson platform by adding the following line in the `/etc/docker/daemon.json` file. Restart the Docker service using `sudo systemctl restart docker` after editing the file.

    ```bash
   "default-runtime": "nvidia"
   ```

## Models Available for Deployment

There is a push-button deployment option to deploy Riva Speech AI, which uses [pretrained models](https://catalog.ngc.nvidia.com/models?query=label:%22Riva%22) available from the NGC catalog:

**Local Docker:** You can use the Quick Start scripts to set up a local workstation and deploy the Riva services using Docker. Continue with this guide to use the Quick Start scripts.

In addition to using pretrained models, Riva Speech AI can run with fine-tune custom models using [NVIDIA NeMo](https://github.com/NVIDIA/NeMo). Refer to the {ref}`nemo-development` section for details regarding the advanced option to create a model repository with NVIDIA NeMo.


## Getting Started with Riva for Embedded Platforms

Riva includes Quick Start scripts to help you get started with Riva Speech AI Skills. These scripts are meant for deploying the services locally, for testing, and running the example applications.

1. Download the Riva Quick Start scripts. You can either use the command-line interface or you can download the scripts directly from your browser. Click the **Download** drop-down button in the upper right corner and select:

    - **CLI** - the download command is copied. Ensure you have the [NGC CLI tool](https://ngc.nvidia.com/setup/installers/cli) installed. Once installed, open the command prompt and paste the copied command to start your download.

    - **Browser (Direct Download)** - the download begins in a location of your choosing.

2. Initialize and start Riva. The initialization step downloads and prepares Docker images and models. The start script launches the server.

    :::{note}
    This process can take up to an hour on an average internet connection. On embedded platforms, preoptimized models for the GPU on the NVIDIA Jetson are downloaded.
    :::

    *Optional:* Modify the `config.sh` file within the `quickstart` directory with your preferred configuration. Options include:

      - which services to enable
      - which models to retrieve from NGC
      - where to store them
      - which GPU to use if more than one is installed on your system (refer to [Local (Docker)](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/installation/deploy-local.html#local-docker) for more details)
      - locations of the SSL/TLS certificate
      - key files if using a secure connection

   :::{note}
    For using the Riva translation services, refer to the `Configure translation services` instructions in the `config.sh` file within the `quickstart` directory.
    :::

   **Embedded**
   ```bash
   cd riva_quickstart_arm64_v2.12.0
   ```

   :::{note}
    If you are using the Jetson AGX Xavier or the Jetson NX Xavier platform, set the `$riva_tegra_platform` variable to `xavier` in the `config.sh` file within the `quickstart` directory.
    :::

   To use a USB device for audio input/output, connect it to the Jetson platform so it gets auto mounted into the container.


   **Initialize and start Riva**
   ```bash
   bash riva_init.sh
   bash riva_start.sh
   ```


3. Try walking through the different [tutorials on GitHub](https://github.com/nvidia-riva/tutorials). If running the Riva Quick Start scripts on a cloud service provider (such as AWS or GCP), ensure that your compute instance has an externally visible IP address. To run the tutorials, connect a browser window to the correct port (`8888` by default) of that external IP address.

4. Shut down the server when finished. After you have completed these steps and experimented with inferencing, run the `riva_stop.sh` script to stop the server.

For further details on how to customize a local deployment, refer to the [Local (Docker)](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/installation/deploy-local.html#local-docker) section.


## Transcribe Audio Files with Riva

For Automatic Speech Recognition (ASR), run the following commands from inside the Riva server container to perform streaming and offline transcription of audio files. If using SSL/TLS, ensure to include the `--ssl_server_cert /ssl/server.crt` option.

1. For offline recognition, run:

    ```bash
    riva_asr_client --audio_file=/opt/riva/wav/en-US_sample.wav
    ```

2. For streaming recognition, run:

    ```bash
    riva_streaming_asr_client --audio_file=/opt/riva/wav/en-US_sample.wav
    ```


## Synthesize Speech with Riva

From within the Riva server container, run the following command to synthesize the audio files.

```bash
riva_tts_client --voice_name=English-US.Female-1 \
                --text="Hello, this is a speech synthesizer." \
                --audio_file=/opt/riva/wav/output.wav
```

The audio files are stored in the `/opt/riva/wav` directory.

The streaming API can be tested by using the command-line option `--online=true`. However, there is no difference between both
options with the command-line client since it saves the entire audio to a `.wav` file.


## Translate Text or Speech with Riva

### Translate Text-to-Text (T2T)

From within the Riva server container, run the following command to perform a text-to-text translation from English to German.

```bash
riva_nmt_t2t_client --source_language_code="en-US" --target_language_code="de-DE" --text="This will become German words."
```

### Translate Speech-to-Text (S2T)

From within the Riva server container, run the following commands to perform a speech-to-text translation from English audio to German text.

```bash
riva_nmt_streaming_s2t_client --audio_file=/opt/riva/wav/en-US_sample.wav --source_language_code="en-US" --target_language_code="de-DE"
```

### Translate Speech-to-Speech (S2S)

From within the Riva server container, run the following commands to perform an speech-to-speech translation from Spanish audio to English audio.

```bash
riva_nmt_streaming_s2s_client --audio_file=/opt/riva/wav/es-US_sample.wav --source_language_code="es-US" --target_language_code="en-US"
```


## Riva Collections

The [Riva Collection](https://registry.ngc.nvidia.com/orgs/nvidia/collections/riva) contains the Riva Speech AI server, the Riva Speech AI client containers, the Riva Quick Start scripts resources, and the Riva Speech AI Skills Helm chart.

## Suggested Reading

For the latest product documentation, supported hardware and software, and release notes, refer to the [Riva User's Guide](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/index.html).

## Additional Resources

For organizations looking to deploy Riva-based applications in production to unlimited workloads and get full NVIDIA support with direct access to NVIDIA AI experts globally, explore [NVIDIA Riva](https://www.nvidia.com/en-us/ai-data-science/products/riva/) or [try NVIDIA Riva for free](https://www.nvidia.com/en-us/launchpad/ai/riva/) on NVIDIA LaunchPad.

## License

By downloading and using Riva software, you accept the terms and conditions of this [license](https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/).

riva_quickstart_arm64

Scripts and utilities for getting started with Riva Speech Skills on Embedded platforms

Riva Skills Embedded Quick Start

Description

Publisher

Latest Version

Modified

Compressed Size