Maxine Audio Effects Microservice

Description

Audio effects microservice constitutes ready-to-use containerized packages of cloud applications that are built from Maxine algorithms. These containers contain all end-to-end applications with necessary dependencies, which can be easily deployed on public and private clouds and enable client applications to provide the benefits of NVIDIA Maxine algorithms via cloud-based GPU computing. For example, mechanisms described in this document can be used to easily launch the audio effects microservice in a public/private GPU-based cloud for the purpose of using Maxine background denoiser, room echo canceller, and other Maxine Audio effects algorithms. The microservice instance receives and processes the audio stream from a client and returns the processed audio stream to the client.

Features

The microservice offers the following GPU-accelerated AI-based audio effects:

• Denoiser (Backround Noise removal)
• Dereverb (Room Echo Cancelation)
• Audio Super-resolution
• Studio Voice
• Chaining of audio effects
  • supported chainings:
    1. any combination of denoiser, dereverb and superres

Usage

Ensure the following prerequisites are available on your system for running the container:

• Docker at least version 19.03 or later.

  Refer to nvidia-docker for more informaiton about the latest nvidia-container-toolkit.

• Nvidia driver: 535+

  Driver dependency is aligned to CUDA version at least 11.8 or later.

  • Machine with GPU of any of below architectures:
    • sm_70 (e.g. V100)
    • sm_75 (e.g. T4)
    • sm_80 (e.g. A100)
    • sm_86 (e.g. A10, A40)
    • sm_89 (e.g. L4/L40)
    • sm_90 (e.g. H100)

Deployment requirements

• Make sure a kubernetes cluster is setup with knowledge of the GPU on which Audio Effects Microservice is being deployed on.
• Make sure kubectl (and helm) is pointing to the cluster where the microservice needs to be deployed.

Deploying Audio Effects Microservice Server helm chart

Download helm

• Ensure you have installed the NGC CLI tool.
• Copy the command from Fetch Command section and run it to download the chart.
• Extract downloaded tar file.

Configure parameters for Maxine Audio Effects microservice launch

• With default configuration in helm chart, the microservice has Denoiser (Background Noise removal) feature enabled.
• To customize, change values.yaml file in the extracted helm directory.
• Ensure correct GPU arch and change variables in "env:" section, as per requirement

- name: GST_DEBUG
  value: '1'
- name: GLOG_v
  value: '3'
- name: GLOG_logtostderr
  value: '1'
- name: GRPC_WORKER_THREADS
  value: '2'
- name: USE_DENOISER
  value: 'true'
- name: USE_DEREVERB
  value: 'false'
- name: USE_SUPERRES
  value: 'false'
- name: USE_STUDIO_VOICE_PERF
  value: 'false'
- name: USE_STUDIO_VOICE_QUALITY
  value: 'false'
- name: ENABLE_BNR_V2
  value: 'false'
- name: NUM_PIPELINES
  value: '1'
- name: AUDIO_CHUNK_DURATION
  value: '10'
- name: INPUT_SAMPLE_RATE
  value: '48000'
- name: OUTPUT_SAMPLE_RATE
  value: '48000'
- name: MAX_UDP_PORTS_PER_STREAM
  value: '2'
- name: MAX_STREAMS
  value: '200'
- name: INPUT_IO_TUNING_CONFIG
  value: /opt/configs/input_io_props.json
- name: OUTPUT_IO_TUNING_CONFIG
  value: /opt/configs/output_io_props.json
- name: NUM_OF_UDP_PORTS
  value: '400'
- name: INTERNAL_UDP_PORT_START
  value: '3308'
- name: EXTERNAL_UDP_PORT_START
  value: '3308'
- name: MY_POD_IP
  valueFrom:
    fieldRef:
      fieldPath: status.podIP

Configure Kubernetes service as per requirement

To connect with a client running within the cluster

• No need to change defaults in values.yaml.
• Use following ClusterIP endpoint to connect to service: ucf-maxine-audio-multistream-default-service.

To connect with a client running outside the cluster

• Change type of service from ClusterIP to NodePort in values.yaml, line 114.

type: ClusterIP

• Note: Understand considerations of using kubernetes service type=NodePort
• Note: After deployment, get NodePort information from kubectl get svc to connect to service.

Setup NGC secret

You will need to setup the NGC secret for the container image to be succesfully pulled. Follow the steps below to setup the NGC secret:

• Run,

echo -n '$oauthtoken:<YOUR API KEY>' | base64 -w0

Replace <YOUR API KEY> with your NGC API key.

• Create a json file named config.json with following content:

{
    "auths": {
        "nvcr.io": {
            "auth": "<NGC_REG_KEY>"
        }
    }
}

Replace <NGC_REG_KEY> with the base64 value from the previous step.

• Run,

kubectl create secret generic ngc-docker-reg-secret --from-file=.dockerconfigjson=config.json --type=kubernetes.io/dockerconfigjson

This will create a secret named ngc-docker-reg-secret.

Deploy helm

helm install maxine-audio-effects {{PATH TO EXTRACTED HELM}}

Performance

The server can support upto 600 concurrent streams for each of the individual effects. This number might vary if chaining of effects is performed.

Exceptions:

1. Studio Voice Effect only supports one stream per service instance

Supported Platforms

Machine with GPU of any of below architectures is supported:

• sm_70 (e.g. V100)
• sm_75 (e.g. T4)
• sm_80 (e.g. A100)
• sm_86 (e.g. A10, A40)
• sm_89 (e.g. L4/L40)
• sm_90 (e.g. H100)

Known Issues / Limitations

• Connecting to Maxine Audio Effects microservice from outside cluster using NodePort is not supported with UDP streaming type. It is only supported with GRPC streaming type.

Suggested Reading

Please refer the Audio Effects User Guide under resources for a more detailed documentation.

Additional Installation Resource

• AWS EKS Graphics driver setup
• Instance type: g4dn.xlarge, g5.xlarge
• AMI type: AL2_x86_64_GPU [ default]
• Graphics driver installation at VM Bootstrap:

sudo wget https://us.download.nvidia.com/tesla/525.60.13/NVIDIA-Linux-x86_64-525.60.13.run -c -P /usr/bin
sudo chmod +x /usr/bin/NVIDIA-Linux-x86_64-525.60.13.run
sudo init 3
sudo systemctl stop nvidia-persistenced
sudo systemctl stop nvidia-persistenced.service
sudo /usr/bin/NVIDIA-Linux-x86_64-525.60.13.run -s --no-x-check

• AWS VM Bootstrap reference
• GPU operator:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm fetch nvidia/gpu-operator --version v22.9.0--untar --untardir {fetch_directory}
helm upgrade --install {name} {fetch_directory } --values {parameter_override_file_path}

• GPU operator helm default parameters
• We override following parameters:

driver:
  enabled: false
toolkit:
  enabled: false
migManager:
  enabled: false
dcgmExporter:
  serviceMonitor:
    enabled: true

License

By pulling and using Maxine Audio Effects software, you accept the terms and conditions of the corresponding License.