Prerequisites

Ensure the following prerequisites are available on your system for running the container:

• Docker at least version 19.03 or later.

  Refer to nvidia-docker for more informaiton about the latest nvidia-container-toolkit.

• Nvidia driver: 535+

  Driver dependency is aligned to CUDA version at least 11.8 or later.

  • Machine with GPU of any of below architectures:
    • sm_70 (e.g. V100)
    • sm_75 (e.g. T4)
    • sm_80 (e.g. A100)
    • sm_86 (e.g. A10, A40)
    • sm_89 (e.g. L4, L40)

Deployment Requirements

• Make sure a kubernetes cluster is setup with knowledge of the GPU on which Video Effects microservice is being deployed on.
• Make sure kubectl (and helm) is pointing to the cluster where the microservice needs to be deployed.

Deploying Video helm chart

Download helm

• Ensure you have installed the NGC CLI tool.
• Copy the command from Fetch Command section and run it to download the chart.
• Extract downloaded tar file.

Configure Kubernetes service as per requirement

To connect with a client running within the cluster

• No need to change defaults in values.yaml
• Default ClusterIP endpoint: ucf-video-effects-default-service:8036

To connect with a client running outside the cluster

• Change type of service from ClusterIP to NodePort in values.yaml

type: ClusterIP

• Note: Understand considerations of using kubernetes service type=NodePort
• Note: After deployment, get NodePort information from kubectl get svc to connect to service.

Setup NGC secret

You will need to setup the NGC secret for the container image to be succesfully pulled. Follow the steps below to setup the NGC secret:

• Run,

echo -n '$oauthtoken:<YOUR API KEY>' | base64 -w0

Replace <YOUR API KEY> with your NGC API key.

• Create a json file named config.json with following content:

{
    "auths": {
        "nvcr.io": {
            "auth": "<NGC_REG_KEY>"
        }
    }
}

Replace <NGC_REG_KEY> with the base64 value from the previous step.

• Run,

kubectl create secret generic ngc-docker-reg-secret --from-file=.dockerconfigjson=config.json --type=kubernetes.io/dockerconfigjson

This will create a secret named ngc-docker-reg-secret.

Deploy helm

helm install video-effects-service {{PATH TO EXTRACTED HELM}}

You can test the deployment by deploying a test client. Refer to Video Effects Client Helm chart for more details.

Known Issues / Limitations

• Connecting to Video microservice from outside cluster using NodePort is not supported with UDP streaming type. It is only supported with GRPC streaming type.

Suggested Reading

Please refer to the Video Effects User Guide under resources for a more detailed documentation.

Additional Installation Resource

• AWS EKS Graphics driver setup
• Instance type: g4dn.xlarge, g5.xlarge
• AMI type: AL2_x86_64_GPU [ default]
• Graphics driver installation at VM Bootstrap:

sudo wget https://download.nvidia.com/XFree86/Linux-x86_64/535.161.07/NVIDIA-Linux-x86_64-535.161.07.run -c -P /usr/bin
sudo chmod +x /usr/bin/NVIDIA-Linux-x86_64-535.161.07.run
sudo init 3
sudo systemctl stop nvidia-persistenced
sudo systemctl stop nvidia-persistenced.service
sudo /usr/bin/NVIDIA-Linux-x86_64-535.161.07.run -s --no-x-check

• AWS VM Bootstrap reference
• GPU operator:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm fetch nvidia/gpu-operator --version v22.9.0--untar --untardir {fetch_directory}
helm upgrade --install {name} {fetch_directory } --values {parameter_override_file_path}

• GPU operator helm default parameters
• We override following parameters:

driver:
  enabled: false
toolkit:
  enabled: false
migManager:
  enabled: false
dcgmExporter:
  serviceMonitor:
    enabled: true

License

By pulling and using Maxine software, you accept the terms and conditions of the corresponding License.