Omniverse streaming AWS NLB Manager

Overview

The omniverse AWS NLB Manager service helps with the management of a pool of targetgroups and listeners for a set of preconfigured Network load balancers in AWS.

This chart assumes that a Kubernetes cluster is already available and configured.

Quick start

Before installing it's worth considering Image Pull Secrets and Ingress access.

NOTE: use --dry-run to inspect the resulting Kubernetes manifests generated from the helm chart.

helm upgrade \
    --install \
    nv.ov.svc.streaming.aws.nlb \
    . \
    --create-namespace \
    --namespace <<NAMESPACE-NAME>> \
    --set global.imagePullSecrets[0].name=registry-secret

Image Pull Secrets

Container images are hosted on nvcr.io behind a private repository.

Therefore, you will need to create an Image Pull Secret to be able to pull the containers within Kubernetes.

NOTE: if the <<NAMESPACE-NAME>> namespace does not exist, create it prior to creating the secret.

kubectl create namespace <<NAMESPACE-NAME>>

kubectl create secret docker-registry \
  registry-secret \
  --namespace <<NAMESPACE-NAME>> \
  --docker-server="nvcr.io" \
  --docker-username='$oauthtoken' \
  --docker-password=<<YOUR-NGC-API-KEY>>

Then during install, specify the global image pull secret.

--set global.imagePullSecrets[0].name=registry-secret

Ingress Access.

The default ingress class used in this chart is nginx, but this can be changed by supplying --set global.ingress.annotations."kubernetes\.io/ingress\.class"="INGRESS-CONTROLLER"

The following two examples make use of nginx ingress, but other ingress providers can be utilized such as traefik.

It's worth getting familiar with the set of options available under the global.ingress setting.

Access via DNS

This example uses a simple local DNS setup.

For production setups you'll likely need to configure TLS cert secrets for the hosts (this is beyond the scope of this guide).

To use a custom local DNS, for example ov.local, we'll need to update /etc/hosts and map it to the Node's Internal IP.

The resulting values to specify during helm install will be --set global.ingress.host="ov.local"

This example also uses nginx ingress:

# install nginx controller if not already present.

helm install ingress-nginx \
  ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.service.type=NodePort \
  --set controller.service.nodePorts.http=32080

helm upgrade \
    --install \
    nv.ov.svc.streaming.aws.nlb \
    . \
    --create-namespace \
    --namespace <<NAMESPACE-NAME>> \
    --set global.imagePullSecrets[0].name=registry-secret
    --set global.ingress.annotations."kubernetes\.io/ingress\.class"="nginx" \
    --set global.ingress.host="ov.local"

kubectl get nodes -o=wide

NAME                          STATUS   ROLES                  AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE       KERNEL-VERSION      CONTAINER-RUNTIME
ov-local-control-plane   Ready    control-plane,master   3h22m   v1.23.4   172.25.0.2    <none>        Ubuntu 21.10   5.15.0-27-generic   containerd://1.5.10

Add an entry to your /etc/hosts:

sudo vi /etc/hosts

172.25.0.2      ov.local

Get the port your NGINX is exposing in the NodePort service (32080 in this case)

kubectl get svc ingress-nginx-controller -n ingress-nginx

NAME                       TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller   NodePort   10.96.68.95   <none>        80:32080/TCP,443:31312/TCP   47m

A quick check confirms we can reach the service from outside the cluster:

curl http://ov.local:32080/status
"OK"

Access via Node's IP

When the ingress host is not specified, the ingress controller will default to using the controller service's IP which is then routed via the Node's IP.

Here's a setup using nginx ingress without a hostname.

# install nginx controller if not already present.

helm install ingress-nginx \
  ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.service.type=NodePort \
  --set controller.service.nodePorts.http=32080

helm upgrade \
    --install \
    nv.ov.svc.streaming.aws.nlb \
    . \
    --create-namespace \
    --namespace <<NAMESPACE-NAME>> \
    --set global.imagePullSecrets[0].name=registry-secret
    --set global.ingress.annotations."kubernetes\.io/ingress\.class"="nginx" \
    --set global.ingress.host=null

kubectl get svc ingress-nginx-controller -n ingress-nginx
NAME                       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller   NodePort   10.43.230.160   <none>        80:32080/TCP,443:32681/TCP

kubectl -n <<NAMESPACE-NAME>> get ingress
NAME                   CLASS    HOSTS   ADDRESS         PORTS   AGE
omniverse-example   <none>   *       10.43.230.160   80      10m

kubectl get nodes -o wide
NAME                    STATUS   ROLES                  AGE   VERSION        INTERNAL-IP   EXTERNAL-IP   OS-IMAGE   KERNEL-VERSION      CONTAINER-RUNTIME
k3d-ov-local-server-0   Ready    control-plane,master   21h   v1.23.6+k3s1   172.18.0.3    <none>        K3s dev    5.15.0-43-generic   containerd://1.5.5

From above, port 80 is mapped to 32080 (and 443 to 32681) on the ingress-nginx-controller, and the node's internal IP is 172.18.0.3

A quick check confirms we can reach the service from outside the cluster:

curl http://172.18.0.3:32080/status
"OK"

Configuration

To view values via helm use helm show values

NOTE: There are global values.

Values