Verification Tool

The Holoscan for Media Verification Tool validates cluster readiness and assesses key performance characteristics for Holoscan for Media deployments.

For more details, refer to the user guide.

Prerequisites

Cluster Environment

• Holoscan for Media certified cluster
• Holoscan for Media local development setup with Kubernetes

Workstation Tooling

The jump node used to run the tool must have the following installed:

• Python 3.10 or 3.12
• kubectl installed and configured for the target cluster
• oc installed (for OpenShift token-based login when applicable)

Cluster Access and Permissions

• A valid kubeconfig that includes a user token for the target cluster (in case of an OpenShift cluster)
• Permissions to create commonly used resources (namespaces, pods, jobs, configmaps, secrets)

Verify Cluster Access

Before proceeding, verify that you can access the cluster using a token:

1. Check the current context using:

   oc config view --minify --raw
   

2. If users > user > token is not present, then log in to the cluster using a token generated from the OpenShift web console:

   oc login --token=<token> --server=<cluster_url>
   

NGC API Key (NVIDIA Container Registry)

• Required to pull container images from NVIDIA NGC
• Create an NGC API key following the NGC user guide
• Create a secret in the default namespace to pull images from the NGC registry:
  kubectl create secret docker-registry <ngc-secret-name> --docker-server=nvcr.io '--docker-username=$oauthtoken' --docker-password=<API-KEY> --docker-email=<your-email>
  

  Replace the placeholders as appropriate. The <ngc-secret-name> will be used below to install the verification tool.

Rivermax License and Secret

Network validation tests require the Rivermax license. Create a Kubernetes secret from the license file:

1. Create Rivermax license secret (replace <path_to_license> with the actual path)

   kubectl create secret generic rivermax-license --from-file=<path_to_license>
   

2. Verify that you created the created

   kubectl get secret rivermax-license
   

Installation of the Verification Tool

1. Download the binary h4m-verification-tool.tar.gz

2. Extract the archive:

   tar -xvf h4m-verification-tool.tar.gz
   

3. Configure the test input by editing sample_test.yaml:

   Name	Description	Default
   image-pull-secret	Name of the NGC image pull secret. Use kubectl get secrets to get the list of secrets	""
   image-name	Name of the image to use for the test	nvcr.io/nvidia/holoscan-for-media/h4m-verification-tool-internals:0.1.0

   Example

   image-pull-secret: <ngc-secret-name>
   image-name: nvcr.io/nvidia/holoscan-for-media/h4m-verification-tool-internals:0.1.0
   

4. Set up a Python virtual environment:

   a. Install pip, if not present:

   sudo apt update && sudo apt install -y python3-pip python3-venv
   

   b. Execute the following commands to create a Python virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate
   pip install --upgrade pip
   pip install -r requirements.txt
   

Running the Verification Tool

Run the tool using the following command:

python3 run_tests.py sample_test.yaml

The tool saves test results in a timestamped directory similar to the following:

cluster-validation-suite_<date_time>/
├── test_runner_<date_time>.log
├── Platform_Checks/
├── Rivermax_Generic_API/
├── summary_report.json
└── test_report.html

• test_runner_<date_time>.log: Verification tool execution log
• Platform_Checks/: Platform validation outputs and logs
• Rivermax_Generic_API/: Test artifacts
• summary_report.json: Summary of test outcomes
• test_report.html: HTML report containing the result of each of the tests

Troubleshooting

kubectl Command Not Found

Install kubectl using the instructions from the Kubernetes documentation.

Permission Denied When Creating Secrets

Verify RBAC permissions:

kubectl auth can-i create secrets
kubectl auth can-i create pods

Test Fails With "Failed to start pods"

Confirm the following:

• Valid NGC API key configured
  kubectl run test --image=nvcr.io/nvidia/holoscan-for-media/h4m-verification-tool-internals:0.1.0 --overrides='{"spec":{"imagePullSecrets":[{"name":"<ngc-secret-name>"}]}}' --rm -i --restart=Never --command -- echo "Image pulled successfully"
  

• Appropriate nodes are present with worker role
  kubectl get nodes -l node-role.kubernetes.io/worker -o name
  

• Required resources are available and allocatable (check using the following)
  kubectl describe nodes | awk '/^Name:/ {node=$2} /^Capacity:/ {print "\n" node; print; getline; while ($0 !~ /^Allocatable:/ && $0 !~ /^Conditions:/) {print; getline}} /^Allocatable:/ {print; getline; while ($0 !~ /^System Info:/ && $0 !~ /^Non-terminated Pods:/) {print; getline}}'
  

• Try increasing the timeout using export H4M_TEST_TIMEOUT=600 and re-run the test.