Clara Holoscan Sample Applications

Prerequisites

The Sample Applications container of the Clara Holoscan Embedded SDK is designed to run on any of the Clara Developer Kits. Before pulling and running the container, make sure you've set up your developer kit following the Clara AGX Developer Kit User Guide. For a full list of Holoscan documentation, visit the Holoscan developer page.

Requirements include:

• NVIDIA Jetson Linux: 34.1.2 ¹
• NVIDIA dGPU drivers: 510.73.08 ²
• NVIDIA Container Toolkit ¹
• For AJA support: AJA drivers (refer to the user guide)

<sup>1. included when installing JetPack 5.0 HP1 on your Clara Developer Kit with SDK Manager
2. included when running the nvgpuswitch script on your Clara Developer Kit, installed with the SDK Manager</sup>

Running from container

1. Log in to the NGC docker registry

2. Press the Pull Tag button at the top of the container page on NGC and paste that command in your terminal.

   • Note: take note of the last token in the command (docker pull ${image_name:tag}), you'll use that information to run the container in step 4

3. Ensure that X11 is configured to allow commands from docker:

xhost +local:docker

4. Start the container
   • Specify --device /dev/ajantv20:/dev/ajantv20 in the docker run command if you also have an AJA capture card you'd like to access from the run container.

docker run -it --rm \
  --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=graphics,video,compute,utility \
  -e DISPLAY=${DISPLAY} -v /tmp/.X11-unix:/tmp/.X11-unix \
  ${image_name:tag}

5. Run any of the four applications available in the container (descriptions below)

# Endoscopy tool tracking from recorded video
cd /opt/holoscan_sdk/tracking_replayer && ./apps/endoscopy_tool_tracking/run_tracking_replayer

# Ultrasound segmentation from recorded video
cd /opt/holoscan_sdk/segmentation_replayer && ./apps/ultrasound_segmentation/run_segmentation_replayer

# Endoscopy tool tracking with AJA
cd /opt/holoscan_sdk/tracking_aja && ./apps/endoscopy_tool_tracking/run_tracking_aja

# Ultrasound segmentation with AJA
cd /opt/holoscan_sdk/segmentation_aja && ./apps/ultrasound_segmentation/run_segmentation_aja

Reference applications

Endoscopy Tool Tracking

Based on a LSTM (long-short term memory) stateful model, these applications demonstrate the use of custom components for tool tracking, including composition and rendering of text, tool position, and mask (as heatmap) combined with the original video stream .

• tracking_aja: uses an AJA capture card for input stream
• tracking_replayer: uses a pre-recorded video as input

Ultrasound Bone Scoliosis Segmentation

Full workflow including a generic visualization of segmentation results from a spinal scoliosis segmentation model of ultrasound videos. The model used is stateless, so this workflow could be configured to adapt to any vanilla DNN model.

• segmentation_aja: uses an AJA capture card for input stream
• segmentation_replayer: uses a pre-recorded video as input