Clara Holoscan Sample Applications

Prerequisites

The Sample Applications container of the Clara Holoscan Embedded SDK is designed to run on any of the Clara Developer Kits as well as standard x86 devices.

Running on x86 device

Before running this container on x86 make sure that you have the following installed:

• NVIDIA dGPU drivers: 510.73.08
• NVIDIA Container Toolkit

Note that it is recommended to run this container with Ampere or Turing GPU for best performances.

Running on NVIDIA AGX Developer Kit and NVIDIA IGX

Before running this container on the NVIDIA AGX Developer Kit or NVIDIA IGX, make sure you have set up your developer kit following the Clara AGX Developer Kit User Guide or NVidia IGX Orin Developer Kit User Guide.

For a full list of Holoscan documentation, visit the Holoscan developer page.

Requirements for these platforms include:

• NVIDIA Jetson Linux: 34.1.2 ¹
• NVIDIA dGPU drivers: 510.73.08 ²
• NVIDIA Container Toolkit ¹
• For AJA support: AJA drivers (refer to the user guide)

<sup>1. included when installing Holopack 1.1 on your Clara Developer Kit with SDK Manager. Holopack includes among other things the Board Support Package, L4T, Drivers, CUDA, cuDNN, TensorRT. Also, make sure you have joined the Clara Holoscan SDK Program and, if needed, the RiverMax SDK Program before using the NVIDIA SDK Manager.
2. included when running the nvgpuswitch script on your Clara Developer Kit, installed with the SDK Manager</sup>

Running from container

1. Log in to the NGC docker registry

2. Press the Copy Image Path button at the top of the container page on NGC

3. Paste the following command in your terminal

docker pull <NGC_CONTAINER_IMAGE_PATH>

Since v0.3.0 this container supports x86 and arm64 architectures and if you want to pull the image for a specific platform (x86 or arm64) you can run the following commands:

• amd64 architecture

docker pull --platform linux/amd64 <NGC_CONTAINER_IMAGE_PATH>

• arm64 architecture

docker pull --platform linux/arm64 <NGC_CONTAINER_IMAGE_PATH>

4. Ensure that X11 is configured to allow commands from docker:

xhost +local:docker

5. Start the container
   • Specify --device /dev/ajantv20:/dev/ajantv20 in the docker run command if you also have an AJA capture card you'd like to access from the run container.

since v0.3.0

docker run -it --rm \
  --runtime=nvidia \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -v /usr/share/vulkan:/usr/share/vulkan \
  -e NVIDIA_DRIVER_CAPABILITIES=graphics,video,compute,utility,display \
  -e DISPLAY=$DISPLAY \
  -w /opt/holoscan_sdk/ \
  ${image_name:tag}

for v0.2.0

docker run -it --rm \
  --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=graphics,video,compute,utility \
  -e DISPLAY=${DISPLAY} -v /tmp/.X11-unix:/tmp/.X11-unix \
  ${image_name:tag}

5. Run any of the four applications available in the container (descriptions below)

since v0.3.0

# Endoscopy tool tracking (C++ API) from recorded video
cd /opt/holoscan_sdk \
    && sed -i -e 's#^source:.*#source: replayer#' ./apps/endoscopy_tool_tracking/app_config.yaml \
    && ./apps/endoscopy_tool_tracking/endoscopy_tool_tracking

# Endoscopy tool tracking (GXF) from recorded video
cd /opt/holoscan_sdk && ./apps/endoscopy_tool_tracking_gxf/tracking_replayer

# Ultrasound segmentation (GXF) from recorded video
cd /opt/holoscan_sdk && ./apps/ultrasound_segmentation_gxf/segmentation_replayer

# Endoscopy tool tracking (C++ API) with AJA
cd /opt/holoscan_sdk \
    && sed -i -e 's#^source:.*#source: aja#' ./apps/endoscopy_tool_tracking/app_config.yaml \
    && ./apps/endoscopy_tool_tracking/endoscopy_tool_tracking

# Endoscopy tool tracking (GXF) with AJA
cd /opt/holoscan_sdk && ./apps/endoscopy_tool_tracking_gxf/tracking_aja

# Ultrasound segmentation (GXF) with AJA
cd /opt/holoscan_sdk && ./apps/ultrasound_segmentation_gxf/segmentation_aja

for v0.2.0

# Endoscopy tool tracking from recorded video
cd /opt/holoscan_sdk/tracking_replayer && ./apps/endoscopy_tool_tracking/run_tracking_replayer

# Ultrasound segmentation from recorded video
cd /opt/holoscan_sdk/segmentation_replayer && ./apps/ultrasound_segmentation/run_segmentation_replayer

# Endoscopy tool tracking with AJA
cd /opt/holoscan_sdk/tracking_aja && ./apps/endoscopy_tool_tracking/run_tracking_aja

# Ultrasound segmentation with AJA
cd /opt/holoscan_sdk/segmentation_aja && ./apps/ultrasound_segmentation/run_segmentation_aja

Sample applications

Endoscopy Tool Tracking

Based on a LSTM (long-short term memory) stateful model, these applications demonstrate the use of custom components for tool tracking, including composition and rendering of text, tool position, and mask (as heatmap) combined with the original video stream .

• endoscopy_tool_tracking: uses an AJA capture card or a pre-recorded video as input, based on the configuration in ./apps/endoscopy_tool_tracking/app_config.yaml (source: aja or source: replayer)
• tracking_aja: uses an AJA capture card for input stream
• tracking_replayer: uses a pre-recorded video as input

Ultrasound Bone Scoliosis Segmentation

Full workflow including a generic visualization of segmentation results from a spinal scoliosis segmentation model of ultrasound videos. The model used is stateless, so this workflow could be configured to adapt to any vanilla DNN model.

• segmentation_aja: uses an AJA capture card for input stream
• segmentation_replayer: uses a pre-recorded video as input

License

By pulling and using the container, you accept the terms and conditions of this End User License Agreement.