Clara Deploy SDK is being consolidated into Clara Holoscan SDK

More info https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/collections/claradeploy

Clara AI Pancreas Tumor Segmentation

This asset requires the Clara Deploy SDK. Follow the instructions on the Clara Ansible page to install the Clara Deploy SDK.

Overview

This example is a containerized AI inference application, developed for use as one of the operators in the Clara Deploy pipelines. The application is built on the base AI application container, which provides the application framework to deploy Clara Train TLT trained models. The same execution configuration file, set of transform functions, and scanning window inference logic are used; however, inference is performed on the Triton Inference Server.

Inputs

This application, in the form of a Docker container, expects an input folder (/input by default), which can be mapped to the host volume when the Docker container is started. This folder must contain a volume image file in the Nifti or MetaImage format. Furthermore, the volume image must be constructed from a single series of a DICOM study, typically an axial series with the data type of the original primary.

Outputs

This application saves the segmentation results to an output folder, /output by default, which can also be mapped to a folder on the host volume. After the application completes successfully, a segmentation volume image of format MetaImage is saved in the output folder. The name of the output file is the same as that of the input file due to certain limitations of the downstream consumer.

The example container also publishes data for the Clara Deploy Render Server to the /publish folder by default. The original volume image, segmented volume image, and metadata file, along with a render configuration file, are saved in this folder.

AI Model

The application uses the segmentation_ct_pancreas_and_tumor model, which uses the tensorflow_graphdef platform. The input tensor is of shape 96x96x96 with a single channel. The output is of the same shape with three channels.

The NVIDIA® Clara Train Transfer Learning Toolkit (TLT) for Medical Imaging provides pre-trained models unique to medical imaging, with additional capabilities such as integration with the AI-assisted Annotation SDK for speeding up annotation of medical images. This allows access to AI-assisted labeling [Reference].

The application uses the segmentation_ct_pancreas_and_tumor model provided by the NVIDIA Clara Train TLT for pancreas tumor segmentation, which is converted from the TensorFlow Checkpoint model to tensorflow_graphdef using the TLT model export tool.

You can download the model using the following commands:

# Download NGC Catalog CLI
wget https://ngc.nvidia.com/downloads/ngccli_cat_linux.zip && unzip ngccli_cat_linux.zip && rm ngccli_cat_linux.zip ngc.md5 && chmod u+x ngc

# Configure API key (Refer to https://docs.nvidia.com/ngc/ngc-getting-started-guide/index.html#generating-api-key)
./ngc config set

# Download the model
./ngc registry model download-version nvidia/med/segmentation_ct_pancreas_and_tumor:1

Note: NGC Catalog CLI is needed to download models without Clara Train SDK: Please follow the NGC documentation to configure the CLI API key.

Detailed model information can be found at (downloaded model folder)/docs/Readme.md.

This application also uses the same transforms library and configuration file for the validation/inference pipeline during TLT model training. The key model attributes (e.g. the model name and network input dimensions), are saved in the config_inference.json file and consumed by the application at runtime.

NVIDIA Triton Inference Server (formerly known as TRTIS)

The application performs inference on the NVIDIA Triton Inference Server, which provides a cloud inferencing solution optimized for NVIDIA GPUs. The server provides an inference service via an HTTP or gRPC endpoint, allowing remote clients to request inferencing for any model being managed by the server.

Directory Structure

The application source code files are in the directory structure shown below.

/
├── app_base_inference_v2
 ├── ai4med
 ├── config
 │   ├── config_render.json
 │   ├── config_inference.json
 │   └── __init__.py
 ├── dlmed
 ├── inferers
 ├── model_loaders
 ├── ngc
 ├── public
 ├── utils
 ├── writers
 ├── app.py
 ├── Dockerfile
 ├── executor.py
 ├── logging_config.json
 ├── main.py
 └── requirements.txt

The following describes the directory contents:

• The ai4med and dlmed directories contain the library modules shared with Clara Train SDK, mainly for its transforms functions and base inference client classes.
• The config directory contains model-specific configuration files, which is needed when building a customized container for a specific model.
• The config_inference.json file contains the configuration sections for pre- and post-transforms, as well as the model loader, inferer, and writer.
• The config_render.json contains the configuration for the Clara Deploy Render Server.
• The inferers directory contains the implementation of the simple and scanning window inference client using the Triton API client library
• The model_loaders directory contains the implementation of the model loader that gets model details from Triton Inference Server.
• The ngc and public directories contain the user documentation.
• The utils directory contains utilities for loading modules and creating application objects.
• The Writers directory contains the specialized output writer required by Clara Deploy SDK, which saves the segmentation result to a volume image file as MetaImage.

Executing Operator as Docker Container

Prerequisites

1. Check if the Docker image of Triton has been imported into the local Docker repository with the following command:

docker images | grep tritonserver

2. Look for the image name tritonserver and the correct tag for the release, e.g. 20.07-v1-py3. If the image does not exist locally, it will be pulled from NVIDIA Docker registry.
3. Download both the input dataset and the trained model from the MODEL SCRIPTS section for this container on NGC, following the steps in the Setup section.

Step 1

Change to your working directory (e.g. test_pancreas_tumor).

Step 2

Create, if they do not exist, the following directories under your working directory:

• input containing the input image file
• output for the segmentation output
• publish for publishing data for the Render Server
• logs for the log files
• models containing models copied from the segmentation_ct_pancreas_and_tumor_v1 folder

Step 3

In your working directory,

• Create a shell script (run_docker.sh, or another name if you prefer.
• Copy the sample content below, change the APP_NAME to the full name of this docker, e.g. nvcr.io/ea-nvidia-clara/clara/ai-pancreastumor:0.7.2-2009.3.
• Save the file.

#!/bin/bash

# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# NVIDIA CORPORATION and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA CORPORATION is strictly prohibited.

# Define the name of the app (aka operator); assumed the same as the project folder name
APP_NAME="app_pancreastumor"

# Define the model name for use when launching TRTIS with only the specific model
MODEL_NAME="segmentation_ct_pancreas_and_tumor_v1"

# Define the Triton Inference Server Docker image, which will be used for testing
# Use either local repo or NVIDIA repo
TRITON_IMAGE="nvcr.io/nvidia/tritonserver:20.07-v1-py3"

# Launch the container with the following environment variables
# to provide runtime information
export NVIDIA_CLARA_TRTISURI="localhost:8000"

# Create a Docker network so that containers can communicate on this network
NETWORK_NAME="container-demo"

# Create network
docker network create ${NETWORK_NAME}

# Run TRTIS(name: triton), maping ./models/${MODEL_NAME} to /models/${MODEL_NAME}
# (localhost:8000 will be used)
RUN_TRITON="nvidia-docker run --name triton --network ${NETWORK_NAME} -d --rm --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
 -p 8000:8000 \
 -v $(pwd)/models/${MODEL_NAME}:/models/${MODEL_NAME} ${TRITON_IMAGE} \
 tritonserver --model-repository=/models"

# Display the command
echo ${RUN_TRITON}
# Run the command to start the inference server Docker
eval ${RUN_TRITON}

# Wait until Triton is ready
triton_local_uri=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' triton)
echo -n "Wait until Triton ${triton_local_uri} is ready..."
while [ $(curl -s ${triton_local_uri}:8000/api/status | grep -c SERVER_READY) -eq 0 ]; do
 sleep 1
 echo -n "."
done
echo "done"

export NVIDIA_CLARA_TRTISURI="${triton_local_uri}:8000"

# Run ${APP_NAME} container
# Launch the app container with the following environment variables internally,
# to provide input/output path information
docker run --name ${APP_NAME} --network ${NETWORK_NAME} -it --rm \
 -v $(pwd)/input:/input \
 -v $(pwd)/output:/output \
 -v $(pwd)/logs:/logs \
 -v $(pwd)/publish:/publish \
 -e NVIDIA_CLARA_TRTISURI \
 -e DEBUG_VSCODE \
 -e DEBUG_VSCODE_PORT \
 -e NVIDIA_CLARA_NOSYNCLOCK=TRUE \
 ${APP_NAME}

echo "${APP_NAME} is done."

# Stop Triton container
echo "Stopping Triton"
docker stop triton > /dev/null

# Remove network
docker network remove ${NETWORK_NAME} > /dev/null

Step 4

Execute the script as shown below and wait for the application container to finish:

./run_docker.sh

Step 5

Check for the following output files:

1. Segmentation results in the output directory:

• One file of the same name as your input file, with extension .mhd
• One file of the same name, with extension .raw

2. Published data in the publish directory:

• Original volume image, in either MHD or NIfTI format
• Segmentation volume image (.output.mhd and .output.raw)
• Render Server config file (config_render.json)
• Metadata file describing the above file (config.meta)

Step 6

To visualize the segmentation results, any tool that support MHD or NFiTI can be used, e.g. 3D Slicer.

Executing Operator Docker Container Interactively

To see the internals of the container or to run the application within the container, please follow the following steps.

1. See the above section on how to run the container with the required environment variables and volume mapping, and start the container by replacing the docker run command with the following:

docker run -it --rm --entrypoint /bin/bash

2. Once in the Docker terminal, ensure the current directory is /.
3. Execute the following command:

python3 ./app_base_inference_v2/main.py

4. When finished, type exit.

License

An End User License Agreement is included with the product. By pulling and using the Clara Deploy asset on NGC, you accept the terms and conditions of these licenses.

Suggested Reading

Release Notes, the Getting Started Guide, and the SDK itself are available at the NVIDIA Developer forum.

For answers to any questions you may have about this release, visit the NVIDIA Devtalk forum.