Clara Holoscan Sample App Data for AI Colonoscopy Segmentation of Polyps

Overview

This resource contains a segmentation model for the identification of polyps during colonoscopies, as well as a sample colonoscopy video for inference and testing. This model was trained on the Kvasir-SEG dataset [1], using the ColonSegNet model architecture [2].

[1] Jha, Debesh, Pia H. Smedsrud, Michael A. Riegler, Pål Halvorsen, Thomas de Lange, Dag Johansen, and Håvard D. Johansen, "Kvasir-seg: A segmented polyp dataset" Proceedings of the International Conference on Multimedia Modeling, pp. 451-462, 2020.

[2] Jha D, Ali S, Tomar NK, Johansen HD, Johansen D, Rittscher J, Riegler MA, Halvorsen P. Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep Learning. IEEE Access. 2021 Mar 4;9:40496-40510. doi: 10.1109/ACCESS.2021.3063716. PMID: 33747684; PMCID: PMC7968127.

Model

The AI model for polyp identification is a segmentation model which given an RGB image of 512 x 512 provides

• semantic segmentation of the polyps. Each pixel stores a confidence score of [0,1] for polyp presence.

For details on the model inputs and outputs see below.

Inputs

• INPUT__0 - Input RGB image (batchsize, height, width, channels)
  • shape=[1, 512, 512, 3]
  • dtype=float32
  • range=[0, 255]

Outputs

• output_old - Segmentation output with per-pixel confidence [0,1].
  • shape=[1, 512, 512]
  • dtype=float32
  • range=[0, 1]

Video Data

Files:

• video/colon_exam_720x576.mp4: Colonoscopy video sample sequence provided by Simula Research Laboratory. The video sample includes a scene with a polyp identified by the model.
• video/colonoscopy.gxf_*: Converted colon_exam_720x576.mp4 for use with GXF replayer extension.

Directory Structure

The package contains two folders:

• the model folder with the source model in ONNX format, and .engine files for the TensorRT models optimized for the Clara AGX and Holoscan developer kits.
• the video folder with the recorded video data, both in mp4 (source) and converted to the NVIDIA GXF tensor format.

/
├── NVIDIA-Clara-Holoscan-SDK-EULA.pdf
├── model
│   ├── colon.onnx
│   ├── graph_surgeon.py
│   └── colon_engines
│       ├── NVIDIA-RTX-A6000_c86_n84.engine
│       └── Quadro-RTX-6000_c75_n72.engine
└── video
    ├── colon_exam_720x576.mp4
    ├── colonoscopy.gxf_entities
    └── colonoscopy.gxf_index

License

Refer to the license agreement for use of the sample data.