Holoscan Sample App Data for AI Colonoscopy Segmentation of Polyps

This resource contains a segmentation model for the identification of polyps during colonoscopies trained on the Kvasir-SEG dataset [1], using the ColonSegNet model architecture [2], as well as a sample surgical video.

[1] Jha, Debesh, Pia H. Smedsrud, Michael A. Riegler, Pål Halvorsen, Thomas de Lange, Dag Johansen, and Håvard D. Johansen, "Kvasir-seg: A segmented polyp dataset" Proceedings of the International Conference on Multimedia Modeling, pp. 451-462, 2020.

[2] Jha D, Ali S, Tomar NK, Johansen HD, Johansen D, Rittscher J, Riegler MA, Halvorsen P. Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep Learning. IEEE Access. 2021 Mar 4;9:40496-40510. doi: 10.1109/ACCESS.2021.3063716. PMID: 33747684; PMCID: PMC7968127.

Model

Given an RGB image of 512 x 512, this model provides semantic segmentation of the polyps. Each pixel stores a confidence score of [0,1] for polyp presence.

Note: The provided model is in ONNX format. It will automatically be converted into a TensorRT model (.engine) the first time it is processed by a Holoscan application.

Inputs

INPUT__0 - Input RGB image (batchsize, height, width, channels)
- shape=[1, 512, 512, 3]
- dtype=float32
- range=[0, 255]

Outputs

output_old - Segmentation output with per-pixel confidence [0,1].
- shape=[1, 512, 512]
- dtype=float32
- range=[0, 1]

Video Data

The sample data, provided by Simula Research Laboratory, is an .mp4 video of a colonoscopy scene with a polyp identified by the model.

Note: the .mp4 file must be converted into a GXF tensor file using the convert_video_to_gxf_entities.py script on GitHub to be used with the VideoStreamReplayer Holoscan operator.

License