HoloHub Sample App Model for SSD Detection for Endoscopy Surgical Tools

This resource contains a SSD Detection model for the identification of surgical tools, meant to be run on the video in Holoscan Endoscopy Sample Data. The SSD model is from the NVIDIA DeepLearningExamples repository.

Go to the HoloHub repository to find the sample app that utilizes this model.

The data used to create the SSD model is derived from the model and video data in Holoscan Endoscopy Sample Data, which are kindly provided by Research Group Camma, IHU Strasbourg and University of Strasbourg.

Model

There are two ONNX models provided in this resource:

epoch24.onnx
epoch24_nms.onnx

as well as the original PyTorch model epoch_24.pt.

Please see the model conversion process from PyTorch epoch_24.pt to the two ONNX models, the addition of Non Maximum Suppression in epoch24_nms.onnx compared to epoch24.onnx, and description of the models in the Model Conversion to ONNX section in the application README on HoloHub.

Note: The provided model is in ONNX format. It will automatically be converted into a TensorRT model (.engine) the first time it is processed by a Holoscan application.

Inputs

For both epoch24.onnx and epoch24_nms.onnx

INPUT__0 - Input RGB image (batchsize, height, width, channels)
- shape=[1, 300, 300, 3]
- dtype=float32
- range=[-1, 1]

Outputs

For epoch24.onnx

OUTPUT__LOC - location of bounding box candidates.
- shape=[batch_size, 4, 8732]
- dtype=float32
OUTPUT__LABEL - scores of labels for locations of bounding box candidates.
- shape=[batch_size, 81, 8732]
- dtype=float32

For epoch24_nms.onnx

num_detections - number of valid detections.
- shape=[batch_size, 1]
- dtype=int32
detection_boxes - coordinates of non-max suppressed boxes, here 20 is max number of output boxes.
- shape=[batch_size, 20, 4]
- dtype=float32
detection_scores - the scores for the boxes.
- shape=[batch_size, 20]
- dtype=float32
detection_classes - the classes for the boxes.
- shape=[batch_size, 20]
- dtype=int32

The NMS layer in epoch24_nms.onnx is EfficientNMS_TRT. See the Model Conversion to ONNX section in the application README for details on how we utilize it.

Video Data

This model is meant to be run on the video in Holoscan Endoscopy Sample Data.

License