HoloHub Sample App Model for SSD Detection for Endoscopy Surgical Tools

This resource contains a SSD Detection model for the identification of surgical tools, meant to be run on the video in Holoscan Endoscopy Sample Data. The SSD model is from the NVIDIA DeepLearningExamples repository.

Go to the HoloHub repository to find the sample app that utilizes this model.

The data used to create the SSD model is derived from the model and video data in Holoscan Endoscopy Sample Data, which are kindly provided by Research Group Camma, IHU Strasbourg and University of Strasbourg.

Model

There are two ONNX models provided in this resource:

• epoch24.onnx
• epoch24_nms.onnx

as well as the original PyTorch model epoch_24.pt.

Please see the model conversion process from PyTorch epoch_24.pt to the two ONNX models, the addition of Non Maximum Suppression in epoch24_nms.onnx compared to epoch24.onnx, and description of the models in the Model Conversion to ONNX section in the application README on HoloHub.

Note: The provided model is in ONNX format. It will automatically be converted into a TensorRT model (.engine) the first time it is processed by a Holoscan application.

Inputs

For both epoch24.onnx and epoch24_nms.onnx

• INPUT__0 - Input RGB image (batchsize, height, width, channels)
  • shape=[1, 300, 300, 3]
  • dtype=float32
  • range=[-1, 1]

Outputs

For epoch24.onnx

• OUTPUT__LOC - location of bounding box candidates.
  • shape=[batch_size, 4, 8732]
  • dtype=float32
• OUTPUT__LABEL - scores of labels for locations of bounding box candidates.
  • shape=[batch_size, 81, 8732]
  • dtype=float32

For epoch24_nms.onnx

• num_detections - number of valid detections.
  • shape=[batch_size, 1]
  • dtype=int32
• detection_boxes - coordinates of non-max suppressed boxes, here 20 is max number of output boxes.
  • shape=[batch_size, 20, 4]
  • dtype=float32
• detection_scores - the scores for the boxes.
  • shape=[batch_size, 20]
  • dtype=float32
• detection_classes - the classes for the boxes.
  • shape=[batch_size, 20]
  • dtype=int32

The NMS layer in epoch24_nms.onnx is EfficientNMS_TRT. See the Model Conversion to ONNX section in the application README for details on how we utilize it.

Video Data

This model is meant to be run on the video in Holoscan Endoscopy Sample Data.

License

Refer to the file NVIDIA Clara Holoscan SSD Evaluation License (28Feb2023) in the downloaded files.