Orsi Academy Sample Applications

This resource contains a binary tool segmentation model [1], a binary out-of-body classification model and sample surgical videos as accompanying assets for the Orsi Academy holohub applications.

[1] Hofman, J., De Backer, P., Manghi, I., Simoens, J., De Groote, R., Van Den Bossche, H., D’Hondt, M., Oosterlinck, T., Lippens, J., Van Praet, C., Ferraguti, F., Debbaut, C., Li, Z., Kutter, O., Mottrie, A., Decaestecker, K.: First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery. Healthc. Technol. Lett. 1–7 (2023)

Models

The binary segmentation model is used for organic vs. non-organic pixel classification and was trained on a dataset comprised of 31812 images covering 37 classes of non-organic items, including robotic and laparoscopic instruments, needles, wires, clips, vessel loops, bulldogs, gauzes, and more. A train/val/test split of 24087/4545/3180 images was used yielding a 0.94 test set mean IoU.

• Input: RGB image (bathsize, height, width, channels)
  • shape=[1, 512, 512, 3]
  • dtype=float32
  • range=[0, 255]
• Output: probability output tensor (batchsize, class, height, width)
  • shape=[1, 2, 512, 512]
  • dtype=float32
  • range=[0, 1]

The pipeline postprocesses the output tensor using a softmax operator to obtain the final classification.

The binary out-of-body classification model was trained on a dataset comprised of 496828 images covering 6 procedure types and 4 robotic systems, yielding a ROC-AUC score of 99.46%.

• Input: RGB image (batchsize, height, width, channels)
  • shape=[1, 224, 224, 3]
  • dtype=float32
  • range=[0, 255]
• Output: probability output tensor (batchsize,)
  • shape [1,]
  • dtype=float32
  • range=[0, 1]

The pipeline postprocesses the output tensor using a sigmoid operator to obtain the final classification.

Video Data

The sample data, kindly provided by Orsi Academy are snippets of 3 surgical videos. For the Anonymization and MultiAI applications, snippets from partial nephrectomy procedures were used. For the segmentation application, a snippet was used from a procedure where a migrated endovascular stent was removed as a result of a nutcracker syndrome. The paper describing the deployment of the holohub segmentation application [1] was awarded an outstanding paper award at the joint 17th AE-CAI, 10th CARE and 6th OR 2.0 MICCAI workshop 2023. The work contributing to the anonymization pipeline was presented at CLINICCAI22 and was awarded a best presentation award.

Note: the .mp4 files must be converting into a GXF entity using the convert_videos_to_gxf.sh script.

License

The sample data and models are free for non-commercial use under the Creative Commons Attribution CC BY-NC-ND 4.0. Open to commercial use after review. Please contact Pieter.De.Backer@orsi.be for inquiries. When using segmentation or anonymization models, users must cite [1]. Please also refer to license agreement.