NGC | Catalog
CatalogModelsclara_train_covid19_exam_ehr_xray

clara_train_covid19_exam_ehr_xray

For downloads and more information, please view on a desktop device.
Logo for clara_train_covid19_exam_ehr_xray

Description

The ultimate goal of this model is to predict the likelihood that a person showing up in the emergency room will need supplemental oxygen, which can aid physicians in determining the appropriate level of care for patients, including ICU placement.

Publisher

NVIDIA

Use Case

Classification

Framework

Medical

Latest Version

1

Modified

August 20, 2021

Size

1.06 GB

Model Overview

The goal of this model is to predict the likelihood that a person showing up in the emergency room will need supplemental oxygen, which can aid physicians in determining the appropriate level of care for patients, including ICU placement.

Model Architecture

The model uses a pre-trained ResNet34 [1] for image feature extraction together with a deep & cross network [2] to combine it with EMR features.

The models were developed within the EXAM consortium using federated learning.

Training

The model was trained across 20 different FL client sites using their local data. We provide both trained checkpoints for predicting the oxygen needs within a 24 hour and 72 hour period.

Dataset

The total dataset size across clients contained over 16 thousand cases. 70%, 10%, and 20% of the cases were used for training, validation, and testing, respectively.

The chest x-ray feature extraction branch of the network was Pretrained on >200,000 images from CheXpert dataset (on pneumonia vs. rest task) & fine-tuned on ~500 images from Mass Gen Brigham to predict RALE [3] score to evaluate lung edema on CXR.

Data JSON Configuration

The image file path and preprocessed (normalized) EMR data is provided to the model via a JSON file (generated by this csv_to_json.py under tutorial folder). An example JSON file dataset.json is shown below and included in the config data of this MMAR.

Note, this example shows randomly generated EMR features.

{
"label_format": [
 1
],
"training": [
 {
 "image": "png/COVID-19-AR-16434366_49342890_1.png",
 "label": [
 0.0
 ],
 "attribute": [
 1.0,
 0.0,
 0.0,
 0.0,
 -5.383737433496347,
 -5.529068235231984,
 -3.0449836092695035,
 -3.3071614888070617,
 -1.5085171464277605,
 -21.321931502297016,
 -3.2349338138535173,
 -0.5587819750563531,
 -0.2358153052578452,
 -3.2894831447364865,
 -1.943975338316161,
 -1.8036506672969472,
 -2.355162416789821,
 -0.5912857371345801,
 -2.7495268748977426,
 -1.7618679561085948,
 -1.4315354870229484
 ]
 },
...
}

Performance

The average area under the curve (AUC) across different FL clients' test sets was 0.94 and 0.91 for the 24 hour and 72 hour models, respectively.

How to Use this Model

This model needs to be used with NVIDIA hardware and software. For hardware, the model can run on any NVIDIA GPU with memory greater than 12 GB. For software, this model can be used with NVIDIA Clara Train. Users can test the pre-trained models with chest x-ray images and randomly generated EMR data using the COVID-19-AR dataset available for download using the NBIA Data Receiver with the provided manifest file inside tutorial folder and preprocessing steps described in the TUTORIAL.md file.

(Optional) Validate checksum of pretrained model ckpts

md5sum -c models/models.md5

Run docker:

For single GPU:

cd commands
./docker.sh 0

Or for multi GPU:

cd commands
./docker.sh all

Full instructions for the training and validation workflow can be found in our documentation.

Input

For data preparation and model input preprocessing, please follow the steps provided in the TUTORIAL.md file under tutorial folder of this MMAR.

Output

The model predicts a patient's oxygen needs based on a chest x-ray and patient vitals and lab values (EMR features) by giving a risk score [0...1].

Example: .37 (Low-Flow to High-Flow)

Risk Scoring:

Oxygen Treatment Room Air Low-Flow High-Flow Ventilator Death
Score 0.0 0.25 0.5 0.75 1.0

Custom Components

The model architecture EXAMnet is available as a custom component to the Clara Train SDK.

The config_train.json specifies a typical training configuration for this model. We set layer-wise learning rates for the EXAMnet, which can be set and dynamically adjusted during training using the AdjustLayerLRs handler. By default, this handler is disabled using "disabled": true.

Other custom components like EqualizeHistogram are also disabled in the example config_train.json but can be enabled using "disabled": false.

Limitations

This training and inference pipeline was developed by NVIDIA. It is based on an initial model developed at Massachusetts General Brigham, which was retrained using Federated Learning through a multinational hospital collaboration. The Software is for Research Use Only. Software’s recommendation should not be solely or primarily relied upon to diagnose or treat COVID-19 by a Healthcare Professional. This research use only software has not been cleared or approved by FDA or any regulatory agency. THIS SOFTWARE CANNOT BE USED FOR ANY COMMERCIAL PURPOSES WITHOUT THE EXPLICIT APPROVAL OF MASS GENERAL HOSPITAL AND WILL BE GOVERNED BY THEIR SPECIFIC LICENCE TERMS.

The provided trained checkpoints have trained using NVIDIA Clara Train's federated learning (FL) capabilities within the EXAM (EMR CXR AI Model) consortium, including Massachusetts General Brigham and its affiliated hospitals. Other participants included: Children's National Hospital in Washington, D.C.; NIHR Cambridge Biomedical Research Centre; The Self-Defense Forces Central Hospital in Tokyo; National Taiwan University MeDA Lab and MAHC and Taiwan National Health Insurance Administration; Tri-Service General Hospital in Taiwan; Kyungpook National University Hospital in South Korea; Faculty of Medicine, Chulalongkorn University in Thailand; Diagnosticos da America SA in Brazil; University of California, San Francisco; VA San Diego; University of Toronto; National Institutes of Health in Bethesda, Maryland; University of Wisconsin-Madison School of Medicine and Public Health; Memorial Sloan Kettering Cancer Center in New York; and Mount Sinai Health System in New York.

The software is for research use only. The software's recommendation should not be solely or primarily relied upon to diagnose or treat COVID-19 by a Healthcare Professional. This research use only software has not been cleared or approved by FDA or any regulatory agency.

The model was trained following a particular image and EMR feature preprocessing steps (see the TUTORIAL.md file under tutorial folder of this MMAR). Not following these steps might result in unreliable predictions.

Reference

[1] He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

[2] Wang, Ruoxi, et al. "Deep & cross network for ad click predictions." Proceedings of the ADKDD'17. 2017. 1-7.

[3] Zimatore, Claudio, et al. "The radiographic assessment of lung edema (RALE) score has excellent diagnostic accuracy for ARDS." (2019).

License

End User License Agreement is included with the product. Licenses are also available along with the model application zip file. By pulling and using the Clara Train SDK container and downloading models, you accept the terms and conditions of these licenses. THIS SOFTWARE CANNOT BE USED FOR ANY COMMERCIAL PURPOSES WITHOUT THE EXPLICIT APPROVAL OF MASS GENERAL HOSPITAL AND WILL BE GOVERNED BY THEIR SPECIFIC LICENCE TERMS.