NGC | Catalog
CatalogResourcesBERT QA on Azure ML with Triton Demo

BERT QA on Azure ML with Triton Demo

Logo for BERT QA on Azure ML with Triton Demo
Demo notebook to deploy BERT QA model on Azure ML with Triton Inference Server
Latest Version
April 4, 2023
Compressed Size
350.45 KB

BERT QA on Azure ML with Triton Demo


This repository shows to deploy BERT QA model on Azure Machine Learning with NVIDIA Triton Inference Server for high performance inferencing. It includes a set of scripts and a Jupyter Notebook that details the step by step guides.


To successfully run the included Jupyter Notebook, you need the followings:

  1. Ensure you have a config.json from Azure ML, saved in the root of the directory. To create one use the create_config method.
  2. Azure SDK for Python. Refer to the documentation.
  3. NVIDIA TensorRT optimized BERT QA model. Refer to the developer blog to create one before you proceed.

Viewing the Jupyter Notebook in NGC

To take a look at this, or any other, notebook; follow these steps:

  1. Navigate to the File Browser tab of the asset in NGC
  2. Select the version you'd like to see
  3. Under the actions menu (three dots) for the .ipynb file select "View Jupyter"
  4. There you have it! You can read a notebook for documentation and copy code samples without ever leaving NGC.


If you're new to Azure Machine Learning deployments, check out How and where to deploy models and Troubleshooting and debugging for additional resources.
If you're new to the NVIDIA Triton Inference Server, check out this information page and the Github page

End User License Agreements

Refer to the following NVIDIA End User License Agreements, included in LICENSE file.