NGC | Catalog
CatalogResourcesBERT QA on Azure ML with Triton Demo

BERT QA on Azure ML with Triton Demo

For downloads and more information, please view on a desktop device.
Logo for BERT QA on Azure ML with Triton Demo

Description

Demo notebook to deploy BERT QA model on Azure ML with Triton Inference Server

Publisher

NVIDIA

Latest Version

v1

Modified

April 4, 2023

Compressed Size

350.45 KB

BERT QA on Azure ML with Triton Demo

Description

This repository shows to deploy BERT QA model on Azure Machine Learning with NVIDIA Triton Inference Server for high performance inferencing. It includes a set of scripts and a Jupyter Notebook that details the step by step guides.

Prerequisite

To successfully run the included Jupyter Notebook, you need the followings:

  1. Ensure you have a config.json from Azure ML, saved in the root of the directory. To create one use the create_config method.
  2. Azure SDK for Python. Refer to the documentation.
  3. NVIDIA TensorRT optimized BERT QA model. Refer to the developer blog to create one before you proceed.

Viewing the Jupyter Notebook in NGC

To take a look at this, or any other, notebook; follow these steps:

  1. Navigate to the File Browser tab of the asset in NGC
  2. Select the version you'd like to see
  3. Under the actions menu (three dots) for the .ipynb file select "View Jupyter"
  4. There you have it! You can read a notebook for documentation and copy code samples without ever leaving NGC.

References

If you're new to Azure Machine Learning deployments, check out How and where to deploy models and Troubleshooting and debugging for additional resources.
If you're new to the NVIDIA Triton Inference Server, check out this information page and the Github page

End User License Agreements

Refer to the following NVIDIA End User License Agreements, included in LICENSE file.