NGC | Catalog
CatalogResourcesOpenShift BERT Example

OpenShift BERT Example

For downloads and more information, please view on a desktop device.
Logo for OpenShift BERT Example

Description

This notebook demonstrates how to optimize a fine-tuned BERT TF checkpoint to TensorRT and then how to deploy it for inference using Triton inference server on OpenShift cluster.

Publisher

NVIDIA

Use Case

Nlp

Framework

TensorFlow2

Latest Version

1.0.0

Modified

November 10, 2020

Compressed Size

28.3 KB

Instructions on running the sample notebook

So how do you get this going with BERT fine-tuning, optimization and deployment? Well, thanks to the power of NGC, deploying this demo couldn't be easier. We've taken care of most of the heavy lifting and the NGC Catalog and Private Registry are geared towards simplifying the workflow. Follow these steps to get the notebook running: Step 1 Make sure you have set up OpenShift CLI and you can access the cluster

oc whoami

This command should print out the user Step 2 Download and save the notebook from File Browser tab of the asset in NGC Step 3 Start the notebook

jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root

Step 4 Follow the steps in the notebook to replicate the demo (or tweak to be even more awesome).. Step 5 Show off your work by tweeting @nvidiaai with the hashtag #NGC

Viewing the Jupyter Notebook in NGC

To take a look at this, or any other, notebook; follow these steps:

  1. Navigate to the File Browser tab of the asset in NGC
  2. Select the version you'd like to see
  3. Under the actions menu (three dots) for the .ipynb file select "View Jupyter"
  4. There you have it! You can read a notebook for documentation and copy code samples without ever leaving NGC.

System Requirements

  1. OpenShift
  2. Nvidia GPU Operator
  3. BERT pre-trained checkpoint
  4. TensorRT
  5. Triton Inference Server
  6. SQuAD v1.1 dataset