This notebook demonstrates how to optimize a fine-tuned BERT TF checkpoint to TensorRT and then how to deploy it for inference using Triton inference server on OpenShift cluster.
Instructions on running the sample notebook
So how do you get this going with BERT fine-tuning, optimization and deployment? Well, thanks to the power of NGC, deploying this demo couldn't be easier. We've taken care of most of the heavy lifting and the NGC Catalog and Private Registry are geared towards simplifying the workflow. Follow these steps to get the notebook running: Step 1 Make sure you have set up OpenShift CLI and you can access the cluster
oc whoami
This command should print out the user Step 2 Download and save the notebook from File Browser tab of the asset in NGC Step 3 Start the notebook
jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root
Step 4 Follow the steps in the notebook to replicate the demo (or tweak to be even more awesome).. Step 5 Show off your work by tweeting @nvidiaai with the hashtag #NGC
Viewing the Jupyter Notebook in NGC
To take a look at this, or any other, notebook; follow these steps:
- Navigate to the File Browser tab of the asset in NGC
- Select the version you'd like to see
- Under the actions menu (three dots) for the .ipynb file select "View Jupyter"
- There you have it! You can read a notebook for documentation and copy code samples without ever leaving NGC.
System Requirements