NGC | Catalog
Welcome Guest
CatalogHelm ChartsMorpheus MLflow Triton Plugin

Morpheus MLflow Triton Plugin

For versions and more information, please view on a desktop device.
Logo for Morpheus MLflow Triton Plugin

Description

This Helm chart deploys a MLflow server with NodePort for accessing the UI. It also includes a MLflow deployment plugin for deploying from MLflow into Triton.

Publisher

NVIDIA

Latest Version

22.04

Compressed Size

10.16 KB

Modified

April 28, 2022

Overview

NVIDIA Morpheus is an open AI application framework that provides cybersecurity developers with a highly optimized AI pipeline and pre-trained AI capabilities that, for the first time, allow them to instantaneously inspect all IP traffic across their data center fabric. Bringing a new level of security to data centers, Morpheus provides dynamic protection, real-time telemetry, adaptive policies, and cyber defenses for detecting and remediating cybersecurity threats.

Setup

The Morpheus MLflow container is packaged as a Kubernetes (aka k8s) deployment using a Helm chart. NVIDIA provides installation instructions for the NVIDIA Cloud Native Core Stack which incorporates the setup of these platforms and tools.

NGC API Key

First, you will need to set up your NGC API Key to access all the Morpheus components, using the instructions from the NGC Registry CLI User Guide. Once you have created your API key, create an environment variable containing your API key for use by the commands used further in these instructions:

export API_KEY="<your key>"

After installing EGX Stack, install and configure the NGC Registry CLI using the instructions from the NGC Registry CLI User Guide.

Create Namespace for Morpheus

Create a namespace and an environment variable for the namespace to organize the k8s cluster deployed via EGX Stack and logically separate Morpheus-related deployments from other projects using the following command:

kubectl create namespace <some name>
export NAMESPACE="<some name>"

Install Morpheus MLflow

Install the chart as follows:

helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-mlflow-22.04.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm install --set ngc.apiKey="$API_KEY" \
 --namespace $NAMESPACE \
 mlflow1 \
 morpheus-mlflow

Download Morpheus reference models

You can download the Morpheus reference models from the models area of this registry.

Interacting with the plugin

Once the MLflow server pod is deployed, you can make use of the plugin by running a bash shell in the pod container like this:

kubectl exec -it deploy/mlflow -- /bin/bash
(mlflow) root@mlflow-6cdd744679-9lb82:/mlflow#

Publish reference models to MLflow

The publish_model_to_mlflow script is used to publish onnx or tensorrt models to MLflow.

cd /scripts

python publish_model_to_mlflow.py \
     --model_name ref_model_1 \
     --model_file /sid-bert-onnx/1/sid-bert.onnx \
     --model_config /sid-bert-onnx/config.pbtxt \
 --flavor onnx 

python publish_model_to_mlflow.py \
     --model_name ref_model_1 \
     --model_file /sid-bert-onnx/1/sid-bert.onnx \
     --model_config /sid-bert-onnx/config.pbtxt \
 --flavor tensorrt

Deploy reference models to Triton

mlflow deployments create -t triton --flavor onnx --name ref_model_1 -m models:/ref_model_1/1 -C "version=1"

mlflow deployments create -t triton --flavor onnx --name ref_model_2 -m models:/ref_model_2/1 -C "version=1"

Deployments

The following deployment functions are implemented within the plugin. The plugin will deploy the associated config.pbtxt with the saved model version.

Create Deployment

To create a deployment use the following command:

mlflow deployments create -t triton --flavor onnx --name mini_bert_onnx -m models:/mini_bert_onnx/1 -C "version=1"

Delete Deployment

mlflow deployments delete -t triton --name mini_bert_onnx/1 

Update Deployment

mlflow deployments update -t triton --flavor onnx --name mini_bert_onnx/1 -m models:/mini_bert_onnx/1

List Deployments

mlflow deployments list -t triton

Get Deployment

mlflow deployments get -t triton --name mini_bert_onnx