NemoCurator Instruction Data Guard

Model Overview

Description:

Instruction Data Guard is a deep-learning classification model that helps identify LLM poisoning attacks in datasets. It is trained on an instruction:response dataset and LLM poisoning attacks of such data. Note that optimal use for Instruction Data Guard is for instruction:response datasets.

License/Terms of Use:

NVIDIA Open Model License Agreement

Reference:

The Internal State of an LLM Knows When It's Lying: https://arxiv.org/pdf/2304.13734

Model Architecture:

Architecture Type: FeedForward MLP
Network Architecture: 4 Layer MLP

Input:

Input Type(s): Text Embeddings
Input Format(s): Numerical Vectors
Input Parameters: 1D Vectors
Other Properties Related to Input: The text embeddings are generated from the Aegis Defensive Model. The length of the vectors is 4096.

Output:

Output Type(s): Classification Scores
Output Format: Array of shape 1
Output Parameters: 1D
Other Properties Related to Output: Classification scores represent the confidence that the input data is poisoned or not.

Software Integration:

Runtime Engine(s):

• NeMo Curator: https://github.com/NVIDIA/NeMo-Curator
  
• Aegis: https://huggingface.co/nvidia/Aegis-AI-Content-Safety-LlamaGuard-Defensive-1.0
  

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
  
• NVIDIA Hopper
  

Preferred Operating System(s):

• Linux
  
• Windows
  

Model Version(s):

v1.0

Training, Testing, and Evaluation Datasets:

Data Collection Method by Dataset:

• Synthetic
  
• Hybrid: derived, open-source
  

Labeling Method by Dataset:

• Synthetic
  

Evaluation Benchmarks:

Instruction Data Guard is evaluated based on two overarching criteria:

• Success on identifying LLM poisoning attacks, after the model was trained on examples of the attacks.
  
• Success on identifying LLM poisoning attacks, but without training on examples of those attacks, at all.
  

Success is defined as having an acceptable catch rate (recall scores for each attack) over a high specificity score (ex. 95%). Acceptable catch rates need to be high enough to identify at least several poisoned records in the attack.

Inference:

Engine: NeMo Curator and Aegis
Test Hardware:

• A100 80GB GPU
  

How to Use in NeMo Curator:

The inference code is available on NeMo Curator's GitHub repository.
Check out this example notebook to get started.

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here.