Image Classification is a popular computer vision technique in which an image is classified into one of the designated classes based on the image features. The model described in this card detects missing Printed Circuit Board (PCB) component defects using component level images extracted from a PCB.
NV-Dinov2 is a visual foundational model trained on NVIDIA proprietary large scale dataset. Dinov2 is a self-supervised learning method that uses a combination of two SSL techniques : DINO and iBOT. These models could greatly simplify the use of images in any system by producing all purpose visual features, i.e., features that work across image distributions and tasks without finetuning. Trained on large curated datasets, our model has learnt robust fine-grained representation useful for localization and classification tasks. This model can be used as a foundation model for a variety of downstream tasks with few labeled examples. For more details on the method please refer: Dinov2.
The model in this model card uses the NVDinov2 pre-trained ViT-L backbone and fine-tunes a Linear Regression Classifier on an NVIDIA proprietary PCB (printed circuit board) dataset.
PCBClassification model was fine-tuned on a proprietary dataset with more than 19600 images of individual components extracted from 71 PCB boards. The training dataset consists of a mix of components (Resistors, Capacitors, Inductors, etc) from different PCBs which can be present or absent (defective) in the images.
Dataset | Total # of images | Training images | Testing images |
---|---|---|---|
NV-PCB Internal Dataset | 19605 | 15573 | 2079 |
The dataset distribution is represented as under:
Dataset | Total # of images | # Component present images | # Component missing images |
---|---|---|---|
NV-PCB Internal Dataset | 19605 | 14226 | 5379 |
The components that are present in the image are additionally labelled with their component types divided into 11 categories. All these are treated as a single class (component-present) during training/evaluation. Missing components of all types have the same "missing" label.
The inference performance of PCBClassification model was measured against more than 2000 proprietary images. The component level images are varying resolution images padded/resized to 224x224 pixels before passing to the PCBClassification model.
The performance of the PCBClassification Model is mainly measured using accuracy, which is the proportion of correct predictions (all classes) made by the model out of all predictions.
Model | Model Architecture | Testing Images | Accuracy |
---|---|---|---|
PCBClassification Model | NV-Dinov2 | 2079 | 0.99 |
The inference is run on the provided unpruned model at FP16 precision. The inference performance is run using trtexec
. The end-to-end performance with streaming video data might slightly vary depending on other bottlenecks in the hardware and software.
NVDinoV2 (224x224 resolution)
Platform | BS | FPS |
---|---|---|
Orin NX 16GB | 16 | 31.55 |
AGX Orin 64GB | 16 | 81.41 |
A2 | 16 | 72.7 |
T4 | 4 | 110.3 |
A30 | 16 | 461.0 |
L4 | 4 | 275.0 |
L40 | 8 | 579.0 |
A100 | 32 | 1031.0 |
H100 | 64 | 2500.6 |
These models need to be used with NVIDIA Hardware and Software. For Hardware, the models can run on any NVIDIA GPU including NVIDIA Jetson devices. These models can only be used with DeepStream SDK.
The primary use case intended for these models is detecting missing component defects in RGB PCB component level images. The model can be used to classify objects from photos and videos by using appropriate video or image decoding and pre-processing. The model is a binary classifier which predicts whether a component is present or missing.
The models are intended for easy edge deployment using DeepStream SDK. DeepStream provides the facilities to create efficient video analytic pipelines to capture, decode, and pre-process the data before running inference.
RGB Image of dimensions: 224 X 224 X 3 (W x H x C)
Channel Ordering of the Input: NCHW, where N = Batch Size, C = number of channels (3), H = Height of images (224), W = Width of the images (224)
The output is a binary label assigned to the image by the PCBClassification model. Category labels (2 classes - Missing/Present) for each input image containing a single component.
Channel Ordering of the Input: NC, where N = Batch Size, C = number of classes (2).
Here are two sample images representing a Capacitor - [Present(left), Missing(right)]:
To create an end-to-end video analytics application, deploy this model with DeepStream SDK. DeepStream SDK is a streaming analytics toolkit to accelerate deployment of AI-based video analytics applications. The model can be integrated directly into deepstream by following the instructions mentioned below.
To deploy these models with DeepStream 6.1, please follow the instructions below:
Download and install DeepStream SDK. The installation instructions for DeepStream are provided in DeepStream development guide.
/opt/nvidia/deepstream
is the default DeepStream installation directory. This path will be different if you are installing in a different directory.
Two extra files are required which are provided in NVIDIA-AI-IOT.
Missing;Present
configs/multi_task_tao/pgie_multi_task_tao_config.txt
gpu-id=0
net-scale-factor=0.01735207357279195
offsets=123.675;116.28;103.53
model-color-format=0
labelfile-path=/path/to/label/file.txt
onnx-file=/path/to/onnx/model
batch-size=1
network-mode=2
interval=0
gie-unique-id=1
network-type=1
scaling-filter=1
scaling-compute-hw=1
classifier-threshold=0.5
Run ds-tao-classifier
:
ds-tao-classifier -c configs/multi_task_tao/pgie_multi_task_tao_config.txt -i file:///path/to/img.jpg
Documentation to deploy with DeepStream is provided in "Deploying to DeepStream" chapter of TAO User Guide.
The PCBClassification model was trained on RGB images in good lighting conditions. Therefore, images captured in dark lighting conditions or a monochrome image or IR camera image may not provide good detection results.
The PCBClassification model was not trained on fish-eye lense cameras or moving cameras. Therefore, the models may not perform well for warped images and images that have motion-induced or other blur.
License to use this model is covered by the Model EULA. By downloading the unpruned or pruned version of the model, you accept the terms and conditions of these licenses
NVIDIA PCBClassification model detects missing component defects using component level PCB images. NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.