ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
MixNet
Model
NVIDIA
NVIDIA
MixNet

MixNet is an optical character detection model that aims to detect text in images.

MixNet Overview

Model Overview

Description

MixNet is an optical character detection model that aims to detect text in images. It is a deep learning model designed for accurate detection of challenging scene text in natural images, particularly focusing on small and irregularly positioned text under diverse lighting and orientations. This model is ready for commercial/non-commercial use.

License/Terms of Use

License to use these models is covered by the NVIDIA Open Model License. By downloading the model, you accept the terms and conditions of these licenses.

Deployment Geography

Global

Use Case

This model can be used in any computer vision application, which aims to detect text characters in images.

Release Date

NGC [06/30/2025]

Model Architecture

Architecture Type: Convolution Neural Network + Transformer Block. Network Architecture: This model was developed based on the MixNet. MixNet is a cutting-edge model for scene text detection, notable for its hybrid CNN-Transformer design and strong benchmark performance. Main features include FSNet(Feature Shuffle Network) and CTBlock(Central Transformer Block). FSNet serves as the backbone, introducing a novel feature shuffling strategy to exchange features across multiple scales. CTBlock exploits the 1D manifold constraint of scene text by focusing on center line features, which helps in distinguishing closely located small text better than contour-based methods.

Input

  • Input Type: Image
  • Input Formats: Red, Green, Blue (RGB)
  • Input Parameters: Two-Dimensional (2D)
  • Other Properties Related to Input: Width and height are multiples of 32.

Output

  • Input Type: Image
  • Input Formats: Red, Green, Blue (RGB)
  • Output Parameters: Two-Dimensional (2D)
  • Other Properties Related to Output: Spatial maps or coordinates indicating detected text regions.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s):

  • TAO 5.5.0

Supported Hardware Microarchitecture Compatibility:

  • NVIDIA Ampere
  • NVIDIA Jetson
  • NVIDIA Hopper
  • NVIDIA Volta

[Preferred/Supported] Operating System(s):

  • Linux

Model versions:

  • deployable_v1.0 - Models deployable with MixNet.

Training and Evaluation Datasets

  • The total size: 77K images
  • Total number of datasets: 3 training datasets, 2 evaluation datasets
  • Dataset partition: training and evaluation are different datasets

Training Datasets

Link:

Data Collection Method by dataset:

  • Hybrid: Automated, Synthetic, Human

Labeling Method by dataset:

  • Hybrid: Automated, Synthetic

Properties:
The MixNet pretrained model was trained on three datasets, totaling about 77K images. One is ICDAR2015 training dataset, we oversample it to 20K images. Another is Uber-Text dataset. We fiter the train_4Kx4K and train_1Kx1K datasets to make sure each image has word-only text instead of sentence text. We get 29,992 images from train_4Kx4K dataset and 12,312 images from train_1Kx1K dataset. The third dataset is a synthetic dataset based on the SynthText background images. They are official background images which every user can generate synthetic texts on them. We generate 7,674 images which have large texts and 7,331 images which have small single texts.

Evaluation Datasets

Link:

Data Collection Method by dataset:

  • Hybrid: Automated, Synthetic, Human

Labeling Method by dataset:

  • Hybrid: Automated, Synthetic

Properties:
We evaluate the MixNet model on two different datasets: word-only images from Uber-Text 1Kx1K dataset, ICDAR15 test dataset.

dataseteval image numbers
ICDAR15500
Uber-Text7461

Performance

Methodology and KPI

In text characters detection models, the F1-score is a key performance metric that measures how well the model balances precision (the proportion of correct positive detections among all positive detections) and recall (the proportion of actual positives that are correctly detected). The KPI for the evaluation data are reported below.

modeldatasetF1-score
MixNetICDAR1586.5%
MixNetUber-Text88.0%
ocdnet-vitICDAR1585.3%
ocdnet-vitUber-Text86.0%
ocdnet_deformable_resnet50Uber-Text82.2%
ocdnet_deformable_resnet18Uber-Text81.1%

Inference

Acceleration Engine: TensorRT

Test Hardware:

  • A40

The inference uses FP16 precision. The inference performance runs against a deployable model with trtexec on NVIDIA A40 GPU. The data is for inference-only performance. The end-to-end performance with streaming video data might vary slightly depending on the application's use case.

modelgpuprecisioninput-sizeBatch-sizeFPS
MixNetA40FP161024x1024152
MixNetA40FP16960x960159

To find more performance data about other kinds of detection models, please refer to ocdnet.

How to use this model

This model is suggested to be used with NVIDIA Hardware and Software. For Hardware, the model can run on any NVIDIA GPU including NVIDIA Jetson devices. For software, this model can run with NVIDIA-Optical-Character-Detection-and-Recognition-Solution or onnxruntime.

There are two types of models provided:

  • deployable (unpruned)

The deployable model is in onnx format. The deployable models can be deployed in TensorRT and nvOCDR, or onnxruntime.

Instructions to use the model with nvOCDR

Please refer to the C++ Sample.

Limitations

Restricted usage in different fields:

The model was trained on the ICDAR2015, Uber-text, and augmented SynthText datasets. Its generalization performance may be inadequate in scenarios that differ significantly from the training data, such as images containing numerous small text elements on PCB boards. In these cases, the model may struggle to accurately detect text. To address this limitation, augmenting the training dataset with PCB-specific images can enhance the model's ability to generalize to such challenging scenarios. Generally, achieving a better F1-score in a specific domain requires more data.

Reference

Citations

  • Zeng, Yu-Xiang, et al. "MixNet: Toward Accurate Detection of Challenging Scene Text in the Wild." arXiv preprint arXiv:2308.12817, 2023.

Using TAO Pre-trained Models

Technical blogs

Suggested reading

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns here.

Publisher
NVIDIA
NVIDIA
Latest Versiondeployable_v1.0
UpdatedAugust 5, 2025 UTC
Compressed Size112.63 MB
Labels
AIEnglishNSPECT-FY5H-V6CONSPECT-UAGN-IT0BObject DetectionSmart Cities / SpacesTAO Toolkit