BodyPoseNet Model Card

Model Overview

Description:

BodyPoseNet predicts the skeleton for every person in a given input image. The model predicts 18 key points including:

nose,
neck,
right shoulder,
right elbow,
right wrist,
left shoulder,
left elbow,
left wrist,
right hip,
right knee,
right ankle,
left hip,
left knee,
left ankle,
right eye,
left eye,
right ear,
left ear

This model is ready for commercial use.

References:

Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, Yaser Sheikh (2017). Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

Using TAO Pre-trained Models

Get TAO Container
Get other purpose-built models from the NGC model registry:\n
- TrafficCamNet
- PeopleNet
- PeopleNet-Transformer
- DashCamNet
- FaceDetectIR
- VehicleMakeNet
- VehicleTypeNet
- PeopleSegNet
- PeopleSemSegNet
- License Plate Detection
- License Plate Recognition
- PoseClassificationNet
- Facial Landmark
- FaceDetect
- 2D Body Pose Estimation
- ActionRecognitionNet
- People ReIdentification
- PointPillarNet
- CitySegFormer
- Retail Object Detection
- Retail Object Embedding
- Optical Inspection
- Optical Character Detection
- Optical Character Recognition
- PCB Classification
- PeopleSemSegFormer
\n

Model Architecture:

Architecture Type: Convolution Neural Network (CNN)
\nNetwork Architecture: Visual Geometry Group (VGG)

The BodyPoseNet models described in this card are used for multi-person human pose estimation network, which aims to predict the skeleton for every person in a given input image which consists of keypoints and the connections between them. This follows a single shot bottom-up methodology and there is no need for a person detector. Hence, the compute does not scale linearly with the number of people in the scene.\nThe pose / skeleton output is commonly used as input for applications like activity/gesture recognition, fall detection, posture analysis, among others.

The default model predicts 18 keypoints including nose, neck, right_shoulder, right_elbow, right_wrist, left_shoulder, left_elbow, left_wrist, right_hip, right_knee, right_ankle, left_hip, left_knee, left_ankle, right_eye, left_eye, right_ear, left_ear.

\n\n

\n Fig 1. Example illustration of BodyPoseNet output\n\n

Input:

Input Type(s): Image
\nInput Format(s): Red, Green, Blue (RGB)
\nInput Parameters: 3D
\nOther Properties Related to Input: RGB Format : H X W x 3, No minimum bit depth, alpha, or gamma

The images are pre-processed to handle normalization, resizing while maintaining the aspect ratio etc.

Output:

Output Type(s): Tensors
\nOutput Format: N-Dimentional array
\nOutput Parameters: 3D
\nOther Properties Related to Output:
\nNetwork outputs two tensors: confidence maps (H1' x W1' x C) and part affinity fields (H2' x W2' x P). After NMS and bipartite graph matching, we obtain final results with M x N X 3\nwhere

N is the number of keypoints.
M is the number of humans detected in the image.
C is the number of confidence map channels - corresponds to number of keypoints + background
P is the number of part affinity field channels - corresponds to the (2 x number of edges used in the skeleton)
H1', W1' are the height and width of the output confidence maps respectively
H2', W2' are the height and width of the output part affinity fields respectively

Software Integration:

Runtime Engine(s):

TAO - 5.2

Supported Hardware Architecture(s):

Ampere
Jetson
Hopper
Lovelace
Pascal
Turing
Volta

Supported Operating System(s):

Linux
Linux 4 Tegra

Model Version(s):

trainable_v1.0 - this pretrained model is intended to be used for finetuning on custom datasets using TAO.
deployabale_v1.0 - this deployable model is intended to run on the inference pipeline. There are INT8 calibration files provided for three resolutions including 224x320, 288x384 and 320x448. These calibration files are generated for TensorRT 7.
deployabale_v1.0.1 - this deployable model is intended to run on the inference pipeline. There are INT8 calibration files provided for three resolutions including 224x320, 288x384 and 320x448. These calibration files are generated for TensorRT 8.

Training & Evaluation:

Training Dataset:

Link: https://storage.googleapis.com/openimages/web/download_v7.html
\nData Collection Method by dataset:

Unknown

Labeling Method by dataset:

Unknown

Properties:
\nRoughly 400,000 images and 7,000 validation images across thousands of classes as defined by Google OpenImages Version Three (3) dataset. Most of the human verifications have been done with in-house annotators at Google. A smaller part has been done with crowd-sourced verification from Image Labeler: Crowdsource app, g.co/imagelabeler.

Evaluation Dataset:

Link: https://cocodataset.org/#download
\nData Collection Method by dataset:

Unknown

Labeling Method by dataset:

Unknown

Properties:
\nCOCO validation sets, containing more than 200,000 images and 250,000 person instances labeled with keypoints.

Methodology and KPI

The KPI for the evaluation data are reported in the table below.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Metric	IoU	Area	Score
AP	0.50:0.95	all	56.2
AP	0.5	all	79.3
AP	0.50:0.95	medium	57.2
AP	0.50:0.95	large	54.9

Inference:

Engine: Tensor(RT)
\nTest Hardware:

Jetson AGX Xavier
Xavier NX
Orin
Orin NX
NVIDIA T4
Ampere GPU
A2
A30
L4
T4
DGX H100
DGX A100
DGX H100
L40
JAO 64GB
Orin NX16GB
Orin Nano 8GB

The inference performance is measured for INT8 precision and for a input dimension of 288x384. The inference performance runs with trtexec on Jetson Nano, AGX Xavier, Xavier NX and NVIDIA T4 GPU. The Jetson devices run at Max-N configuration for maximum system performance. The end-to-end performance with streaming video data might slightly vary depending on use cases of applications.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Device	Precision	Batch_size	FPS	Latency
Nano	INT8	8	5	200.0ms
NX	INT8	8	93	10.71ms
Xavier	INT8	8	160	6.25ms
T4	INT8	8	555	1.80ms

How to use this model

The models in this page can only be used with Train Adapt Optimize (TAO) Toolkit. TAO provides a simple command line interface to train a deep learning model for body pose estimation.

Primary use case for this model is to detect human poses in a given RGB image. BodyPoseNet is commonly used for activity/gesture recognition, fall detection, posture analysis etc.

\n
Install the NGC CLI from ngc.nvidia.com
\n
\n
Configure the NGC CLI using the following command
\n

ngc config set\n

To view all the models that are supported in TAO:

ngc registry model list nvidia/tao/bodyposenet:*\n

To download the model:

--dest \n\">

ngc registry model download-version nvidia/tao/bodyposenet:<template> --dest <path>\n

Technical blogs

Read the 2 part blog on training and optimizing 2D body pose estimation model with TAO - Part 1 | Part 2
Read the technical tutorial on how PeopleNet model can be trained with custom data using Transfer Learning Toolkit

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Promise and the Explainability, Bias, Safety & Security, and Privacy Subcards.

","resourceVersions",[212,213,214,215,216],"resourceVersionsTotal","hasMoreVersions","versionMetadata",{"_185":22,"_186":187,"_49":188,"_189":-5,"_190":191,"_192":193,"_194":195,"_196":16,"_197":31,"_198":-5,"_199":31,"_200":-5,"_201":31,"_202":-5,"_203":31,"_204":-5,"_205":31,"_206":-5,"_207":31,"_208":-5,"_209":31,"_210":211},"versionNotFound","isLoggedIn","latestVersionPath","/orgs/nvidia/tao/models/bodyposenet/-","version","imageReference","cliDownloadCommand","ngc registry model download-version nvidia/tao/bodyposenet:deployable_v1.0.1","helmFetchCommand","viewAllVersionsPath","/orgs/nvidia/tao/models/bodyposenet/deployable_v1.0.1/version-history","accessCheckResult",{"_184":107},"hasRelatedCollections","context",{"_180":181,"_182":183},[],[],[174,175,176,177,178,179],{"_45":60,"_47":71,"_49":62,"_51":63,"_53":72,"_55":56},{"_45":60,"_47":69,"_49":62,"_51":63,"_53":70,"_55":56},{"_45":60,"_47":67,"_49":62,"_51":63,"_53":68,"_55":56},{"_45":60,"_47":61,"_49":62,"_51":63,"_53":64,"_55":56},{"_45":57,"_47":58,"_49":123,"_51":-5,"_53":-5,"_55":56},{"_45":46,"_47":48,"_49":50,"_51":52,"_53":54,"_55":56},"orgName","nvidia","teamName","tao","allowed","versionId","description","","UPLOAD_COMPLETE","releaseType","createdDate",["D",1635264128838],"updatedDate",["D",1740840142150],"totalSizeInBytes",67198981,"totalFileCount","isSigned","malwareScanStatus","isScanned","releaseNotes","hasReleaseNotes","setup","hasSetup","quickStartGuide","hasQuickStartGuide","advanced","hasAdvanced","performance","hasPerformance","policies",[],{"_81":224,"_83":225,"_189":-5,"_197":107,"_199":31,"_210":223,"_201":31},{"_81":222,"_83":222,"_189":-5,"_197":107,"_199":31,"_210":223,"_201":31},{"_81":22,"_83":22,"_189":-5,"_197":31,"_199":31,"_210":221,"_201":31},{"_81":219,"_83":219,"_189":-5,"_197":31,"_199":31,"_210":220,"_201":31},{"_81":217,"_83":217,"_189":-5,"_197":31,"_199":31,"_210":218,"_201":31},"deployable_v1.0",[],"trainable_v1.0",[],[],"deployable_onnx_v1.0.1",[],"-","Latest (deployable_onnx_v1.0.1)","nvidia/tao/bodyposenet","key","nvidia/tao/models/bodyposenet","resourceType","models","resourceId","bodyposenet","shortDescription","Detect body pose from an image.","# BodyPoseNet Model Card\n\n# Model Overview\n\n## Description:\nBodyPoseNet predicts the skeleton for every person in a given input image. The model predicts 18 key points including:\n- nose, \n- neck, \n- right shoulder, \n- right elbow, \n- right wrist, \n- left shoulder, \n- left elbow, \n- left wrist, \n- right hip, \n- right knee, \n- right ankle, \n- left hip, \n- left knee, \n- left ankle, \n- right eye, \n- left eye, \n- right ear, \n- left ear\n\nThis model is ready for commercial use.\n\n## References:\n- Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, Yaser Sheikh (2017). Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.\n\n### Using TAO Pre-trained Models \n- Get [TAO Container](https://ngc.nvidia.com/catalog/containers/nvidia:tao:tao-toolkit)\n- Get other purpose-built models from the NGC model registry:\n - [TrafficCamNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:trafficcamnet)\n - [PeopleNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:peoplenet)\n - [PeopleNet-Transformer](https://ngc.nvidia.com/catalog/models/nvidia:tao:peoplenet_transformer)\n - [DashCamNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:dashcamnet)\n - [FaceDetectIR](https://ngc.nvidia.com/catalog/models/nvidia:tao:facedetectir)\n - [VehicleMakeNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:vehiclemakenet)\n - [VehicleTypeNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:vehicletypenet)\n - [PeopleSegNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:peoplesegnet)\n - [PeopleSemSegNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:peoplesemsegnet)\n - [License Plate Detection](https://ngc.nvidia.com/catalog/models/nvidia:tao:lpdnet)\n - [License Plate Recognition](https://ngc.nvidia.com/catalog/models/nvidia:tao:lprnet)\n - [PoseClassificationNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:poseclassificationnet)\n - [Facial Landmark](https://ngc.nvidia.com/catalog/models/nvidia:tao:fpenet)\n - [FaceDetect](https://ngc.nvidia.com/catalog/models/nvidia:tao:facenet)\n - [2D Body Pose Estimation](https://ngc.nvidia.com/catalog/models/nvidia:tao:bodyposenet)\n - [ActionRecognitionNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:actionrecognitionnet)\n - [People ReIdentification](https://ngc.nvidia.com/catalog/models/nvidia:tao:reidentificationnet)\n - [PointPillarNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pointpillarnet)\n - [CitySegFormer](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/citysemsegformer)\n - [Retail Object Detection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/retail_object_detection)\n - [Retail Object Embedding](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/retail_object_recognition)\n - [Optical Inspection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/optical_inspection)\n - [Optical Character Detection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/ocdnet)\n - [Optical Character Recognition](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/ocrnet)\n - [PCB Classification](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pcb_classification)\n - [PeopleSemSegFormer](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplesemsegformer)\n\n## Model Architecture: \n**Architecture Type:** Convolution Neural Network (CNN)
\n**Network Architecture:** Visual Geometry Group (VGG)
\n\nThe BodyPoseNet models described in this card are used for multi-person human pose estimation network, which aims to predict the skeleton for every person in a given input image which consists of keypoints and the connections between them. This follows a single shot bottom-up methodology and there is no need for a person detector. Hence, the compute does not scale linearly with the number of people in the scene.\nThe pose / skeleton output is commonly used as input for applications like activity/gesture recognition, fall detection, posture analysis, among others.\n\nThe default model predicts 18 keypoints including *nose, neck, right_shoulder, right_elbow, right_wrist, left_shoulder, left_elbow, left_wrist, right_hip, right_knee, right_ankle, left_hip, left_knee, left_ankle, right_eye, left_eye, right_ear, left_ear*.\n\n

Fig 1. Example illustration of BodyPoseNet output

\n\n## Input:\n**Input Type(s):** Image
\n**Input Format(s):** Red, Green, Blue (RGB)
\n**Input Parameters:** 3D
\n**Other Properties Related to Input:** RGB Format : H X W x 3, No minimum bit depth, alpha, or gamma
\n\nThe images are pre-processed to handle normalization, resizing while maintaining the aspect ratio etc.\n\n\n## Output:\n**Output Type(s):** Tensors
\n**Output Format:** N-Dimentional array
\n**Output Parameters:** 3D
\n**Other Properties Related to Output:**
\nNetwork outputs two tensors: confidence maps (H1' x W1' x C) and part affinity fields (H2' x W2' x P). After NMS and bipartite graph matching, we obtain final results with M x N X 3 \nwhere\n- N is the number of keypoints.\n- M is the number of humans detected in the image.\n- C is the number of confidence map channels - corresponds to number of keypoints + background\n- P is the number of part affinity field channels - corresponds to the (2 x number of edges used in the skeleton)\n- H1', W1' are the height and width of the output confidence maps respectively\n- H2', W2' are the height and width of the output part affinity fields respectively \n\n## Software Integration:\n**Runtime Engine(s):** \n* TAO - 5.2
\n\n**Supported Hardware Architecture(s):**
\n* Ampere
\n* Jetson
\n* Hopper
\n* Lovelace
\n* Pascal
\n* Turing
\n* Volta
\n\n**Supported Operating System(s):**
\n* Linux
\n* Linux 4 Tegra
\n\n## Model Version(s): \n- **trainable_v1.0** - this pretrained model is intended to be used for finetuning on custom datasets using TAO.\n- **deployabale_v1.0** - this deployable model is intended to run on the inference pipeline. There are INT8 calibration files provided for three resolutions including 224x320, 288x384 and 320x448. These calibration files are generated for TensorRT 7.\n- **deployabale_v1.0.1** - this deployable model is intended to run on the inference pipeline. There are INT8 calibration files provided for three resolutions including 224x320, 288x384 and 320x448. These calibration files are generated for TensorRT 8.\n\n# Training & Evaluation: \n\n## Training Dataset:\n\n**Link:** https://storage.googleapis.com/openimages/web/download_v7.html
\n**Data Collection Method by dataset:**
\n* Unknown
\n\n**Labeling Method by dataset:**
\n* Unknown
\n\n**Properties:**
\nRoughly 400,000 images and 7,000 validation images across thousands of classes as defined by [Google OpenImages Version Three (3) dataset](https://github.com/openimages/dataset/blob/main/READMEV3.md). Most of the human verifications have been done with in-house annotators at Google. A smaller part has been done with crowd-sourced verification from Image Labeler: [Crowdsource app](https://play.google.com/store/apps/details?id=com.google.android.apps.village.boond&pli=1), [g.co/imagelabeler](http://g.co/imagelabeler).\n\n## Evaluation Dataset:\n**Link:** https://cocodataset.org/#download
\n**Data Collection Method by dataset:**
\n* Unknown
\n\n**Labeling Method by dataset:**
\n* Unknown
\n\n**Properties:**
\nCOCO validation sets, containing more than 200,000 images and 250,000 person instances labeled with keypoints. \n\n### Methodology and KPI \nThe KPI for the evaluation data are reported in the table below.\n| Metric | IoU | Area | Score\n| ---- | ---- | ---- | ---- |\n| AP | 0.50:0.95 | all | 56.2 |\n| AP | 0.5 | all | 79.3 |\n| AP | 0.50:0.95 | medium | 57.2 |\n| AP | 0.50:0.95 | large | 54.9 |\n\n## Inference:\n**Engine:** Tensor(RT)
\n**Test Hardware:**
\n- Jetson AGX Xavier\n- Xavier NX\n- Orin\n- Orin NX\n- NVIDIA T4\n- Ampere GPU\n- A2\n- A30\n- L4\n- T4\n- DGX H100\n- DGX A100\n- DGX H100\n- L40\n- JAO 64GB\n- Orin NX16GB\n- Orin Nano 8GB\n\nThe inference performance is measured for INT8 precision and for a input dimension of 288x384. The inference performance runs with [`trtexec`](https://github.com/NVIDIA/TensorRT/tree/master/samples/trtexec) on Jetson Nano, AGX Xavier, Xavier NX and NVIDIA T4 GPU. The Jetson devices run at Max-N configuration for maximum system performance. The end-to-end performance with streaming video data might slightly vary depending on use cases of applications.\n\n| Device | Precision | Batch_size | FPS | Latency |\n| ---- | ---- | ---- | --- | --- |\n| Nano | INT8 | 8 | 5 | 200.0ms |\n| NX | INT8 | 8 | 93 | 10.71ms |\n| Xavier | INT8 | 8 | 160 | 6.25ms |\n| T4 | INT8 | 8 | 555 | 1.80ms |\n\n## How to use this model \n\nThe models in this page can only be used with [Train Adapt Optimize (TAO) Toolkit](https://developer.nvidia.com/tao-toolkit). TAO provides a simple command line interface to train a deep learning model for body pose estimation.\n\nPrimary use case for this model is to detect human poses in a given RGB image. BodyPoseNet is commonly used for activity/gesture recognition, fall detection, posture analysis etc.\n\n1. Install the NGC CLI from ngc.nvidia.com\n\n2. Configure the NGC CLI using the following command\n\n ```sh\n ngc config set\n ```\n\n3. To view all the models that are supported in TAO:\n\n ```sh\n ngc registry model list nvidia/tao/bodyposenet:*\n ```\n\n4. To download the model:\n\n ```sh\n ngc registry model download-version nvidia/tao/bodyposenet:

BodyPoseNet Model Card

Model Overview

Description:

References:

Using TAO Pre-trained Models

Model Architecture:

Input:

Output:

Software Integration:

Model Version(s):

Training & Evaluation:

Training Dataset:

Evaluation Dataset:

Methodology and KPI

Inference:

How to use this model

Technical blogs

Suggested reading

Ethical Considerations:

Name	Size	Updated	Actions
int8_calibration_224_320.txt	2.24 KB	March 1, 2025 UTC	curl wget Download
int8_calibration_288_384.txt	2.24 KB	March 1, 2025 UTC	curl wget Download
int8_calibration_320_448.txt	2.24 KB	March 1, 2025 UTC	curl wget Download
labels.txt	1 KB	March 1, 2025 UTC	curl wget Download
model.etlt	64.08 MB	March 1, 2025 UTC	curl wget Download