What is the NeMo Gym Container?

NVIDIA NeMo™ Gym is an open-source library under the NVIDIA NeMo Framework for evaluating and improving models and agents using environments. An environment is the complete system an agent interacts with to complete a task: a dataset, agent harness, verifier, and per-task state. The same setup drives both evaluation and training.

The NeMo Gym container is a production-ready runtime for running that stack on GPU clusters and workstations. It ships the full gym CLI, all built-in environments, and a pinned vLLM inference stack so you can collect rollouts and run local model serving without assembling dependencies yourself.

Use it for large-scale agent evaluation, reward profiling, and rollout collection. Pair it with NeMo RL when you need distributed post-training (GRPO, DPO, SFT) on the same environments.

Pull the Container

1. Select Get Container on this page.
2. Copy the image path for the tag you need.

# example
docker pull nvcr.io/nvidia/nemo-gym:v0.5.1

Refer to the NeMo Gym releases page for release notes and other available versions.

What You Get with the NVIDIA NeMo Gym Container

The container is an opinionated, reproducible runtime for NeMo Gym evaluation and rollout collection, not a minimal Python base image you extend by hand.

Platform & Runtime

• CUDA 13.2 / Ubuntu 24.04: GPU-ready base aligned with current NVIDIA stacks.
• Python 3.12 via uv: fast, reproducible dependency management.
• Entrypoint: gym: the full NeMo Gym CLI works immediately (gym env start, gym eval run, gym list, and the rest).
• Locked core install: the NeMo Gym runtime is installed from uv.lock; --locked fails rather than silently re-resolving, so builds and pulls stay reproducible.

Environments & CLI

• All 101 built-in environments: source, configs, and example data ship in the nemo-gym wheel; no separate environment checkout required to discover or wire configs.
• Composable microservice architecture: resources servers (verifiers + tools), agent harnesses, and model servers start from YAML and communicate over async HTTP, the same model as a local install.
• Scale-ready: designed for high-concurrency rollout collection (thousands of concurrent requests) with session isolation via cookies and bounded concurrency patterns.

Local Inference (vLLM)

• vLLM stack pre-installed: vllm==0.20.0 and flashinfer-python==0.6.8.post1.
• local_vllm_model: serve open models in-process on GPUs without a separate inference container.
• Pre-compiled FlashInfer kernels: the FlashInfer pin ships pre-built CUDA kernels, avoiding multi-minute JIT compilation on first generation.

Training Framework Integration

NeMo Gym produces rollouts and reward signals that downstream training frameworks consume. The container is the evaluation and rollout side of that pipeline:

For combined Gym and NeMo RL training jobs, see the RL Framework Compatibility table for container pairings by Gym version and model recipe.

Opt-In: Environment Dependencies (Prefetch)

The base image includes the core NeMo Gym runtime and every environment’s source and configs. It does not bake a separate virtual environment for all 101 environments by default. Many environments pull heavy dependencies (PyTorch, vLLM, domain-specific tools), and installing all of them in one image would mean 101 independent venvs and a very large, slow-to-build artifact.

Instead, per-environment venvs are opt-in at build time:

docker build \
  --build-arg NEMO_GYM_PREFETCH_CONFIGS="resources_servers/code_gen/configs/code_gen.yaml,responses_api_agents/simple_agent/configs/simple_agent.yaml" \
  -t nemo-gym:my-benchmarks \
  .

This runs the gym env prefetch step during the image build and prebuilds venvs only for the YAML configs you name. The model is one image per environment set you actually run: prefetch the configs for your eval or training job, not the full catalog.

Custom Base Images

Swap the platform base without forking the Gym Dockerfile:

docker build \
  --build-arg BASE_IMAGE=your-registry/your-hardened-base:tag \
  -t nemo-gym:custom-base \
  .

Use this for hardened images, UBI bases, or org-standard CUDA images while keeping the same NeMo Gym layer and prefetch behavior.

Getting Started

Inside the container (or after docker run with GPU access):

# Verify install
gym --version

# List available agents and environments
gym list

# Start servers for an environment (example)
gym env start --resources-server example_single_tool_call --model-type vllm_model

# Run evaluation
gym eval run --config resources_servers/example_single_tool_call/configs/example_single_tool_call.yaml

For step-by-step guides (quickstart, rollout collection, reward profiling, and cluster deployment), see the NeMo Gym documentation.

Explore source, configs, and examples on the NVIDIA-NeMo/Gym GitHub repository.

Documentation

More detail is in the NeMo Gym User Guide:

Developer Container

For features not yet in a tagged release, or to prefetch a custom environment set, build from source. Refer to the Docker build documentation in the NeMo Gym repository for:

• BASE_IMAGE overrides
• NEMO_GYM_PREFETCH_CONFIGS for build-time venv prefetch
• Multi-stage build targets and CI parity with NGC tags

Support & Community

NeMo Gym is actively developed with regular releases. Follow GitHub releases for new environments, CLI features, and container tags.

Need help?

• Report issues on GitHub Issues (include Gym version, container tag, and config paths)
• Read the docs

License

GOVERNING TERMS: The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.