NVIDIA
NVIDIA MTBench
Container
NVIDIA
NVIDIA MTBench

NVIDIA NeMo Evaluator-compatible container with MTBench support. Based on FastChat available at: https://github.com/lm-sys/FastChat

  • LayerLabelCreated
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
    org.opencontainers.image.licenses=3rd_party_licenses.txt
    10/30/2025 11:49 AM UTC
    sha256:c234614c262d2628b0c1056c8b596fc4dcbb066eebe26ec97445c0bf69d22d45COPY
    README.md README.md
    10/30/2025 11:49 AM UTC
    sha256:f58ebae6cc03b37bc19980a38f5224ec6dcc69c7a5d3bb42098e28e4a2680f3dCOPY
    3rd_party_licenses.txt .
    10/30/2025 11:49 AM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    README_FILE=README.md
    10/30/2025 11:49 AM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    THIRD_PARTY_LICENSES_FILE=3rd_party_licenses.txt
    10/30/2025 11:49 AM UTC
    sha256:d0f73a09a63f831829b4e9f7f24f3d174cac4506d75ae9c756953db58de01b36RUN
    pip uninstall -y nemo-evaluator-internal
    10/30/2025 11:49 AM UTC
    sha256:3c42196afb1a88bd9d9a3fd69734ef6ee7b65aed02ea2d1bc9db08a04c746707COPY
    tests/* /workspace/tests/
    10/29/2025 8:22 PM UTC
    sha256:805f4d1e6032774976caf65605d7013019c34b339743dbc6c31eda6dbeaf3af4RUN
    CI_COMMIT_SHA=fc8906b9f7ea5881368f032ca1cb4f119bd513a6 EVAL_WHEEL_VERSION=25.10 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install nvidia_mtbench_evaluator==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
    10/29/2025 8:22 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
    OPENAI_SCOPE=azureopenai-readwrite
    10/29/2025 8:22 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
    OPENAI_MODEL_VERSION=2024-06-01
    10/29/2025 8:22 PM UTC