NVIDIA
NVIDIA
NVIDIA MTBench
Container
NVIDIA
NVIDIA
NVIDIA MTBench

NVIDIA NeMo Evaluator-compatible container with MTBench support. Based on FastChat available at: https://github.com/lm-sys/FastChat

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
org.opencontainers.image.licenses=3rd_party_licenses.txt
08/04/2025 3:01 PM UTC
sha256:f2da50c52dce1db06e060105e543cc03ebf27e877465aeff69ab546ccaf2f21aCOPY
3rd_party_licenses.txt .
08/04/2025 3:01 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
THIRD_PARTY_LICENSES_FILE=3rd_party_licenses.txt
08/04/2025 3:01 PM UTC
sha256:bdc428cc7f01ba5310647873067190c84b237d9f3b2e350bd97a2a5549b915c3RUN
pip uninstall -y nvidia-eval-commons-internal
08/04/2025 3:01 PM UTC
sha256:b62375b84aaa520641ddb77ce37b31848b59dd9f7959292632e7a14cb117ef05COPY
tests/* /workspace/tests/
08/04/2025 1:44 PM UTC
sha256:9f25223ad83dd8b8f1a6adaa1b9d0cf0408a2660a2cf685ea23432969fd8d06cRUN
CI_COMMIT_SHA=360de12fe9a031dd0475ee807b1f04531e9917cb EVAL_WHEEL_VERSION=25.07.1 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple printf '#!/bin/bash\ncore-evals-mtbench "$@"' > /usr/bin/nv-eval &&
  chmod +x /usr/bin/nv-eval &&
  printf '#!/bin/bash\ncore-evals-mtbench "$@"' > /usr/bin/nv_eval &&
  chmod +x /usr/bin/nv_eval
08/04/2025 1:44 PM UTC
sha256:b114a51acaa56eec8e220e0f7bfeece5f3717a7702a252e35a2b447e88bd6493RUN
CI_COMMIT_SHA=360de12fe9a031dd0475ee807b1f04531e9917cb EVAL_WHEEL_VERSION=25.07.1 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install nvidia_mtbench_evaluator==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
08/04/2025 1:44 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
OPENAI_SCOPE=azureopenai-readwrite
08/04/2025 1:44 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
OPENAI_MODEL_VERSION=2024-06-01
08/04/2025 1:44 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
OPENAI_MODEL_URL=https://prod.api.nvidia.com/llm/v1/azure/
08/04/2025 1:44 PM UTC