NVIDIA
NVIDIA
NVIDIA MTBench
Container
NVIDIA
NVIDIA
NVIDIA MTBench

NVIDIA NeMo Evaluator-compatible container with MTBench support. Based on FastChat available at: https://github.com/lm-sys/FastChat

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
org.opencontainers.image.licenses=3rd_party_licenses.txt
09/16/2025 6:05 AM UTC
sha256:d0d75945553cab50ae4e6a9a57bb94307c880fa8e3ca5915d6303045ff0b140eCOPY
README.md README.md
09/16/2025 6:05 AM UTC
sha256:b7666b10979dbaad5acab62dc9178940822d6d957f1427e384c29bbef8dc6501COPY
3rd_party_licenses.txt .
09/16/2025 6:05 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
README_FILE=README.md
09/16/2025 6:05 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
THIRD_PARTY_LICENSES_FILE=3rd_party_licenses.txt
09/16/2025 6:05 AM UTC
sha256:025f9356ebde7364d395b642b995a35003c298b93fc9a31d71c394d940c5d659RUN
pip uninstall -y nvidia-eval-commons-internal
09/16/2025 6:05 AM UTC
sha256:3f44b512bccfdaeb825a220233c981aa9c02b606c822c3ab569b8630a09704fcCOPY
tests/* /workspace/tests/
09/15/2025 4:30 PM UTC
sha256:b2302d7fdff6575fd7efb0dac1a3902dad739a7779c8895d45480730fb91a7ebRUN
CI_COMMIT_SHA=001b448ca8a445a7688a02ec1a7aa7424613e7b7 EVAL_WHEEL_VERSION=25.08.1 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple printf '#!/bin/bash\ncore-evals-mtbench "$@"' > /usr/bin/nv-eval &&
  chmod +x /usr/bin/nv-eval &&
  printf '#!/bin/bash\ncore-evals-mtbench "$@"' > /usr/bin/nv_eval &&
  chmod +x /usr/bin/nv_eval
09/15/2025 4:30 PM UTC
sha256:e012376260f4c7cf0a7483a5c19c3ebb9894c075ae890a66cb1e1e0ee2bdf37dRUN
CI_COMMIT_SHA=001b448ca8a445a7688a02ec1a7aa7424613e7b7 EVAL_WHEEL_VERSION=25.08.1 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install nvidia_mtbench_evaluator==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
09/15/2025 4:30 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
OPENAI_SCOPE=azureopenai-readwrite
09/15/2025 4:30 PM UTC