NVIDIA
NVIDIA
NVIDIA LM-Evaluation-Harness
Container
NVIDIA
NVIDIA
NVIDIA LM-Evaluation-Harness

NVIDIA NeMo Evaluator-compatible container with LM-Evaluation-Harness support. Based on lm-evaluation-harness available at: https://github.com/EleutherAI/lm-evaluation-harness

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
org.opencontainers.image.licenses=3rd_party_licenses.txt
03/12/2026 9:49 AM UTC
sha256:dc4c431580905453e296fae37a337fdecc7708532608d6fc267615e04dafffd2COPY
README.md README.md
03/12/2026 9:49 AM UTC
sha256:bc956432f91c92537576d1c77ebdf10651b94af6e38ae677d46eeb227b42a8b1COPY
3rd_party_licenses.txt .
03/12/2026 9:49 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
README_FILE=README.md
03/12/2026 9:49 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
THIRD_PARTY_LICENSES_FILE=3rd_party_licenses.txt
03/12/2026 9:49 AM UTC
sha256:b5dec6fcf339a5cca51e4ec3a763fc3bab7b18000f0c2453455f560c8c8c5209RUN
pip uninstall -y nemo-evaluator-internal
03/12/2026 9:49 AM UTC
sha256:5fa2bcd45fe989490201686b7cddd83b79afa8a69688d3a70817ce98cac76fdbCOPY
--chown=1000:1000 --chmod=777 tests/* /workspace/tests/
03/11/2026 1:55 PM UTC
sha256:61321ddc45ea5a0d823113bc7a4df8b276bac9881c4b6710443a2e18f1d8731dRUN
CI_COMMIT_SHA=83a6bbd46e65be5bceb13b920aa2f90dd73ec155 EVAL_WHEEL_VERSION=26.03 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple python -c "from nemo_evaluator.core.input import _copy_fdfs; _copy_fdfs('/opt/metadata/')"
03/11/2026 1:55 PM UTC
sha256:38f63200cc485a8c7a71fec5a2f03333830bfef4a0a29f4bc2bc58ba56da574eRUN
CI_COMMIT_SHA=83a6bbd46e65be5bceb13b920aa2f90dd73ec155 EVAL_WHEEL_VERSION=26.03 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install nvidia-lm-eval[math,evalplus]==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
03/11/2026 1:55 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple
03/11/2026 1:55 PM UTC