NVIDIA
NVIDIA
NVIDIA LM-Evaluation-Harness
Container
NVIDIA
NVIDIA
NVIDIA LM-Evaluation-Harness

NVIDIA NeMo Evaluator-compatible container with LM-Evaluation-Harness support. Based on lm-evaluation-harness available at: https://github.com/EleutherAI/lm-evaluation-harness

LayerLabelCreated
sha256:02d130a9d50dfffad6127eed174b939ba33b08ef6f6007e1dd4d9c0bf5107307COPY
tests/* /workspace/tests/
01/29/2025 9:30 AM UTC
sha256:e3f99b23734af3a94adf061699d205643464adfc173a656d3153f97751679f83RUN
CI_COMMIT_SHA=9fcc8fd2d39bc7a47f1345a0fdc612ef26dea20a EVAL_WHEEL_VERSION=0.0.dev0+9fcc8fd2 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple printf '#!/bin/bash\ncore-evals-lm-eval "$@"' > /usr/bin/nv-eval &&
  chmod +x /usr/bin/nv-eval &&
  printf '#!/bin/bash\ncore-evals-lm-eval "$@"' > /usr/bin/nv_eval &&
  chmod +x /usr/bin/nv_eval
01/29/2025 9:30 AM UTC
sha256:4b5e3a98eb9fed62792d9cd1f40f64ce785ae8326f25607a919357a2ece5f9c1RUN
CI_COMMIT_SHA=9fcc8fd2d39bc7a47f1345a0fdc612ef26dea20a EVAL_WHEEL_VERSION=0.0.dev0+9fcc8fd2 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install lm-eval[ifeval,sentencepiece,math,api]==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
01/29/2025 9:30 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple
01/29/2025 9:30 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
EVAL_WHEEL_VERSION=0.0.dev0+9fcc8fd2
01/29/2025 9:30 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
CORE_EVALS_GIT_HASH=9fcc8fd2d39bc7a47f1345a0fdc612ef26dea20a
01/29/2025 9:30 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
CI_COMMIT_SHA=9fcc8fd2d39bc7a47f1345a0fdc612ef26dea20a
01/29/2025 9:30 AM UTC
sha256:52f664ada013ffa60537a326d207d53b2dc81c71c7662837c7cb08df7570ff41WORKDIR
/workspace
01/29/2025 9:07 AM UTC
sha256:a44cf788c2a636cc068b23edfa88b34110903d8d06621b5fa7f1efc16e180b6bRUN
CI_COMMIT_SHA= pip install pydantic
01/29/2025 9:07 AM UTC
sha256:ce707357bb8f52e11f5ffbc15cd159cd1a535e5e0aa5970d3c0d29c36d4cccc0RUN
CI_COMMIT_SHA= pip install --no-cache-dir git+https://github.com/NVIDIA/dllogger#egg=dllogger
01/29/2025 9:07 AM UTC