NVIDIA
NVIDIA
NVIDIA BigCode-Evaluation-Harness
Container
NVIDIA
NVIDIA
NVIDIA BigCode-Evaluation-Harness

NVIDIA NeMo Evaluator-compatible container with BigCode-Evaluation-Harness support. Based on BigCode available at: https://github.com/bigcode-project/bigcode-evaluation-harness

  • LayerLabelCreated
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
    org.opencontainers.image.licenses=3rd_party_licenses.txt
    07/23/2025 1:10 PM UTC
    sha256:5a2208c4b85190aabd8d775e0e2b99f3d7fe6272acc89df221b5b27980ba8e00COPY
    3rd_party_licenses.txt .
    07/23/2025 1:10 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    THIRD_PARTY_LICENSES_FILE=3rd_party_licenses.txt
    07/23/2025 1:10 PM UTC
    sha256:56e14861897f62c272acfb3585d75b5a0884e239a9392846a22e4a919012e654RUN
    pip uninstall -y nvidia-eval-commons-internal
    07/23/2025 1:10 PM UTC
    sha256:7ed9b86dc7ad8244c50f6e8dec51efe5bf312d735afe419b799f79ddfa25c82fCOPY
    --chown=1000:1000 --chmod=777 tests/* /workspace/tests/
    07/23/2025 12:12 PM UTC
    sha256:01d3e34ee729af59440849b145db392a5627638dbfb2934fbf38ce9281906348COPY
    tests/* /workspace/tests/
    07/23/2025 12:12 PM UTC
    sha256:367ae361eba57fbc2eea591189febf8489427cbafafcdc598809a520c23150d2RUN
    CI_COMMIT_SHA=23d286845c4fc4106c65a0ebdb89ba4f94e9750f EVAL_WHEEL_VERSION=25.06.2 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple printf '#!/bin/bash\ncore-evals-bigcode "$@"' > /usr/bin/nv-eval &&
      chmod +x /usr/bin/nv-eval &&
      printf '#!/bin/bash\ncore-evals-bigcode "$@"' > /usr/bin/nv_eval &&
      chmod +x /usr/bin/nv_eval
    07/23/2025 12:12 PM UTC
    sha256:a26dabd83db639765ce18470388923ef98c3ed36be8e5d2aa7bc2a3d48ff0d01RUN
    CI_COMMIT_SHA=23d286845c4fc4106c65a0ebdb89ba4f94e9750f EVAL_WHEEL_VERSION=25.06.2 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install nvidia-bigcode-eval==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
    07/23/2025 12:12 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple
    07/23/2025 12:12 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    EVAL_WHEEL_VERSION=25.06.2
    07/23/2025 12:12 PM UTC
    ...