NVIDIA
NVIDIA
NVIDIA BigCode-Evaluation-Harness
Container
NVIDIA
NVIDIA
NVIDIA BigCode-Evaluation-Harness

NVIDIA NeMo Evaluator-compatible container with BigCode-Evaluation-Harness support. Based on BigCode available at: https://github.com/bigcode-project/bigcode-evaluation-harness

  • LayerLabelCreated
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
    org.opencontainers.image.licenses=3rd_party_licenses.txt
    06/09/2025 5:06 PM UTC
    sha256:399af60630fc721366c272143fea19207fe97af2322313e2c46451dd0d59f4ceCOPY
    3rd_party_licenses.txt .
    06/09/2025 5:06 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    THIRD_PARTY_LICENSES_FILE=3rd_party_licenses.txt
    06/09/2025 5:06 PM UTC
    sha256:6d688899fc59347ea2c146c91ed522cf21cd726a8a37308fa495b807e9397901COPY
    --chown=1000:1000 --chmod=777 tests/* /workspace/tests/
    06/09/2025 2:23 PM UTC
    sha256:2a702b379a4409c8152791e6b4e6a9672cb8727a4cae0b41c401674f7ad134f3COPY
    tests/* /workspace/tests/
    06/09/2025 2:23 PM UTC
    sha256:2bbdcb4d123cb84c8764be957134206dd7bf7c26b8144f25506493961cf213aeRUN
    CI_COMMIT_SHA=b91dde956bae7ebeb09ad203d534ce4d54b51b29 EVAL_WHEEL_VERSION=25.05.1 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple printf '#!/bin/bash\ncore-evals-bigcode "$@"' > /usr/bin/nv-eval &&
      chmod +x /usr/bin/nv-eval &&
      printf '#!/bin/bash\ncore-evals-bigcode "$@"' > /usr/bin/nv_eval &&
      chmod +x /usr/bin/nv_eval
    06/09/2025 2:23 PM UTC
    sha256:0e30a2b679af713e53c8278885aac9dba745e347f2b9cc409e90690162478076RUN
    CI_COMMIT_SHA=b91dde956bae7ebeb09ad203d534ce4d54b51b29 EVAL_WHEEL_VERSION=25.05.1 PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple pip install nvidia-bigcode-eval==${EVAL_WHEEL_VERSION} --no-cache-dir --index-url https://gitlab-ci-token:$(cat /run/secrets/ci_job_token)@${PKG_URL}
    06/09/2025 2:23 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    PKG_URL=gitlab-master.nvidia.com/api/v4/projects/153108/packages/pypi/simple
    06/09/2025 2:23 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    EVAL_WHEEL_VERSION=25.05.1
    06/09/2025 2:23 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
    CORE_EVALS_GIT_HASH=b91dde956bae7ebeb09ad203d534ce4d54b51b29
    06/09/2025 2:23 PM UTC
    ...