nemo-retriever Helm chart

A Kubernetes Helm chart for running the service mode of\nnemo-retriever: a FastAPI document ingestion server that\nstreams uploads through a set of NVIDIA NIM microservices\n(object detection, OCR, VLM embed by default) and exposes\nresult + status APIs over HTTP / SSE.

Use Helm (this chart and/or the additional Library charts documented in the\nNeMo Retriever Library)\nfor supported NIM and service deployment.

The chart ships two deployable layers behind feature flags:

the service — always on; one Deployment (standalone) or three\nDeployments (split topology: gateway / realtime / batch), built from\nDockerfile --target service.
the NIMs — optional, GPU-backed NIMCache + NIMService custom\nresources (apiVersion: apps.nvidia.com/v1alpha1) reconciled by the\nNVIDIA NIM Operator. The chart auto-wires the operator-managed\nService URLs into the retriever-service config when the operator CRDs\nare present in the cluster.

\n
NIM Operator prerequisite. The NIM templates are gated on the\napps.nvidia.com/v1alpha1 API group. Install the NIM Operator before\nrunning helm install:\nhttps://docs.nvidia.com/nim-operator/
\n
Without the operator the chart still installs cleanly — every NIMCache /\nNIMService template short-circuits and the service falls back to\nexternal NIM URLs supplied via serviceConfig.nimEndpoints.*.
\n

\n
Gateway scheduler state is explicitly ephemeral. Split mode enforces one\ngateway replica and uses a Recreate rollout so independent in-memory\nschedulers never overlap. A gateway restart, rollout, eviction, or node failure\nloses every accepted job, queued item, active lease, status record, and SSE\ncatch-up event owned by that process. Drain accepted work before any rollout.
\n

\n
For behavioral consistency between local HuggingFace deployments and Helm service deployments:\n`results = ingestor.ingest(...return_results=True)\nreturn_results defaults to True. This incurs a significant performance and system memory usage cost.\nUnless you know explicitly you need to fetch extraction results to the client, you should use:\nreturn_results=False\nIf you must return results, you may need to increase pod memory specs to support the increased pod memory usage.
\n

Layout

nemo_retriever/helm/\n├── Chart.yaml\n├── values.yaml\n├── README.md            <-- this file\n├── openshift.md         <-- OpenShift restricted-v2 install guide\n├── .helmignore\n└── templates/\n    ├── _helpers.tpl\n    ├── NOTES.txt\n    ├── configmap.yaml                         # renders retriever-service.yaml\n    ├── deployment.yaml                        # the service Deployment(s)\n    ├── service.yaml                           # ClusterIP/NodePort for the service\n    ├── ingress.yaml                           # optional Ingress\n    ├── hpa.yaml                               # optional HorizontalPodAutoscaler\n    ├── servicemonitor.yaml                    # optional Prometheus ServiceMonitor\n    ├── serviceaccount.yaml\n    ├── pvc.yaml                               # general persistence PVC\n    ├── secrets.yaml                           # ngc-secret + ngc-api\n    └── nims/\n        ├── nemotron-page-elements-v3.yaml    # NIMCache + NIMService\n        ├── nemotron-table-structure-v1.yaml   # NIMCache + NIMService\n        ├── nemotron-ocr-v2.yaml               # NIMCache + NIMService\n        ├── llama-nemotron-embed-vl-1b-v2.yaml           # NIMCache + NIMService (VLM embed)\n        ├── llama-nemotron-rerank-vl-1b-v2.yaml  # NIMCache + NIMService (optional; auto-wired when enabled)\n        ├── nemotron-parse.yaml                # NIMCache + NIMService (optional; not auto-wired)\n        ├── nemotron-3-nano-omni-30b-a3b-reasoning.yaml  # NIMCache + NIMService (optional; auto-wired when enabled)\n        └── audio.yaml                         # NIMCache + NIMService (optional; not auto-wired)\n

Quick start

Persistent storage prerequisite { #persistent-storage-prerequisite }

The default chart creates seven PersistentVolumeClaims: three\nchart-managed service claims and four NIM Operator NIMCache claims for\nthe core NIMs. helm install reports STATUS: deployed when the\nrelease is rendered, even if every claim stays Pending. Confirm a\nworking persistent-volume binding strategy before you install.

Run the following preflight commands:

kubectl get storageclass\nkubectl get pv\n

Use one of the following strategies:

A default StorageClass ((default) in kubectl get storageclass)\nbacked by a working provisioner.
Explicit storageClass values for every default claim, each backed\nby a working provisioner or matching persistent volumes.
Compatible static persistent volumes, or pre-created claims where\nthe chart supports existingClaim.

When a storageClass value is empty, the chart omits\nstorageClassName. Kubernetes then assigns the default StorageClass\nif one exists. If none exists, the claim binds only to a compatible\nclassless persistent volume that matches the requested size and\nReadWriteOnce access mode.

The following table lists the default claims for a release named\nretriever:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Example claim name	Default size	Helm value path
`retriever-nemo-retriever-data`	`50Gi`	`persistence.storageClass`
`retriever-nemo-retriever-retriever-results`	`50Gi`	`retrieverResults.storageClass`
`retriever-nemo-retriever-vectordb-data`	`50Gi`	`topology.vectordb.persistence.storageClass`
`nemotron-page-elements-v3-pvc`	`25Gi`	`nimOperator.page_elements.storage.pvc.storageClass`
`nemotron-table-structure-v1-pvc`	`25Gi`	`nimOperator.table_structure.storage.pvc.storageClass`
`nemotron-ocr-v2-pvc`	`25Gi`	`nimOperator.ocr.storage.pvc.storageClass`
`llama-nemotron-embed-vl-1b-v2-pvc`	`50Gi`	`nimOperator.vlm_embed.storage.pvc.storageClass`

When nims.enabled=false, the four NIMCache claims are not created.\nThe three chart-managed claims still are, unless you disable those\npersistence blocks. Enabling an optional NIM adds another NIMCache\nclaim for that key.

To pin a named StorageClass on every default claim, pass the following\n--set flags with your cluster class name:

\\\n --set retrieverResults.storageClass= \\\n --set topology.vectordb.persistence.storageClass= \\\n --set nimOperator.page_elements.storage.pvc.storageClass= \\\n --set nimOperator.table_structure.storage.pvc.storageClass= \\\n --set nimOperator.ocr.storage.pvc.storageClass= \\\n --set nimOperator.vlm_embed.storage.pvc.storageClass= \\\n --set ngcImagePullSecret.create=true \\\n --set ngcImagePullSecret.password=$NGC_API_KEY \\\n --set ngcApiSecret.create=true \\\n --set ngcApiSecret.password=$NGC_API_KEY\n\">

helm install retriever ./nemo_retriever/helm \\\n  --set persistence.storageClass=<STORAGE_CLASS> \\\n  --set retrieverResults.storageClass=<STORAGE_CLASS> \\\n  --set topology.vectordb.persistence.storageClass=<STORAGE_CLASS> \\\n  --set nimOperator.page_elements.storage.pvc.storageClass=<STORAGE_CLASS> \\\n  --set nimOperator.table_structure.storage.pvc.storageClass=<STORAGE_CLASS> \\\n  --set nimOperator.ocr.storage.pvc.storageClass=<STORAGE_CLASS> \\\n  --set nimOperator.vlm_embed.storage.pvc.storageClass=<STORAGE_CLASS> \\\n  --set ngcImagePullSecret.create=true \\\n  --set ngcImagePullSecret.password=$NGC_API_KEY \\\n  --set ngcApiSecret.create=true \\\n  --set ngcApiSecret.password=$NGC_API_KEY\n

Set each per-NIM nimOperator.<key>.storage.pvc.storageClass path.\nThe chart-level nimOperator.nimCache.pvc.storageClass value is not\napplied to the four core NIMCache resources.

persistence.existingClaim and retrieverResults.existingClaim skip\nchart PVC creation and mount the named claim. The VectorDB claim does\nnot have an existingClaim path.

If helm install already succeeded and claims stay Pending, refer\nto Helm install succeeds but PersistentVolumeClaims stay Pending.

GPU scheduling prerequisite { #gpu-scheduling-prerequisite }

The model hardware requirements\ntable lists Total GPUs: 1 for Core Features because the four\ndefault NIMs together use about 4.8 GiB of GPU memory and can\nco-reside on one A10G or better GPU. That figure is VRAM capacity.\nIt is not the number of exclusive Kubernetes GPU requests this chart\nmakes.

A default install creates four NIMService resources\n(page_elements, table_structure, ocr, vlm_embed). Each\nrenders spec.resources.limits.nvidia.com/gpu: 1 from\nnimOperator.nimServiceGpuLimit (default 1). On a conventional\ncluster without MIG, time-slicing, or another sharing mechanism, the\nscheduler consumes four allocatable GPU slots across eligible\nnodes. Four one-GPU nodes satisfy that topology. A cluster that\nhas only one allocatable GPU schedules only one NIM pod. The other\nthree stay Pending.

Choose one of the following before you install:

Provide four allocatable nvidia.com/gpu slots across eligible\nnodes for the default core topology. Each optional NIM adds its\nper-NIM GPU request. Most optional keys use nimServiceGpuLimit\n(one GPU). The default answer_llm Super-49B resources request\ntwo physical GPUs.
Configure GPU sharing so the cluster advertises at least four\nnvidia.com/gpu slots. Time-slicing is the documented sharing\npath. It works on GPUs that do not support Multi-Instance GPU\n(MIG), including A10G.

This chart does not pack the four NIMs onto a single nvidia.com/gpu\nrequest. Sharing is cluster configuration through the\nNVIDIA GPU Operator.\nTime-slicing does not isolate GPU memory. Combined VRAM of the\nscheduled NIMs must still fit on the physical GPU.

Confirm allocatable GPU slots across the cluster:

kubectl get nodes -o custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\\.com/gpu\n

Sum GPU across eligible nodes. A default core install needs four\nslots across the cluster. Four nodes that each report 1 are\nenough. A single node needs a value of 4 or greater only when you\npack all four core NIMs onto one physical GPU with sharing and\nplacement constraints.

The following GPU Operator time-slicing ConfigMap advertises four\nreplicas per physical GPU. On a node with one physical GPU, that\ncreates four logical slots, which is enough for the four default\nNIMServices. A node with two physical GPUs advertises eight slots.\nThe NVIDIA GPU Operator must already be installed. Apply the\nConfigMap in the GPU Operator namespace (commonly gpu-operator)\nbefore helm install:

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: time-slicing-config-all\ndata:\n  any: |-\n    version: v1\n    flags:\n      migStrategy: none\n    sharing:\n      timeSlicing:\n        resources:\n        - name: nvidia.com/gpu\n          replicas: 4\n

kubectl create -n gpu-operator -f time-slicing-config-all.yaml\nkubectl patch clusterpolicies.nvidia.com/cluster-policy \\\n    -n gpu-operator --type merge \\\n    -p '{\"spec\": {\"devicePlugin\": {\"config\": {\"name\": \"time-slicing-config-all\", \"default\": \"any\"}}}}'\n

That ClusterPolicy patch is a cluster-administrator change.\ndefault: \"any\" applies the four-replica configuration to all\neligible GPU Operator nodes. It can oversubscribe unrelated GPU\nworkloads that share those nodes.

Time-slicing oversubscribes allocatable slots. It does not place\nthe four NIM pods on one physical GPU. The scheduler can spread\nthem across GPUs or nodes. The one-physical-GPU recipe requires\nboth of the following: the target node has one physical GPU and\nadvertises at least four replicas, and all four NIMServices are\npinned to that node. Confirm that node's Allocatable\nnvidia.com/gpu is 4 or greater, and pin the four core\nNIMServices as shown below.

The default answer_llm Super-49B NIMService is outside that\none-physical-GPU recipe. It requests two GPUs (nvidia.com/gpu: 2\nand NIM_TENSOR_PARALLEL_SIZE=2). A time-sliced request for more\nthan one GPU does not provide two physical GPUs or proportional\ncompute, so extra time-slice replicas cannot satisfy that\ntensor-parallel requirement. Keep Super-49B on two physical GPUs\nunless you override the slot with a separately validated model and\nprofile. Refer to Answer generation.

On a multi-GPU or multi-node cluster, pin the four core\nNIMServices to a single-GPU node. Set\nnimOperator.<key>.nodeSelector on page_elements,\ntable_structure, ocr, and vlm_embed. nodeSelector\nconstrains node placement. On a multi-GPU node, it does not\nensure all four pods receive logical replicas from the same\nphysical GPU. This chart does not render pod affinity or\ntopology spread constraints:

\n table_structure:\n nodeSelector:\n kubernetes.io/hostname: \n ocr:\n nodeSelector:\n kubernetes.io/hostname: \n vlm_embed:\n nodeSelector:\n kubernetes.io/hostname: \n\">

nimOperator:\n  page_elements:\n    nodeSelector:\n      kubernetes.io/hostname: <gpu-node>\n  table_structure:\n    nodeSelector:\n      kubernetes.io/hostname: <gpu-node>\n  ocr:\n    nodeSelector:\n      kubernetes.io/hostname: <gpu-node>\n  vlm_embed:\n    nodeSelector:\n      kubernetes.io/hostname: <gpu-node>\n

On a mixed cluster, omit devicePlugin.config.default and label\nonly the target node so other GPU nodes keep exclusive access:

nvidia.com/device-plugin.config=any\n\">

kubectl patch clusterpolicies.nvidia.com/cluster-policy \\\n    -n gpu-operator --type merge \\\n    -p '{\"spec\": {\"devicePlugin\": {\"config\": {\"name\": \"time-slicing-config-all\"}}}}'\nkubectl label node <gpu-node> nvidia.com/device-plugin.config=any\n

Increase replicas if you also enable optional NIMs that request\none GPU each on the same shared GPU and combined VRAM still fits.\nCount each of those Helm GPU requests. Do not increase time-slice\nreplicas to satisfy tensor-parallel GPU counts. For the full\nprocedure, node labels, and limitations, refer to\nTime-Slicing GPUs in Kubernetes.\nMIG is an advanced GPU Operator configuration outside this chart.\nThe chart does not set a MIG strategy, MIG profile, or MIG\nresource requests. Default NIMServices request nvidia.com/gpu.\nA working MIG deployment is GPU-specific and profile-specific and\ncan require per-NIM resources overrides. For GPU Operator MIG,\nrefer to\nGPU Operator with MIG.\nThe NIM Operator also documents DRA-based sharing. This chart does\nnot render draResources; refer to\nNIM Operator DRA\nif you manage ResourceClaims outside the chart.

If helm install already succeeded and NIM pods stay Pending on\nnvidia.com/gpu, refer to\nCore NIM pods stay Pending for GPU.

1. Service image { #1-service-image }

The chart defaults to the image published to NGC:

nvcr.io/nvidia/nemo-microservices/nrl-service:26.5.0\n

Pulling from nvcr.io requires an NGC pull secret — either set\nngcImagePullSecret.create=true (see below) or pre-create one in the\nnamespace named ngc-secret.

To run a locally built image instead, build and push it from the repo root,\nthen override service.image.repository / service.image.tag:

\\\n --build-arg RETRIEVER_RELEASE_TYPE=release \\\n -t /nemo-retriever-service: .\ndocker push /nemo-retriever-service:\n\">

# from the repo root:\ndocker build \\\n    --target service \\\n    --build-arg DOWNLOAD_DEFAULT_TOKENIZER=True \\\n    --build-arg RETRIEVER_VERSION=<TAG> \\\n    --build-arg RETRIEVER_RELEASE_TYPE=release \\\n    -t <YOUR_REGISTRY>/nemo-retriever-service:<TAG> .\ndocker push <YOUR_REGISTRY>/nemo-retriever-service:<TAG>\n

Audio and video extraction require the ffmpeg and ffprobe system\nbinaries inside the service container. The bundled service image can install\nthem at container startup when you set service.installFfmpeg=true, which\nsets INSTALL_FFMPEG=true for the image entrypoint:

/nemo-retriever-service \\\n --set service.image.tag= \\\n --set service.installFfmpeg=true\n\">

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set service.image.repository=<YOUR_REGISTRY>/nemo-retriever-service \\\n  --set service.image.tag=<TAG> \\\n  --set service.installFfmpeg=true\n

Do not also set INSTALL_FFMPEG in service.env; the chart fails rendering\nwhen both are configured so the rendered Pod does not contain duplicate\nenvironment variables.

When service.installFfmpeg=false (the default), the service still starts\nnormally and processes PDF, image, text and HTML uploads. Audio / video\nuploads are rejected up-front with HTTP 501:

Audio and video ingestion require FFmpeg in the retriever service\ncontainer, but the following dependencies are missing: ffmpeg, ffprobe.\nRe-deploy the Helm chart with `--set service.installFfmpeg=true` …\n

The retriever-service container also logs a WARNING at startup when\nFFmpeg is missing so cluster operators can fix the deployment before\nthe first media upload arrives, instead of debugging a Ray worker\ntraceback (RuntimeError: MediaChunkActor requires media dependencies; missing: ffmpeg, ffprobe) after the fact. The same WARNING is emitted\non every pod (gateway, realtime, batch) because all roles classify\nuploads — flipping service.installFfmpeg=true updates them all.

Runtime installation uses passwordless sudo scoped to installing the\nffmpeg package in the service image. The pod must have network egress to the\nUbuntu package repositories, a writable root filesystem, and a security policy\nthat allows sudo/setuid behavior. Do not set\nservice.securityContext.allowPrivilegeEscalation: false or\nservice.securityContext.readOnlyRootFilesystem: true for this path.

For air-gapped or locked-down clusters, see\nDeployment options — Air-gapped and disconnected deployment.\nOn a connected staging host you can extend the service image, for example:

/nemo-retriever-service:\nUSER root\nRUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \\\n && rm -rf /var/lib/apt/lists/*\nUSER nemo\n\">

FROM <YOUR_REGISTRY>/nemo-retriever-service:<BASE_TAG>\nUSER root\nRUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \\\n    && rm -rf /var/lib/apt/lists/*\nUSER nemo\n

2. Install with external NIM endpoints (operator not required)

Complete the persistent storage prerequisite\nbefore you install. With nims.enabled=false, the chart still creates\nthe three service claims unless you disable those persistence blocks.

If you already have NIM endpoints reachable from the cluster (e.g. another\nnamespace, or NVIDIA Build), turn the master switch off and supply the\nURLs directly:

helm install retriever ./nemo_retriever/helm \\\n  --set nims.enabled=false \\\n  --set ngcImagePullSecret.create=true \\\n  --set ngcImagePullSecret.password=$NGC_API_KEY \\\n  --set ngcApiSecret.create=true \\\n  --set ngcApiSecret.password=$NGC_API_KEY \\\n  --set serviceConfig.nimEndpoints.pageElementsInvokeUrl=http://page-elements.svc:8000/v1/page-elements \\\n  --set serviceConfig.nimEndpoints.tableStructureInvokeUrl=http://table-structure.svc:8000/v1/table-structure \\\n  --set serviceConfig.nimEndpoints.ocrInvokeUrl=http://ocr.svc:8000/v1/ocr \\\n  --set serviceConfig.nimEndpoints.embedInvokeUrl=http://embed.svc:8000/v1/embeddings\n\n

ngcApiSecret materialises an ngc-api Secret containing both\nNGC_API_KEY and NGC_CLI_API_KEY keys; the service container reads it\nvia optional: true secretKeyRef, so the install still succeeds when\nthe secret is absent (useful for fully local NIM endpoints).

3. Install with the NIM Operator (in-cluster NIMs)

Complete the persistent storage prerequisite\nand the GPU scheduling prerequisite\nbefore you install. The default path creates all seven claims and four\nexclusive nvidia.com/gpu: 1 NIMService requests.

Install the NIM Operator first so\nthe NIMCache / NIMService CRDs (apps.nvidia.com/v1alpha1) are\nregistered. A plain helm install reconciles the four core NIMs\n(page_elements, table_structure, ocr, vlm_embed) — every other\nNIM (the VL reranker rerankqa, Nemotron Parse, Omni 30B, and the\nParakeet audio ASR NIM) is disabled by default to honor the\n\"optional and disabled by default\" contract in\ndeployment-options.md;\nrefer to Recommended minimal install\nfor the opt-in --set flags that turn any of them on.

helm install retriever ./nemo_retriever/helm \\\n  --set ngcImagePullSecret.create=true \\\n  --set ngcImagePullSecret.password=$NGC_API_KEY \\\n  --set ngcApiSecret.create=true \\\n  --set ngcApiSecret.password=$NGC_API_KEY\n

Recommended minimal install (26.08.1) { #recommended-minimal-install-26081 }

Complete the persistent storage prerequisite\nand the GPU scheduling prerequisite\nbefore you install.

Deploy only the four core NIMs that the retriever service auto-wires (page_elements, table_structure, ocr, vlm_embed):

helm install retriever ./nemo_retriever/helm \\\n  --set ngcImagePullSecret.create=true \\\n  --set ngcImagePullSecret.password=$NGC_API_KEY \\\n  --set ngcApiSecret.create=true \\\n  --set ngcApiSecret.password=$NGC_API_KEY \\\n  --set service.image.tag=26.8.1\n

\n
The VL reranker (rerankqa), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, the generic answer-generation LLM (answer_llm, Super-49B defaults), and the Parakeet audio ASR NIM are all off by default — they only reconcile when you explicitly opt in. Opt-in flags:
\n
\n
VL reranker — --set nimOperator.rerankqa.enabled=true (auto-wires nim_endpoints.rerank_invoke_url / rerank_model_name — refer to Query-time reranking)
\n
Nemotron Parse — --set nimOperator.nemotron_parse.enabled=true
\n
Omni 30B captioner — --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true
\n
Answer generation LLM — --set nimOperator.answer_llm.enabled=true
\n
Parakeet ASR — --set nimOperator.audio.enabled=true (also set serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051 to wire ASR into the service, plus service.installFfmpeg=true if your image does not bundle ffmpeg)
\n
\n
This matches the \"optional and disabled by default\" contract in deployment-options.md and avoids silently pulling ≈ 62 GiB of Omni weights, loading a large two-GPU LLM, or claiming extra dedicated GPUs on a \"default\" install. Refer to the model hardware requirements table for per-NIM GPU and disk costs.
\n

The chart auto-wires the operator-managed in-cluster URLs of the three\n\"core\" NIMs into the service's nim_endpoints block:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

key	operator-managed Service	invoke path
`nimOperator.page_elements`	`nemotron-page-elements-v3`	`/v1/page-elements`
`nimOperator.table_structure`	`nemotron-table-structure-v1`	`/v1/table-structure`
`nimOperator.ocr`	`nemotron-ocr-v2`	`/v1/ocr`
`nimOperator.vlm_embed`	`llama-nemotron-embed-vl-1b-v2`	`/v1/embeddings`

Query reranking (optional)

The optional nimOperator.rerankqa NIM is not auto-wired into the retriever service. To use POST /v1/query with rerank=true, enable the NIM and configure the service endpoint explicitly:

nimOperator:\n  rerankqa:\n    enabled: true\n\nserviceConfig:\n  nimEndpoints:\n    rerankInvokeUrl: http://llama-nemotron-rerank-vl-1b-v2:8000/v1/ranking\n    rerankModelName: nvidia/llama-nemotron-rerank-vl-1b-v2\n

Enabling nimOperator.rerankqa.enabled=true without serviceConfig.nimEndpoints.rerankInvokeUrl deploys the NIM but does not enable service query reranking.

Track operator reconciliation with:

\nkubectl describe nimservice nemotron-object-detection -n \n\">

kubectl get nimcache,nimservice -n <namespace>\nkubectl describe nimservice nemotron-object-detection -n <namespace>\n

First-time NIMCache reconciliation downloads model weights to a PVC. By\ndefault (nimOperator.nimCache.keepOnUninstall: true) every NIMCache\ncarries helm.sh/resource-policy: keep so those downloads survive\nhelm uninstall. NIMService CRs do not use keep and are removed by\nHelm on uninstall.

Why NIM resources still exist after `helm uninstall`

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

What you see	Typical cause
`NIMCache` + PVC remain	Expected when `keepOnUninstall` is true (default). Helm intentionally skips deleting caches so you do not re-pull multi‑GiB weights.
`NIMService` CR remains	Not expected on a normal uninstall. Usually an orphan from a failed install/upgrade (release never recorded the resource, or the chart renamed a NIM).
Deployments / GPU pods still running	Often the operator workload for a kept `NIMCache`, or a stale `NIMService` that Helm did not own. Check `kubectl get nimservice,nimcache -n <ns>`.
`nemotron--job-` pods in `Error`	The NIM Operator's model-download Job for a `NIMCache` (not the retriever service). Failed cache pulls retry and leave Error pods until the Job or `NIMCache` is deleted. Common after a failed `helm install` when the release is rolled back but `keep` retains the cache CR.
`helm uninstall` appears to do nothing	Release may be missing or failed (`helm list -n <ns> -a`). CRs created before a failed install can be left without a release to clean them up.

To change a NIM image on a later install or upgrade, delete the kept\nNIMCache first. Refer to\nChanging a NIM image repository or tag.

Full teardown (dev cluster — deletes caches and PVCs Helm kept):

/dev/null || true\n\n# Orphans and kept NIMCaches (Helm keep does not block kubectl delete):\nkubectl delete nimservice,nimcache -n "${NS}" --all\n# Optional: drop model PVCs if you will re-pull from NGC\nkubectl delete pvc -n "${NS}" -l 'app.kubernetes.io/managed-by=nvidia-nim-operator' 2>/dev/null || true\n\">

NS=retriever\nREL=nemo-retriever\n\nhelm uninstall \"${REL}\" -n \"${NS}\" 2>/dev/null || true\n\n# Orphans and kept NIMCaches (Helm keep does not block kubectl delete):\nkubectl delete nimservice,nimcache -n \"${NS}\" --all\n# Optional: drop model PVCs if you will re-pull from NGC\nkubectl delete pvc -n \"${NS}\" -l 'app.kubernetes.io/managed-by=nvidia-nim-operator' 2>/dev/null || true\n

Dev installs that should not retain caches on uninstall:

helm upgrade --install \"${REL}\" ./nemo_retriever/helm -n \"${NS}\" \\\n  --set nimOperator.nimCache.keepOnUninstall=false \\\n  ...\n

Values reference (highlights)

The full schema lives in values.yaml. Below is the\nshort list of knobs you'll touch first.

Service

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Default	Notes
`service.image.repository`	`nvcr.io/nvidia/nemo-microservices/nrl-service`	NGC image; override to pin a different build or use a local registry.
`service.image.tag`	`26.5.0`	Also injected as `RETRIEVER_SERVICE_VERSION` so `/openapi.json` `info.version` matches the running image tag.

For audio and video extraction, set service.installFfmpeg=true when your\ncluster allows runtime package installation. OpenShift restricted-v2 blocks\nthat path — use a prebuilt service image instead; refer to Audio and video on restricted OpenShift.\nFor air-gapped clusters, refer to Deployment options — Air-gapped and disconnected deployment.

Audio and video (Parakeet ASR) { #audio-video-parakeet }

Parakeet ASR is disabled by default. The chart default is nimOperator.audio.enabled=false. The chart does not auto-wire the in-cluster ASR endpoint when you enable the audio NIM.

To run self-hosted Parakeet for audio and video extraction, set both of the following values:

nimOperator:\n  audio:\n    enabled: true\n\nserviceConfig:\n  nimEndpoints:\n    audioGrpcEndpoint: audio:50051\n

Equivalent Helm flags are --set nimOperator.audio.enabled=true and --set serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051.

Enabling only nimOperator.audio.enabled=true renders the Parakeet NIMCache and NIMService. The ConfigMap still sets audio_grpc_endpoint to null. The retriever service cannot send ASR traffic until you also set serviceConfig.nimEndpoints.audioGrpcEndpoint. Disable other optional NIMs you do not need. Refer to Recommended minimal install.

After you set those values, complete the following steps:

Pin the ASR NIMService to a dedicated GPU with nimOperator.audio.resources, nodeSelector, or tolerations (refer to NIM Operator).
Confirm the GPU SKU in Model hardware requirements (footnote ⁴ lists Blackwell limitations).
Set service.installFfmpeg=true when the retriever service will process audio or video on clusters that allow runtime package install (refer to service.installFfmpeg above). On OpenShift restricted-v2, use a prebuilt service image instead.

The in-cluster gRPC Service name is audio on port 50051. Graph ingest does not read this Helm value. Pass the same endpoint through ASRParams.audio_endpoints in Python. Refer to NIM Operator sub-stack.

Health probes

The service exposes unauthenticated health endpoints for Kubernetes probes:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Endpoint	Purpose	Default Helm use
`GET /v1/live`	Shallow process liveness. The endpoint does not check worker backends or wait for service readiness.	Startup and liveness probes. In split mode, the realtime and batch `wait-for-gateway` init containers poll this endpoint on the internal gateway startup Service while waiting for the gateway process to start.
`GET /v1/health`	Deep readiness. In split gateway mode, the endpoint checks the realtime and batch workers. It returns HTTP `503` when either required worker is unreachable or returns a non-2xx health response.	Readiness probe on the externally exposed gateway Service.

In split topology, the gateway readiness probe uses /v1/health, which returns\nHTTP 503 until realtime and batch workers are healthy. Worker Pods cannot start\nuntil their wait-for-gateway init container reaches the gateway's shallow\n/v1/live endpoint. The externally exposed gateway Service does not publish\nendpoints for an unready Pod, which would deadlock a clean install. The chart\ntherefore renders an additional cluster-internal Service named\n<release>-nemo-retriever-gateway-startup with\npublishNotReadyAddresses: true. Init containers reach\nhttp://<release>-nemo-retriever-gateway-startup:<networkService.port>/v1/live\nthrough that Service so workers can start while the gateway Pod is still\nunready. Client traffic continues to use the readiness-gated gateway Service, so\n/v1/health still removes an unhealthy gateway from Service endpoints after\nstartup.

When a gateway returns HTTP 503 from /v1/health, Kubernetes removes it from\nthe readiness-gated gateway Service endpoints until its required workers are\nready. The response includes backend health details to help diagnose the\nunavailable dependency.

Service networking

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Default	Notes
`networkService.port`	`7670`	Kubernetes Service port. The chart uses this port for generated Service DNS URLs.
`serviceConfig.server.port`	`7670`	Retriever service container listener port.

You can set these values independently. When they differ, Kubernetes Services\nlisten on networkService.port and route to the container listener on\nserviceConfig.server.port.

In standalone mode, the chart renders one Service named\n<release>-nemo-retriever.

In split topology (topology.mode: split), the chart renders four Services:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Service	Example name (release `retriever`)	Type	Client entrypoint
Gateway	`retriever-nemo-retriever-gateway`	`networkService.type` (default `NodePort`)	Yes
Gateway startup	`retriever-nemo-retriever-gateway-startup`	ClusterIP (cluster-internal only)	No
Realtime	`retriever-nemo-retriever-realtime`	ClusterIP	No
Batch	`retriever-nemo-retriever-batch`	ClusterIP	No

The gateway startup Service publishes the gateway Pod address while it is still\nunready so realtime and batch init containers can reach /v1/live. It is not a\nclient entrypoint. Refer to Health probes.

Service configuration (rendered into `retriever-service.yaml`)

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Default	Notes
`serviceConfig.server.port`	`7670`	Retriever service container listener port. Refer to Service networking.
`serviceConfig.pipeline.realtimeWorkers`	`24`	Per-pod realtime worker count.
`serviceConfig.pipeline.batchWorkers`	`48`	Per-pod batch worker count. Refer to Timeouts and alleviating ingest failures if embed or pool errors appear under load.
`serviceConfig.resources.maxUploadBytes`	`500000000`	Maximum upload file size in bytes; requests exceeding the limit are rejected before buffering.
`serviceConfig.sidecarStore.maxPayloadBytes`	`33554432`	Maximum sidecar metadata upload size in bytes. The service rejects a larger upload with HTTP `413` before buffering the complete payload. This value cannot exceed `serviceConfig.resources.maxUploadBytes`.
`serviceConfig.nimEndpoints.*InvokeUrl`	`\"\"`	Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, `rerankInvokeUrl` (refer to Query-time reranking), and `captionInvokeUrl` (refer to Image captioning (Omni 30B)).
`serviceConfig.nimEndpoints.rerankModelName`	`\"\"`	Model id sent to the remote reranker. Auto-set to `nvidia/llama-nemotron-rerank-vl-1b-v2` whenever a rerank URL is resolved.
`serviceConfig.nimEndpoints.captionModelName`	`\"\"`	Model id sent to the remote VLM. Auto-set to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` whenever a caption URL is resolved.
`serviceConfig.nimEndpoints.rerankInvokeUrl`	`\"\"`	Ranking API URL used by `POST /v1/query` when `rerank=true`. Auto-wired from the optional `rerankqa` NIM when enabled; override to point at a hosted or external ranking endpoint.
`serviceConfig.nimEndpoints.rerankModelName`	`\"\"`	Model ID sent to the ranking API. Auto-set to `nvidia/llama-nemotron-rerank-vl-1b-v2` whenever a rerank URL is resolved; override for a different compatible reranker.
`serviceConfig.nimEndpoints.audioGrpcEndpoint`	`\"\"`	gRPC endpoint for Parakeet ASR. Not auto-wired from `nimOperator.audio`. Set `audio:50051` when you enable the audio NIM.
`serviceConfig.llm.enabled`	`false`	Enables `POST /v1/answer`. Auto-flips to true when `nimOperator.answer_llm` is enabled and the operator URL resolves.
`serviceConfig.llm.apiBase`	`\"\"`	OpenAI-compatible LLM base URL. Explicit value wins; otherwise `answer_llm` opt-in resolves to `http://answer-llm:8000/v1` by default.
`serviceConfig.llm.apiKeySecret.name`	`\"\"`	Optional Secret name for external LLM credentials. Explicit values win; otherwise operator-managed `answer_llm` mounts its `authSecret` as `NEMO_RETRIEVER_LLM_API_KEY` so LiteLLM/OpenAI has a credential value without writing it to the ConfigMap.
`serviceConfig.llm.apiKeySecret.key`	`api_key`	Secret key for external LLM credentials. Operator-managed `answer_llm` uses `NGC_API_KEY` from `nimOperator.answer_llm.authSecret` when no explicit LLM Secret is set.
`serviceConfig.llm.model`	`\"\"`	Optional explicit LiteLLM model id. Leave empty to inherit `nimOperator.answer_llm.model` when using the operator-managed answer LLM; set it for external endpoints.
`serviceConfig.llm.ragSystemPromptPrefix`	`\"\"`	Optional explicit RAG prompt prefix. Leave empty unless an endpoint needs model-specific prompt directives.
`serviceConfig.llm.reasoningEnabled`	`true`	Request-level reasoning toggle for `/v1/answer`. Defaults to true for external OpenAI-compatible providers; set false for Nemotron endpoints that should receive portable no-reasoning controls.
`serviceConfig.agentic.enabled`	`false`	Enables `POST /v1/query` with `agentic=true` and the additive `agentic_query` MCP tool. Not auto-enabled by `nimOperator.answer_llm`. Refer to Agentic retrieval (self-hosted Super-49B).
`serviceConfig.agentic.llmModel`	`\"\"`	Chat model used by the inner agentic retrieval loop. Required when `invokeUrl` is set. Use the NIM-advertised ID (for Super-49B, `nvidia/llama-3.3-nemotron-super-49b-v1.5`), not the LiteLLM `openai/` prefix.
`serviceConfig.agentic.invokeUrl`	`\"\"`	OpenAI-compatible chat completions endpoint used by agentic retrieval. Not auto-populated from `answer_llm`. For the in-cluster Super-49B NIM, set `http://answer-llm:8000/v1/chat/completions`.
`serviceConfig.agentic.requestTimeoutS`	`1800`	Gateway and MCP timeout for the multi-step agentic retrieval call.
`serviceConfig.vectordb.enabled`	`true`	Deploy the LanceDB vectordb Pod. When `true` the chart requires a resolvable embed endpoint (refer to VectorDB and the embed endpoint); `helm install` / `helm upgrade` fails fast otherwise.
`serviceConfig.vectordb.lancedbUri`	`/data/vectordb`	LanceDB on the vectordb Pod's PVC.
`serviceConfig.vectordb.embedModel`	`nvidia/llama-nemotron-embed-vl-1b-v2`	Passed to vectordb + worker `embed_model_name`.
`serviceConfig.vectordb.indexMode`	`hybrid`	Create LanceDB dense-vector and full-text-search indexes. Set to `dense` to create only the dense-vector index.
`serviceConfig.vectordb.embedModelProviderPrefix`	`\"\"`	Optional LiteLLM provider prefix prepended to the remote embed model name.
`serviceConfig.vectordb.writeTimeoutSeconds`	`300`	Rendered as `vectordb.write_timeout_s`. How long a worker waits for the vectordb Pod to acknowledge a record write. A write that is not acknowledged fails the document, so raise this value on slow storage. Refer to Timeouts and alleviating ingest failures.

Sidecar metadata in split topology

ServiceIngestor.vdb_upload() uploads sidecar metadata to\nPOST /v1/ingest/sidecar and uses the returned opaque ID in the subsequent\ningest request. The public API retains its time-to-live and\nconsume_on_read reuse behavior.

Sidecar metadata uses the gateway's in-memory store in standalone and split\ntopologies. Configure the maximum allowed payload size as needed:

serviceConfig:\n  sidecarStore:\n    maxPayloadBytes: 33554432\n

In topology.mode: split, the chart requires one gateway replica. Upload and\nreference each sidecar while that gateway remains running. An uploaded sidecar\nthat has not yet been admitted is lost if the gateway restarts or is replaced.

At ingest admission, the gateway authorizes and binds the sidecar to the\nwork attachment before it leases work to a worker. Workers receive the bound\nattachment with their work and do not resolve sidecar IDs. This supports\nindependent realtime and batch pools, worker replicas, and routing decisions\nwithout requiring sidecar replication between workers.

VectorDB and the embed endpoint { #vectordb-and-the-embed-endpoint }

The vectordb Pod's /v1/query handler embeds the incoming query text\nbefore searching LanceDB. It needs a NIM embedding endpoint to do that,\nand rendering the Deployment with an empty --embed-endpoint produces a\nPod that passes its /v1/health probe but answers every /v1/query\nrequest with HTTP 501 No embedding endpoint configured. — a healthy\ndeployment that silently breaks retrieval.

To prevent this, the chart now refuses to render\ndeployment-vectordb.yaml when no embed endpoint can be resolved.\nhelm install / helm upgrade --install fails with a message listing\nthe three supported escape valves:

:/v1/embeddings\n 2. --set nimOperator.vlm_embed.enabled=true # requires apps.nvidia.com/v1alpha1 CRDs\n 3. --set serviceConfig.vectordb.enabled=false\n\">

serviceConfig.vectordb.enabled=true but the embed endpoint could not be\nresolved.  Pick one of:\n\n  1. --set serviceConfig.nimEndpoints.embedInvokeUrl=http://<host>:<port>/v1/embeddings\n  2. --set nimOperator.vlm_embed.enabled=true   # requires apps.nvidia.com/v1alpha1 CRDs\n  3. --set serviceConfig.vectordb.enabled=false\n

Resolution order matches the rest of the chart (refer to Mix and match NIM\nsources):

Explicit serviceConfig.nimEndpoints.embedInvokeUrl always wins.
Otherwise the operator-managed URL of\nnimOperator.vlm_embed.nimServiceName is used, provided\nnimOperator.vlm_embed.enabled=true and the\napps.nvidia.com/v1alpha1 CRDs are installed in the cluster.
Otherwise the chart fails the install.

Answer generation (operator-managed LLM) { #answer-generation-llm }

Enable the generic answer_llm NIM slot to add service-mode answer\ngeneration on top of the VectorDB query path. The slot defaults to the\nSuper-49B NIM, but the image, model id, service name, resources,\nprofile filter, and environment can be overridden for another\nOpenAI-compatible LLM NIM.

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.answer_llm.enabled=true\n

When the NIM Operator CRDs are present, the chart renders an answer-llm\nNIMCache/NIMService by default and writes this block into\nretriever-service.yaml:

llm:\n  enabled: true\n  model: \"openai/nvidia/llama-3.3-nemotron-super-49b-v1.5\"\n  api_base: \"http://answer-llm:8000/v1\"\n  rag_system_prompt_prefix: null\n  reasoning_enabled: true\n

The retriever service then exposes POST /v1/answer, which calls the\nVectorDB pod's /v1/query endpoint for context and sends those chunks to\nthe configured LLM endpoint. This path does not require tool calling.\nThe answer_llm NIM is not wired into serviceConfig.agentic and is\nnot tool-call ready by default. For agentic retrieval against that NIM,\nrefer to\nAgentic retrieval (self-hosted Super-49B).\nThe answer_llm NIM deployment leaves\nreasoning defaults model-neutral; /v1/answer controls reasoning per\nrequest. By default, serviceConfig.llm.reasoningEnabled=true, so requests\nleave reasoning behavior to the LLM endpoint defaults and avoid sending\nprovider-specific chat_template_kwargs to external OpenAI-compatible\nendpoints. Set serviceConfig.llm.reasoningEnabled=false for Nemotron\nendpoints that should skip reasoning; the service then adds both /no_think\nand chat_template_kwargs.enable_thinking=false. The default Super-49B NIMService\nresources request two physical GPUs (nvidia.com/gpu: 2) to match the bundled\ntensor-parallel NIM profile. Do not satisfy that count with GPU Operator\ntime-slice replicas. Those two GPUs are in addition to the four core NIMs.\nThe chart NIMCache PVC is 250Gi. A100 40GB, A10G, L40S, and RTX PRO 4500\nBlackwell are not supported for that default BF16 TP2 profile. Override\nresources, modelProfile, or env for deployments that use a different\nprofile or hardware topology. Refer to\nModel hardware requirements\nin the Support Matrix.

When answer_llm is enabled and no explicit serviceConfig.llm.apiKeySecret\nis set, the service also mounts nimOperator.answer_llm.authSecret as\nNEMO_RETRIEVER_LLM_API_KEY; OpenAI-compatible clients require a\ncredential value even for in-cluster NIM endpoints, and the key is never\nrendered into the ConfigMap.

For example, to try Nemotron 3 Nano as the answer LLM on an A100 80GB\nnode, override the operator-managed slot instead of adding a second\nhard-coded LLM service:

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.answer_llm.enabled=true \\\n  --set nimOperator.answer_llm.nimServiceName=nemotron-3-nano \\\n  --set nimOperator.answer_llm.image.repository=nvcr.io/nim/nvidia/nemotron-3-nano \\\n  --set nimOperator.answer_llm.image.tag=1.7.0-variant \\\n  --set nimOperator.answer_llm.model=openai/nvidia/nemotron-3-nano-30b-a3b \\\n  --set-json nimOperator.answer_llm.modelProfile='{\"profiles\":[\"5f89f01a0af587fd8bae50c611b1f358f92effdb9fb29362e1af0a986e5561c3\"]}' \\\n  --set-json nimOperator.answer_llm.resources='{\"limits\":{\"nvidia.com/gpu\":1},\"requests\":{\"nvidia.com/gpu\":1}}' \\\n  --set nimOperator.answer_llm.env[0].name=NIM_HTTP_API_PORT \\\n  --set-string nimOperator.answer_llm.env[0].value=8000 \\\n  --set nimOperator.answer_llm.env[1].name=NIM_SERVED_MODEL_NAME \\\n  --set-string nimOperator.answer_llm.env[1].value=nvidia/nemotron-3-nano-30b-a3b \\\n  --set nimOperator.answer_llm.env[2].name=NIM_TENSOR_PARALLEL_SIZE \\\n  --set-string nimOperator.answer_llm.env[2].value=1\n

Use the repository and tag available in your NGC environment; staging\nregistries can use the same override shape with nvstaging image names\nor tags. nimOperator.answer_llm.model is the LiteLLM model id used by\nthe retriever service; for an OpenAI-compatible in-cluster NIM, keep the\nopenai/ prefix there and set NIM_SERVED_MODEL_NAME to the raw model\nname advertised by the NIM. Replace the default Super-49B modelProfile,\nresources, and env when the target model requires a different\nGPU/profile setup. Leaving modelProfile empty preserves NIM\nOperator auto-discovery, but for Nano it can cache every advertised\nprofile on first reconciliation; pin a known-compatible profile when you\nknow the target GPU topology.

serviceConfig.llm.apiBase and serviceConfig.llm.model can be set\nexplicitly to point /v1/answer at an external OpenAI-compatible LLM\ninstead of deploying an answer LLM in-cluster. For external credentials,\ncreate a Kubernetes Secret and set serviceConfig.llm.apiKeySecret.name\nplus serviceConfig.llm.apiKeySecret.key; Helm mounts the Secret as an\nenvironment variable instead of writing the key into the ConfigMap.

nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning deploys Omni for\nimage captioning only. It does not enable /v1/answer. Omni\n(nvidia/nemotron-3-nano-omni-30b-a3b-reasoning) is a supported\nconfigurable VLM-capable answer-generation backend. Use one of the\nfollowing:

Override the generic answer_llm slot with the Omni image\nnvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant,\nset nimOperator.answer_llm.model to\nopenai/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning, set\nNIM_SERVED_MODEL_NAME to nvidia/nemotron-3-nano-omni-30b-a3b-reasoning,\nand size resources plus NIM_TENSOR_PARALLEL_SIZE from the Omni rows in the\nSupport Matrix\n(one GPU on 80 GB or better, two GPUs on L40S). Leave modelProfile\nempty for NIM Operator auto-discovery, or pin a profile for your GPU.
If the Omni caption NIM is already in the cluster, reuse it for\n/v1/answer without deploying Super-49B:

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true \\\n  --set serviceConfig.llm.enabled=true \\\n  --set serviceConfig.llm.apiBase=http://nemotron-3-nano-omni-30b-a3b-reasoning:8000/v1 \\\n  --set serviceConfig.llm.model=openai/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning\n

Enabling caption Omni and the default Super-49B answer_llm as separate\nNIMServices adds their GPU and disk requirements. Reusing the caption\nOmni endpoint for /v1/answer does not add a second Omni GPU or cache.

Agentic retrieval (self-hosted Super-49B) { #agentic-retrieval-llm }

nimOperator.answer_llm.enabled=true deploys Super-49B and auto-wires\nit only to serviceConfig.llm for POST /v1/answer. That answer path\nsends a plain text-generation request and does not require tool\ncalling. serviceConfig.agentic is a separate block. The chart does\nnot populate it from answer_llm.

The default Super-49B NIM starts with\nNIM_PASSTHROUGH_ARGS=--disable-custom-all-reduce. Agentic retrieval\nsends OpenAI-style tool-call messages with tool_choice=auto. A\nself-hosted vLLM-backed Super-49B NIM rejects those requests with\nHTTP 400 unless you also pass --enable-auto-tool-choice and\n--tool-call-parser llama3_json.

You can reuse the same Super-49B NIM for agentic retrieval after you\nadd those arguments. POST /v1/answer continues to work. This gap\ndoes not apply to NVIDIA-hosted Build endpoints.

If you set nimOperator.answer_llm.env in a values file, include\nthe full list. Change only the passthrough value:

nimOperator:\n  answer_llm:\n    enabled: true\n    env:\n      - name: NIM_HTTP_API_PORT\n        value: \"8000\"\n      - name: NIM_TENSOR_PARALLEL_SIZE\n        value: \"2\"\n      - name: NIM_PASSTHROUGH_ARGS\n        value: \"--disable-custom-all-reduce --enable-auto-tool-choice --tool-call-parser llama3_json\"\n      - name: NCCL_IB_DISABLE\n        value: \"1\"\n      - name: NCCL_P2P_DISABLE\n        value: \"1\"\n\nserviceConfig:\n  agentic:\n    enabled: true\n    llmModel: nvidia/llama-3.3-nemotron-super-49b-v1.5\n    invokeUrl: http://answer-llm:8000/v1/chat/completions\n

Equivalent --set form when you do not use a values file.\nHelm --set replaces the env list, so include every Super-49B\nenvironment entry and change only the NIM_PASSTHROUGH_ARGS value:

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.answer_llm.enabled=true \\\n  --set nimOperator.answer_llm.env[0].name=NIM_HTTP_API_PORT \\\n  --set-string nimOperator.answer_llm.env[0].value=8000 \\\n  --set nimOperator.answer_llm.env[1].name=NIM_TENSOR_PARALLEL_SIZE \\\n  --set-string nimOperator.answer_llm.env[1].value=2 \\\n  --set nimOperator.answer_llm.env[2].name=NIM_PASSTHROUGH_ARGS \\\n  --set-string nimOperator.answer_llm.env[2].value=\"--disable-custom-all-reduce --enable-auto-tool-choice --tool-call-parser llama3_json\" \\\n  --set nimOperator.answer_llm.env[3].name=NCCL_IB_DISABLE \\\n  --set-string nimOperator.answer_llm.env[3].value=1 \\\n  --set nimOperator.answer_llm.env[4].name=NCCL_P2P_DISABLE \\\n  --set-string nimOperator.answer_llm.env[4].value=1 \\\n  --set serviceConfig.agentic.enabled=true \\\n  --set serviceConfig.agentic.llmModel=nvidia/llama-3.3-nemotron-super-49b-v1.5 \\\n  --set serviceConfig.agentic.invokeUrl=http://answer-llm:8000/v1/chat/completions\n

serviceConfig.agentic.llmModel is the model ID advertised by the\nNIM, not the LiteLLM openai/ prefix used by\nserviceConfig.llm.model. Change invokeUrl if you override\nnimOperator.answer_llm.nimServiceName.

After the NIM is Ready, confirm the passthrough arguments:

deploy/answer-llm -- printenv NIM_PASSTHROUGH_ARGS\n\">

kubectl exec -n <namespace> deploy/answer-llm -- printenv NIM_PASSTHROUGH_ARGS\n

The value must include --enable-auto-tool-choice and\n--tool-call-parser llama3_json. For one-shot CLI use, port-forward\nservice/answer-llm and point --agentic-invoke-url at\nhttp://localhost:9000/v1/chat/completions. For the CLI command,\nservice request, and MCP notes, refer to\nSelf-hosted Helm Super-49B.

For other self-hosted OpenAI-compatible NIMs, enable automatic tool\nchoice and the parser that model requires. The llama3_json parser\nis the verified Super-49B setting.

NIM Operator sub-stack

Each enabled NIM block under nimOperator.<key> renders operator resources\ngated on three conditions ALL holding:

The apps.nvidia.com/v1alpha1 CRDs are installed in the cluster.
The master switch nims.enabled is true.
The per-NIM nimOperator.<key>.enabled is true.

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Default	Notes
`nims.enabled`	`true`	Master switch. Set false to render no NIM resources.
`nimOperator.page_elements.enabled`	`true`	Page Elements 2.0 service; auto-wired to `/v1/page-elements`.
`nimOperator.table_structure.enabled`	`true`	Table Structure 2.0 service; auto-wired to `/v1/table-structure`.
`nimOperator.<page_elements	table_structure>.image`	`nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1`
`nimOperator.ocr.enabled`	`true`	OCR NIM.
`nimOperator.ocr.image`	`nvcr.io/nim/nvidia/nemotron-ocr-v2:2.0.1`	Default OCR NIM image.
`nimOperator.vlm_embed.enabled`	`true`	Multimodal embedding NIM (also used by the vectordb Pod).
`nimOperator.vlm_embed.nimServiceName`	`llama-nemotron-embed-vl-1b-v2`	NIMService / in-cluster DNS name.
`nimOperator.vlm_embed.image`	`nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:2.3.0`	Default VLM embed NIM image.
`nimOperator.vlm_embed.env`	`NIM_HTTP_API_PORT=8000`, `NIM_TRITON_LOG_VERBOSE=1`, `OMP_NUM_THREADS=1`, `NIM_ENGINE_PRECISION=fp16`	Environment for the default VLM embed NIM. Overrides replace the complete list.
`nimOperator.rerankqa.enabled`	`false`	VL reranker NIM (optional). Set `true` to opt in — refer to Query-time reranking. Default `false` so chart installs honor the \"optional and disabled by default\" contract in deployment-options.md and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the VL SKU (`llama-nemotron-rerank-vl-1b-v2`) per prerequisites-support-matrix.md — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR.
`nimOperator.rerankqa.image`	`nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:2.3.0`	Default optional VL reranker NIM image.
`nimOperator.nemotron_parse.enabled`	`false`	Structured-parse NIM (optional). Set `true` when using `method=\"nemotron_parse\"`. Default `false` so chart installs honor the \"optional and disabled by default\" contract in deployment-options.md. Image tags follow the image tag conventions.
`nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled`	`false`	Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to Image captioning (Omni 30B). This VLM is also a supported configurable `/v1/answer` backend. Enabling this key does not enable `/v1/answer`. Refer to Answer generation (operator-managed LLM). Default `false` so chart installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the image tag conventions.
`nimOperator.answer_llm.enabled`	`false`	Generic answer-generation LLM NIM (optional; Super-49B defaults). Set `true` to enable `/v1/answer` — refer to Answer generation (operator-managed LLM). This opt-in does not enable agentic retrieval. Refer to Agentic retrieval (self-hosted Super-49B). Default `false` so installs do not silently claim answer-generation GPUs.
`nimOperator.answer_llm.model`	`openai/nvidia/llama-3.3-nemotron-super-49b-v1.5`	LiteLLM/OpenAI model id inherited by `serviceConfig.llm.model` when the operator-managed answer LLM is enabled and no explicit service model is set.
`nimOperator.answer_llm.ragSystemPromptPrefix`	`\"\"`	Optional prompt prefix inherited by `serviceConfig.llm.ragSystemPromptPrefix` only when explicitly set. Leave empty to keep the operator-managed LLM model-neutral and use `serviceConfig.llm.reasoningEnabled` for request-level reasoning control.
`nimOperator.audio.enabled`	`false`	Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it.
`nimOperator.<key>.image.repository`	`nvcr.io/nim/nvidia/...`	Per-NIM image.
`nimOperator.<key>.image.pullSecrets`	`[]`	Per-NIM pull Secret name list. Empty inherits `ngcImagePullSecret.name` (default `ngc-secret`) on every NIMCache / NIMService. Non-empty replaces the chart-wide name for that NIM only.
`nimOperator.<key>.authSecret`	`\"\"`	Per-NIM auth Secret name. Empty inherits `ngcApiSecret.name` (default `ngc-api`). Non-empty replaces the chart-wide name for that NIM only.
`nimOperator.<key>.storage.pvc.size`	`25Gi` (50Gi for vlm_embed/rerankqa, 100Gi parse, 300Gi VL)	NIMCache PVC size.
`nimOperator.<key>.storage.pvc.storageClass`	`\"\"`	Per-NIM NIMCache StorageClass. An empty value renders an empty class on the NIMCache CR, so the operator-created claim uses the cluster default when one exists. Set this path for each enabled NIM. `nimOperator.nimCache.pvc.storageClass` is not applied to per-NIM caches.
`nimOperator.<key>.replicas`	`1`	Per-NIMService replica count.
`nimOperator.nimServiceGpuLimit`	`1`	Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Four core NIMs therefore request four GPU slots unless the cluster shares GPUs. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions). Refer to GPU limits and `helm upgrade` and GPU scheduling prerequisite.
`nimOperator.<key>.resources`	`{}`	Per-NIM override of the whole `resources` block. Empty uses `nimServiceGpuLimit`; non-empty replaces the chart default (may require `--force-conflicts` on later `helm upgrade`).
`nimOperator.modelProfile`	`{}`	Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. Refer to Filtering cached GPU profiles.
`nimOperator.<key>.modelProfile`	`{}`	Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). Refer to Filtering cached GPU profiles.
`nimOperator.<key>.expose.service.port`	`8000` (9000 for audio)	HTTP port.
`nimOperator.<key>.expose.service.grpcPort`	`8001` (50051 for audio)	gRPC port.

\n
The four \"core\" NIMs (page_elements, table_structure, ocr, vlm_embed)\nare enabled and auto-wired by default. Optional NIMs stay off until\nnimOperator.<key>.enabled is true. When you opt in, the chart\nauto-wires Omni captioning and VL reranking into nim_endpoints\n(refer to Image captioning (Omni 30B) and\nQuery-time reranking); other optional NIMs\nstill need an explicit serviceConfig hook (for example\naudioGrpcEndpoint for Parakeet ASR). For minimal installs, prefer the\nminimal install overrides.
\n

Filtering cached GPU profiles { #filtering-cached-gpu-profiles }

Every NIMCache the chart renders supports the NIM Operator's\nspec.source.ngc.model block, which restricts which model profiles the\ncache job downloads. The chart exposes this through two values:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Scope	Behaviour
`nimOperator.modelProfile`	Chart-wide	Applied to every NIMCache that doesn't carry its own override.
`nimOperator.<key>.modelProfile`	Per-NIM	When non-empty, REPLACES the chart-wide default (no merge).

Both default to {}. With both empty the chart emits no model:\nblock and the NIM Operator falls back to its \"cache every profile\napplicable to the detected GPUs\" default — fine on a single-GPU\nlaptop, but on heterogeneous clusters (or any cluster with ≥ 3 NIMs)\nthis wastes tens of GiB of PVC storage, NGC bandwidth, and cache-job\nruntime.

The mapping is rendered verbatim under spec.source.ngc.model, so the\nshape lines up 1:1 with the NIMCache CRD.\nTwo filter dimensions are supported (use whichever fits your cluster;\ngpus is the common case):

/:`.\n # - "11111111-2222-3333-4444-555555555555"\n\">

nimOperator:\n  modelProfile:\n    gpus:\n      # NIMCache only downloads profiles compatible with at least one\n      # of these GPU selectors. Each selector is {ids: [...], product: ...}.\n      - ids: [\"26B5\"]                       # PCI device ID(s)\n        product: \"NVIDIA-H100-80GB-HBM3\"    # NVIDIA marketing name\n    # profiles:\n    #   # Alternative: list of exact profile UUIDs from `ngc registry\n    #   # model list-profiles <repo>/<image>:<tag>`.\n    #   - \"11111111-2222-3333-4444-555555555555\"\n

Equivalent overrides via --set:

# Homogeneous H100 80 GB cluster — every NIMCache only pulls the H100 profile:\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n  --set 'nimOperator.modelProfile.gpus[0].ids[0]=26B5' \\\n  --set 'nimOperator.modelProfile.gpus[0].product=NVIDIA-H100-80GB-HBM3'\n\n# Restrict only the page_elements NIMCache to a specific profile UUID, leave the rest alone:\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n  --set 'nimOperator.page_elements.modelProfile.profiles[0]=11111111-2222-3333-4444-555555555555'\n\n# Chart-wide H100 default plus a per-NIM override (the override REPLACES the global; it does NOT merge):\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n  --set 'nimOperator.modelProfile.gpus[0].product=NVIDIA-H100-80GB-HBM3' \\\n  --set 'nimOperator.vlm_embed.modelProfile.profiles[0]=22222222-3333-4444-5555-666666666666'\n

Tips:

Run ngc registry model list-profiles nvcr.io/nim/nvidia/<image>:<tag> to enumerate the available profiles for any chart-pinned NIM image and pick the smallest profile that matches your GPU.
Filter mismatches surface as NIMCache events such as NoCompatibleProfile; check with kubectl describe nimcache <name>.
The chart's defaults ({}) preserve operator behaviour, so adding modelProfile is a strict opt-in — existing releases keep working unchanged.

Image tag conventions { #image-tag-conventions }

Every NIM in this chart pins an exact NGC image tag in values.yaml\n— there is no :latest floating reference. Two tag families show up:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Family	Example	Meaning
Plain semver	`nemotron-object-detection:2.0.1`	A standard NIM release, identical bytes on every pull. Used by the four core NIMs and the reranker / ASR NIMs.
`<semver>-variant`	`nemotron-parse-v1.2:1.7.0-variant`, `nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant`	The Nemotron Parse and Nemotron 3 Nano Omni 30B builds that ship per-GPU TensorRT engine variants the NIM Operator selects from at reconciliation time (refer to the Omni and Parse rows in the model hardware requirements table). The `-variant` suffix is the NGC tag that ships alongside this chart and matches footnote ³ of the support matrix.

For air-gapped mirror pipelines: mirror the exact tag — both the\nplain semver and the -variant form — and do not substitute :latest.\nSubstituting :latest would pin to a moving target that may not match\nthe engine plans the NIM Operator profile expects for a given GPU.

If you want a different NIM build on a new install, override the tag\nexplicitly:

\n\">

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true \\\n  --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.image.tag=<your-tag>\n

and validate against the same release of the retriever service before\nproduction rollout.

If a NIMCache for that NIM already exists, do not change\nimage.repository or image.tag with helm upgrade alone.\nThe NIM Operator rejects in-place modelPuller updates.\nDelete the cache first, then upgrade. Refer to\nChanging a NIM image repository or tag.

Charts and captioning. Charts and infographics use Page Elements, Table Structure\nand ocr. For image\ncaptioning, set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true — refer to\nImage captioning (Omni 30B) for the\nchart-side wiring and\nImage captioning\nfor the product matrix.

Changing a NIM image repository or tag { #changing-nim-image-repository-or-tag }

Each chart NIM renders a NIMCache whose metadata.name stays the same\nacross image changes. spec.source.ngc.modelPuller is the concatenation\nof nimOperator.<key>.image.repository and nimOperator.<key>.image.tag.\nThe NIM Operator NIMCache CRD (including 3.1.2) marks modelPuller\nimmutable. Kubernetes rejects an update with a message similar to:

modelPuller is an immutable field. Please create a new NIMCache resource instead when you want to change this container.\n

A helm upgrade that only changes the repository or tag fails\non the existing object. Other release resources can already have been\napplied before that rejection, which leaves the release partially upgraded.

Do not rename nimOperator.<key>.nimServiceName as a workaround.\nA new name creates a second cache and Service DNS label. The previous\nNIMCache remains, especially when nimOperator.nimCache.keepOnUninstall\nis true (the default).

The following table lists default NIMCache names for the core NIMs.\nOptional NIMs follow the same immutable-modelPuller rule.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Helm key	Default `NIMCache` name
`nimOperator.page_elements`	`nemotron-page-elements-v3`
`nimOperator.table_structure`	`nemotron-table-structure-v1`
`nimOperator.ocr`	`nimOperator.ocr.nimServiceName` (`nemotron-ocr-v2`)
`nimOperator.vlm_embed`	`nimOperator.vlm_embed.nimServiceName` (`llama-nemotron-embed-vl-1b-v2`)

Before you change a repository or tag on an existing release,\ncomplete the following steps for every NIM whose image changes.\nThe affected NIM is unavailable while the operator re-caches weights.

\n
Drain ingest traffic that depends on that NIM.
\n

Confirm the live modelPuller value differs from the new\nrepository:tag:

NS=retriever\nCACHE=nemotron-page-elements-v3\n\nkubectl get nimcache \"${CACHE}\" -n \"${NS}\" \\\n  -o jsonpath='{.metadata.name}{\" \"}{.spec.source.ngc.modelPuller}{\"\\n\"}'\n

Compare that modelPuller value with\n<new-repository>:<new-tag>.

\n
Delete the NIMCache. Helm keep annotations do not block\nkubectl delete:
\n
```
kubectl delete nimcache \"${CACHE}\" -n \"${NS}\"\n
```
\n
\n
If the operator-created PVC remains, delete it so the new image\nre-pulls weights. List operator PVCs, then delete the claim for\nthat cache. Default claim names use a -pvc suffix, for example\nnemotron-page-elements-v3-pvc. Confirm the name from the list\nbefore you delete it. Refer to\nPersistent storage prerequisite.
\n
```
kubectl get pvc -n \"${NS}\" -l 'app.kubernetes.io/managed-by=nvidia-nim-operator'\nkubectl delete pvc \"${CACHE}-pvc\" -n \"${NS}\"\n
```
\n
\n
Run helm upgrade with the new image.repository or image.tag.\nHelm creates a new NIMCache with the updated modelPuller.
\n

Wait until the new cache is ready before you send traffic:

kubectl get nimcache \"${CACHE}\" -n \"${NS}\"\nkubectl get nimservice \"${CACHE}\" -n \"${NS}\"\nkubectl describe nimcache \"${CACHE}\" -n \"${NS}\"\n

Repeat those steps for each NIM whose repository or tag changes,\nincluding chart default tag bumps between releases.

helm uninstall does not remove kept NIMCache objects when\nkeepOnUninstall is true. A later install or upgrade with a\ndifferent image still requires this delete and re-cache sequence.

If helm upgrade already failed with the immutable-modelPuller\nmessage, delete the existing NIMCache and its PVC, then re-run the\nsame upgrade so Helm creates the cache instead of patching it.

Changing service.image.repository or service.image.tag does not\nuse NIMCache and is not subject to this rule.

Query-time reranking { #query-time-reranking }

The VL reranker NIM (llama-nemotron-rerank-vl-1b-v2) backs\nPOST /v1/query with rerank=true. When you enable it,

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.rerankqa.enabled=true \\\n  ...\n

the chart auto-wires two fields into the rendered\nretriever-service.yaml ConfigMap (including every split-topology\nrole ConfigMap):

nim_endpoints:\n  rerank_invoke_url: \"http://llama-nemotron-rerank-vl-1b-v2:8000/v1/ranking\"\n  rerank_model_name: \"nvidia/llama-nemotron-rerank-vl-1b-v2\"\n

Without those fields the gateway returns\nHTTP 400 Reranking is not configured even when the Rerank NIM Pod is\nReady.

Resolution order mirrors every other NIM endpoint (see the\nNIM Operator sub-stack section):

Explicit serviceConfig.nimEndpoints.rerankInvokeUrl always wins\n(use this to point at a hosted or external ranking endpoint).
Otherwise the operator-managed URL of\nllama-nemotron-rerank-vl-1b-v2 is used, provided\nnimOperator.rerankqa.enabled=true and the\napps.nvidia.com/v1alpha1 CRDs are installed.
Otherwise rerank_invoke_url stays null and query-time reranking\nstays disabled.

serviceConfig.nimEndpoints.rerankModelName follows the same order —\nit defaults to the canonical VL reranker model id\n(nvidia/llama-nemotron-rerank-vl-1b-v2) whenever the chart resolves any\nrerank URL. Override only when pointing at a different ranking SKU.

Image captioning (Omni 30B) { #image-captioning-omni-30b }

The Nemotron 3 Nano Omni VLM is the canonical image-caption NIM for\nthis chart. When you enable it,

helm upgrade --install retriever ./nemo_retriever/helm \\\n  --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true \\\n  ...\n

the chart now auto-wires two fields into the rendered\nretriever-service.yaml ConfigMap:

nim_endpoints:\n  caption_invoke_url: \"http://nemotron-3-nano-omni-30b-a3b-reasoning:8000/v1/chat/completions\"\n  caption_model_name: \"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning\"\n

The service derives caption_enabled=true from a non-null\ncaption_invoke_url, so the ingestion pipeline routes caption work to\nthe in-cluster Omni Pod with no manual ConfigMap edits.

Resolution order mirrors every other NIM endpoint (see the\nNIM Operator sub-stack section):

Explicit serviceConfig.nimEndpoints.captionInvokeUrl always wins\n(use this to point at a hosted endpoint, e.g.\nhttps://integrate.api.nvidia.com/v1/chat/completions).
Otherwise the operator-managed URL of\nnemotron-3-nano-omni-30b-a3b-reasoning is used, provided\nnimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true\nand the apps.nvidia.com/v1alpha1 CRDs are installed.
Otherwise caption_invoke_url stays null and the caption stage\nis disabled.

serviceConfig.nimEndpoints.captionModelName follows the same order —\nit defaults to the canonical Omni remote model id\n(nvidia/nemotron-3-nano-omni-30b-a3b-reasoning, matching\nnemo_retriever.common.modality.caption.model_profiles.OMNI_REMOTE_MODEL_ID) whenever\nthe chart resolves any caption URL. Override only when pointing at a\ndifferent VLM SKU.

GPU limits and `helm upgrade` { #gpu-limits-and-helm-upgrade }

The chart defaults to nimOperator.nimServiceGpuLimit: 1, which\nrenders spec.resources.limits.nvidia.com/gpu: 1 on every NIMService\nunless a per-NIM resources map overrides it. A default core install\ntherefore requests four GPU slots (one per enabled NIMService).\nSetting that per-NIM limit is required on NIM Operator v3.1.2\n(and other versions tested on A100/H100): when the chart omits the\nresources block entirely, the operator often does not populate\nGPU limits from the model profile, and NIM pods start without GPU\naccess (The NVIDIA Driver was not detected).

This per-NIM 1 request is not the support-matrix VRAM figure of\none A10G for all four core models combined. On a conventional\ncluster, plan for four allocatable GPUs across eligible nodes or\nconfigure sharing.\nRefer to GPU scheduling prerequisite.

Trade-off: Helm and the NIM Operator may both server-side-apply\nspec.resources.limits.nvidia.com/gpu. A later helm upgrade --install\ncan then fail with:

/ apps.nvidia.com/v1alpha1, Kind=NIMService:\n Apply failed with 1 conflict:\n conflict with "manager" using apps.nvidia.com/v1alpha1:\n .spec.resources.limits.nvidia.com/gpu\n\">

Error: UPGRADE FAILED: conflict occurred while applying object\n  <ns>/<nim> apps.nvidia.com/v1alpha1, Kind=NIMService:\n  Apply failed with 1 conflict:\n  conflict with \"manager\" using apps.nvidia.com/v1alpha1:\n    .spec.resources.limits.nvidia.com/gpu\n

Operator-only mode (omit GPU limits from Helm — only if your NIM\nOperator version reliably reconciles them):

nimOperator:\n  nimServiceGpuLimit: null\n

If upgrades hit SSA conflicts after the operator has reconciled GPU\nlimits, use one of:

helm upgrade --install … --force-conflicts --server-side
kubectl -n <ns> edit nimservice <name> to set GPU limits outside Helm

To pin a non-default GPU count chart-wide, set nimServiceGpuLimit: 2\n(or set per-NIM resources.limits.nvidia.com/gpu).

A failed upgrade that reports an immutable modelPuller field is a\ndifferent class of error. Refer to\nChanging a NIM image repository or tag.

OCR NIM configuration { #ocr-nim-configuration }

The core OCR NIM is configured under nimOperator.ocr (the ocr:\nblock). Confirm image.repository and image.tag before you upgrade.\nIf either value changes on an existing release, delete the OCR NIMCache\nbefore you upgrade. Refer to\nChanging a NIM image repository or tag.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Role
`nimOperator.nimCache.keepOnUninstall`	When `true`, NIMCache CRs survive `helm uninstall` (`helm.sh/resource-policy: keep`). NIMService CRs are always removed. Set `false` for dev clusters that should fully tear down on uninstall.
`nimOperator.ocr.enabled`	Reconcile the OCR `NIMService`
`nimOperator.ocr.image.repository`	NIM image (default `nvcr.io/nim/nvidia/nemotron-ocr-v2`)
`nimOperator.ocr.image.tag`	Pin the image tag for reproducible upgrades

Override the auto-wired in-cluster URL with serviceConfig.nimEndpoints.ocrInvokeUrl\nwhen the OCR service runs outside the operator sub-stack.

Persistence

Complete the persistent storage prerequisite\nbefore a default install. Empty storageClass values omit\nstorageClassName and rely on a default StorageClass or a compatible\nclassless persistent volume.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Default	Notes
`persistence.enabled`	`true`	Mount the pre-existing general PVC for logs and other non-scheduler uses.
`persistence.size`	`50Gi`
`persistence.accessModes`	`[ReadWriteOnce]`	Access mode for the general PVC.
`persistence.storageClass`	`\"\"`	Use cluster default unless set. Use `\"-\"` to disable a `storageClassName`.
`persistence.existingClaim`	`\"\"`	When set, skip PVC creation and mount this claim.
`persistence.mountPath`	`/var/lib/nemo-retriever`	General persistent files only; scheduler state and payloads are never stored here.
`retrieverResults.enabled`	`true`	Create the results PVC unless `existingClaim` is set.
`retrieverResults.storageClass`	`\"\"`	Use cluster default unless set. Use `\"-\"` to disable a `storageClassName`.
`retrieverResults.existingClaim`	`\"\"`	When set, skip PVC creation and mount this claim.
`topology.vectordb.persistence.enabled`	`true`	Create the VectorDB PVC when `serviceConfig.vectordb.enabled` is `true`.
`topology.vectordb.persistence.storageClass`	`\"\"`	Use cluster default unless set. Use `\"-\"` to disable a `storageClassName`. No `existingClaim` path.

The gateway enforces active-lease budgets independently of worker replicas.\nserviceConfig.workQueue.maxActiveLeases.realtime defaults to 8 and\n.batch defaults to 48; treat these as explicit downstream-capacity budgets,\nnot values inferred from pod or NIM counts.

Scheduler loss boundary and upgrade from durable releases

The work spool always uses serviceConfig.workQueue.spoolDirectory under the\ngateway /tmp emptyDir; persistence.enabled does not change scheduler\nbehavior. During one gateway lifetime, FIFO order, lease caps, delivery attempts,\ngenerations, stale-lease rejection, worker recovery after lease expiry, and\nqueued-plus-active demand metrics remain available.

Replacing the gateway loses accepted jobs, queued payloads, active leases, job\nstatus history, and SSE catch-up state. After replacement, status and event\nrequests for old jobs return not found, and old callbacks and heartbeats return\n409. Clients must create a new job and submit the documents again. Worker loss\nremains recoverable through lease expiry only while the same gateway process is\nalive. Public ingest and worker HTTP wire formats are unchanged.

Before upgrading from a release with durable scheduler checkpoints, drain the\ngateway. The ephemeral implementation neither reads nor automatically deletes\ngateway-state.sqlite3 or payload files left under an older PVC. After rollback\nto that release is no longer required, operators may manually remove its old\nwork-queue directory from the general PVC. The removed\nwork_queue.persistence_enabled key was internal and unreleased; delete it from\ncustom service configuration files.

Secrets

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Path	Default	Notes
`ngcImagePullSecret.create`	`false`	Chart-managed dockerconfigjson Secret.
`ngcImagePullSecret.name`	`ngc-secret`	Name referenced by every Pod and, when per-NIM `image.pullSecrets` is empty, by every NIMCache / NIMService.
`ngcImagePullSecret.password`	`\"\"`	NGC API key.
`ngcApiSecret.create`	`false`	Chart-managed Opaque Secret.
`ngcApiSecret.name`	`ngc-api`	Name referenced by NIMCache/NIMService `authSecret` when per-NIM `authSecret` is empty.
`ngcApiSecret.password`	`\"\"`	NGC API key (populates `NGC_API_KEY` + `NGC_CLI_API_KEY`).
`imagePullSecrets`	`[]`	Extra pre-existing pull secrets appended to every Pod.
`serviceConfig.vectordb.internalAuth.enabled`	`false`	Enable the Secret-backed credential for VectorDB traffic and restricted gateway-to-worker handoffs.
`serviceConfig.vectordb.internalAuth.existingSecret.name`	`\"\"`	Existing Secret shared by Retriever and VectorDB pods.
`serviceConfig.auth.scopeTokenSecret.name`	`\"\"`	Existing Secret containing the public scope-token JSON file.
`serviceConfig.auth.enabled`	`false`	Require bearer authentication for the public gateway.
`serviceConfig.auth.allowInsecureInlineApiToken`	`false`	Explicit development-only gate for ConfigMap-backed `apiToken`.

Optional features

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Feature	Toggle	Default
Ingress	`ingress.enabled`	`true`
Autoscaling (HPA)	`autoscaling.enabled`	`false` (max=1 anyway)
ServiceMonitor	`serviceMonitor.enabled`	`false`

Configuration recipes

Mount a custom retriever-service.yaml verbatim

The chart renders retriever-service.yaml from structured values so you\nshouldn't normally need to ship a verbatim file. If you really want to,\nmount one via service.extraVolumes + service.extraVolumeMounts at\n/etc/nemo-retriever/retriever-service.yaml (which silently overrides the\nchart-managed ConfigMap because subPath mounts win).

Use externally managed Secrets

ngcImagePullSecret:\n  create: false        # don't render; reference an existing Secret\n  name: my-org-ngc-pull\nngcApiSecret:\n  create: false\n  name: my-org-ngc-api\n

The chart will skip Secret creation. Make sure my-org-ngc-pull exists\nas kubernetes.io/dockerconfigjson and my-org-ngc-api as Opaque with\nan NGC_API_KEY key, in the release namespace. Retriever Pods and every\nrendered NIMCache / NIMService inherit those names unless you set a\nnon-empty per-NIM image.pullSecrets or authSecret override.

Protect the public gateway and internal service calls with separate\npre-existing Secrets:

serviceConfig:\n  auth:\n    scopeTokenSecret:\n      name: nrl-public-auth\n      key: scope-tokens.json\n    enabled: true\n  vectordb:\n    internalAuth:\n      enabled: true\n      existingSecret:\n        name: nrl-internal-vdb-auth\n        key: token\n

nrl-public-auth must contain a JSON document such as\n{\"tokens\":[{\"token\":\"<secret>\",\"scopes\":[\"workspace-123\"]}]} under the\nconfigured key. nrl-internal-vdb-auth must contain a distinct, high-entropy\ncredential. In split topology, the public token file mounts only on the\ngateway. The gateway authenticates public requests, then uses the internal\ncredential for restricted worker handoffs and pull-worker claims of\n/v1/internal/work/claim. Workers do not receive or validate the public\nbearer token, so serviceConfig.auth.scopeTokenSecret.name also requires\nserviceConfig.vectordb.internalAuth.enabled=true. Internal authentication\nis opt-in for local compatibility; enable it for production deployments. When\nenabled, a missing Secret or key prevents the pods from starting instead of\nfalling back to unauthenticated VectorDB access. Inline\nserviceConfig.auth.apiToken is rejected unless\nallowInsecureInlineApiToken=true, and must never be used for production.

Disable one NIM and supply an external URL for it

nimOperator:\n  vlm_embed:\n    enabled: false   # don't deploy the embed NIM in-cluster\n\nserviceConfig:\n  nimEndpoints:\n    embedInvokeUrl: https://integrate.api.nvidia.com/v1/embeddings\n

The chart's resolution order is explicit URL → operator-managed URL →\nempty, so per-endpoint overrides Just Work.

Roll the service after editing values

The Deployment carries a checksum/config annotation derived from the\nConfigMap, so helm upgrade automatically rolls the pod when any\nserviceConfig.* value changes.

Timeouts and alleviating ingest failures

Batch ingest fans out extract and embed work to remote NIM HTTP endpoints.\nUnder heavy parallelism a single slow or overloaded NIM can cause timeouts,\nand a worker process crash can surface as many simultaneous failed\ndocument callbacks even though only one root cause occurred.

What the chart configures

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Layer	Default	Where it is set
Remote embed HTTP calls	600 s (10 min)	Service image (`EmbedParams.request_timeout_s`); not a Helm value today.
Gateway → realtime/batch proxy	300 s	Rendered `gateway.timeout_s` in `retriever-service.yaml` (split topology).
Worker → vectordb record write	300 s	`serviceConfig.vectordb.writeTimeoutSeconds`, rendered as `vectordb.write_timeout_s` in `retriever-service.yaml`.
VLM embed model name	`serviceConfig.vectordb.embedModel`	Also copied into worker `nim_endpoints.embed_model_name` in the ConfigMap.

Symptoms to look for in pod logs:

Embedding error occurred: timed out or httpx.ReadTimeout on the batch pod.
Batch process pool broken (worker crash) followed by many\nBrokenProcessPool failures on other in-flight documents.
Embed NIM pod messages such as failed to allocate pinned system memory\n(GPU pressure from too many concurrent /v1/embeddings requests).
Failed to POST <n> records to vectordb for <file> on the batch or\nrealtime pod, followed by a failed document. The vectordb Pod did not\nacknowledge the record write in time.

The gateway pod usually only logs status=failed callbacks; diagnose on\nbatch (and realtime for page-sized uploads), plus the embed NIM pod.

Recommended mitigations

1. Lower batch worker concurrency (first step).

The default serviceConfig.pipeline.batchWorkers is 48, which can saturate\na single in-cluster VLM embed NIM. If you see embed timeouts or pool crashes,\nreduce batch parallelism to 16 and redeploy:

helm upgrade retriever ./nemo_retriever/helm \\\n  --reuse-values \\\n  --set serviceConfig.pipeline.batchWorkers=16\n

You can tune further (for example 8 on small GPU nodes), but 16 is a\nreasonable starting point when moving off the default. Realtime workers\n(realtimeWorkers, default 24) are less likely to overload embed NIMs\nbecause they handle smaller units of work; adjust them only if realtime\ningest shows the same timeout pattern.

2. Confirm embed wiring.

Ensure nim_endpoints.embed_model_name in the mounted config matches the\nVLM embed NIM SKU (serviceConfig.vectordb.embedModel, default\nnvidia/llama-nemotron-embed-vl-1b-v2). A model mismatch produces\nHTTP 404 on /v1/embeddings, not a timeout, but is worth ruling out when\ndebugging failed ingests.

3. Retry failed documents.

Failures caused by a one-time pool restart are often transient. After lowering\nbatchWorkers and rolling the batch Deployment, resubmit documents that\nfailed with rows=0.

4. Scale or isolate the embed NIM.

If timeouts persist at batchWorkers: 16, add embed NIM replicas (when your\ncluster has GPU capacity), point serviceConfig.nimEndpoints.embedInvokeUrl\nat an external embed endpoint, or temporarily disable optional NIMs on\ndev clusters to free GPU memory for vlm_embed.

5. Client and ingress timeouts.

Long batch jobs may exceed the gateway proxy timeout (300 s) or an Ingress\nproxy-read-timeout. Increase ingress annotations if clients disconnect\nwhile workers are still processing; see the commented example on\ningress.annotations in values.yaml.

6. Raise the vectordb write timeout on slow storage.

A worker fails the document when the vectordb Pod does not acknowledge its\nrecord write:

. The extracted rows are not\nconfirmed durable, so the document is not queryable.\n\">

VectorDB write failed for report.pdf: <error>. The extracted rows are not\nconfirmed durable, so the document is not queryable.\n

That acknowledgement covers the row commit plus the index maintenance that\nfollows it, so a large batch on slow storage can exceed the default 300 s.\nRaise the timeout and redeploy:

helm upgrade retriever ./nemo_retriever/helm \\\n  --reuse-values \\\n  --set serviceConfig.vectordb.writeTimeoutSeconds=900\n

Concurrent writes to one vectordb Pod do not block each other, so a write\ntimeout points at storage throughput or table size rather than write\nserialization. Refer to\nLanceDB index creation fails during concurrent Helm ingestion.

Queue-depth autoscaling (split mode)

In topology.mode: split deployments the realtime and batch worker\npods scale horizontally based on the gateway's central outstanding demand and\n95th-percentile processing latency. Demand is queued records plus active leases from the gateway while\nlatency comes from workers; both publishers are always on (see\nnemo_retriever_work_queue_demand in\nprometheus.py).\nThe only choice you have to make is how the metrics get from\nPrometheus into the Kubernetes HPA.

Why queue depth (and not CPU)

CPU-based HPA reacts to the pod that has already saturated its work.\nFor an ingest pipeline that fans out to remote NIM endpoints, the work\nspends most of its time blocked on HTTP — CPU stays low even when the\nqueue is full. Queue depth measures demand to be served, which is\nwhat we actually want to scale on. A 95th-percentile-latency signal\nrides alongside to catch the inverse case (a single hot pod whose\nqueue is shallow but whose per-item processing has stalled).

Backend choices

The chart's autoscaling.queueDepth.backend controls which path is\nwired up. All three options leave the metrics publisher untouched:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

backend	When to pick it	Cluster prerequisite
`prometheus-adapter` (default)	Production. One adapter feeds HPA + Grafana + future autoscalers.	Prometheus Operator + `prometheus-community/prometheus-adapter`.
`cpu`	Bootstrap / dev cluster without Prometheus.	None — built-in.
`keda`	Already standardised on KEDA org-wide.	KEDA operator (you install + apply your own `ScaledObject`).

The chart-recommended path is prometheus-adapter. The reasoning is\ndocumented in values.yaml; in short, it keeps a single Prometheus as\nthe source of truth, supports HPA's multi-metric arithmetic-mean\nevaluation out of the box, and doesn't force the chart to bundle new\nCRDs.

Wiring up prometheus-adapter (recommended)

The chart renders a ConfigMap named\n<release>-nemo-retriever-prom-adapter-rules containing PromQL rules\nfor the External Metrics API. You point your existing\nprometheus-adapter at it:

-nemo-retriever-prom-adapter-rules\n\">

helm upgrade prometheus-adapter prometheus-community/prometheus-adapter \\\n  --namespace monitoring \\\n  --reuse-values \\\n  --set rules.existing=<release>-nemo-retriever-prom-adapter-rules\n

Then verify both metrics show up in the External Metrics API:

kubectl get --raw \\\n  \"/apis/external.metrics.k8s.io/v1beta1/namespaces/$NS/nemo_retriever_gateway_work_queue_backlog?labelSelector=pool%3Drealtime\" \\\n  | jq .\n

Once that returns a non-empty items array, the HPAs rendered by this\nchart will start consuming them. The HPA annotation\nnemo-retriever.nvidia.com/hpa-signals documents the active set per\nHPA, e.g. queueRatio=true latencyP95=true cpu=false.

CPU fallback (no Prometheus required)

Set autoscaling.queueDepth.backend: cpu and enable the CPU metric\nunder each role:

autoscaling:\n  queueDepth:\n    backend: cpu\ntopology:\n  realtime:\n    hpa:\n      metrics:\n        queueBacklog: { enabled: false }\n        processingLatencyP95: { enabled: false }\n        cpu: { enabled: true, targetUtilizationPercentage: 60 }\n  batch:\n    hpa:\n      metrics:\n        queueBacklog: { enabled: false }\n        processingLatencyP95: { enabled: false }\n        cpu: { enabled: true, targetUtilizationPercentage: 80 }\n

The legacy topology.<role>.hpa.targetCPUUtilizationPercentage field\nstill works and behaves as an alias for the metrics.cpu block.

KEDA path

Set autoscaling.queueDepth.backend: keda and disable the chart-managed\nHPAs:

autoscaling:\n  queueDepth: { backend: keda }\ntopology:\n  realtime: { hpa: { enabled: false } }\n  batch:    { hpa: { enabled: false } }\n

Then apply your own ScaledObject — example for the realtime pool:

apiVersion: keda.sh/v1alpha1\nkind: ScaledObject\nmetadata:\n  name: nemo-retriever-realtime\nspec:\n  scaleTargetRef:\n    name: nemo-retriever-realtime\n  minReplicaCount: 2\n  maxReplicaCount: 8\n  cooldownPeriod: 300\n  triggers:\n    - type: prometheus\n      metadata:\n        serverAddress: http://prometheus.monitoring.svc:9090\n        metricName: nemo_retriever_pool_queue_depth_ratio\n        threshold: \"0.5\"\n        query: |\n          avg by (pool) (\n            nemo_retriever_pool_queue_depth{pool=\"realtime\"}\n            /\n            on(pool, instance) group_left()\n            nemo_retriever_pool_max_queue_size{pool=\"realtime\"}\n          )\n    - type: prometheus\n      metadata:\n        serverAddress: http://prometheus.monitoring.svc:9090\n        metricName: nemo_retriever_pool_processing_duration_p95\n        threshold: \"30\"\n        query: |\n          histogram_quantile(\n            0.95,\n            sum by (le, pool) (\n              rate(nemo_retriever_pool_processing_duration_seconds_bucket{pool=\"realtime\"}[2m])\n            )\n          )\n

KEDA's biggest win is scale-from-zero, which we don't use today —\nboth minReplicas defaults are ≥ 1 because the realtime pod is on the\nhot path for SSE consumers. If you do want scale-from-zero (e.g. a\nnightly batch-only job tenant), KEDA is the right tool and this is the\nescape hatch.

Tuning the thresholds

Per-role tuning lives under topology.<role>.hpa.metrics:

topology:\n  realtime:\n    hpa:\n      metrics:\n        queueBacklog: { enabled: true, target: \"24\" }\n        processingLatencyP95: { enabled: true, targetSeconds: \"30\" }\n  batch:\n    hpa:\n      metrics:\n        queueBacklog: { enabled: true, target: \"48\" }\n        processingLatencyP95: { enabled: true, targetSeconds: \"120\" }\n

The demand target is outstanding documents per replica because the HPA uses\ntype: AverageValue. When migrating an existing values file, rename\nmetrics.queueDepthRatio to metrics.queueBacklog, choose a document-count\ntarget near the role's execution-slot count, and rename\nprometheusAdapter.queueDepthRatioMetric to queueBacklogMetric. Both legacy\nkeys remain aliases for this release: legacy enable/disable values win, legacy\nmetric names name the new demand metric, and fractional ratio targets are\nreplaced by the role's backlog-count default with an HPA annotation and NOTES\nwarning. The aliases are scheduled for removal in the following release.

Verifying it scales

-realtime -o jsonpath='{.metadata.annotations.nemo-retriever\\.nvidia\\.com/hpa-signals}'\n\">

# Cause realtime pressure (anything that submits to /v1/ingest/job/.../page).\n# Then watch the HPA decide:\nkubectl get hpa -w\n\n# And watch the active signals on each HPA:\nkubectl get hpa <release>-realtime -o jsonpath='{.metadata.annotations.nemo-retriever\\.nvidia\\.com/hpa-signals}'\n

The dashboard's Worker Pool Capacity card on the Overview page\nmirrors the same signal Prometheus is seeing, so it's a quick eyeball\nsanity check before opening Grafana.

Tracing and Zipkin

Helm installs the chart-owned OpenTelemetry Collector and Zipkin backend on by\ndefault. This is intentional: the legacy 26.1.2 Helm chart shipped with a\nmanaged Zipkin deployment enabled, so the new chart keeps a default trace\nbackend available for functional parity. OTLP trace and metric export is also\nenabled by default for retriever service pods and chart-managed NIMs:

topology:\n  otel:\n    enabled: true\n  zipkin:\n    enabled: true\n\nservice:\n  otel:\n    enabled: true\n\nnimOperator:\n  otel:\n    enabled: true\n

Because Zipkin is chart-owned by default, an upgrade with default values can\ncreate a Zipkin Deployment and Service. Set topology.zipkin.enabled=false\nbefore upgrading if your deployment uses an external backend or should not run\nchart-owned Zipkin.

With default values, retriever service pods and chart-managed NIMs emit OTLP to\nthe chart's OpenTelemetry Collector. The Collector exports traces to the\nchart-owned Zipkin service and exposes received metrics in Prometheus format.\nThe chart configures a 5-second metric export interval. Set\nservice.otel.enabled=false or nimOperator.otel.enabled=false to opt out by\nsurface. Open a job and read the Zipkin lookup key from either the JSON body or\nthe x-trace-id response header:

kubectl port-forward svc/tracing-smoke-nemo-retriever 7670:80\n\ncurl -s -D headers.txt -o job.json \\\n  -X POST http://localhost:7670/v1/ingest/job \\\n  -H 'content-type: application/json' \\\n  -d '{\"expected_documents\":1}'\n\nTRACE_ID=$(jq -r .trace_id job.json)\ngrep -i x-trace-id headers.txt\n

Port-forward Zipkin and query the trace directly:

kubectl port-forward svc/tracing-smoke-nemo-retriever-zipkin 9411:9411\ncurl \"http://localhost:9411/api/v2/trace/${TRACE_ID}\"\n

Prometheus metrics from the OpenTelemetry Collector

When topology.otel.enabled=true, the chart-owned OpenTelemetry Collector\nexposes metrics received through OTLP in Prometheus format. The endpoint uses\ntopology.otel.ports.prometheus, which defaults to port 8889. The\nchart-owned OpenTelemetry Collector Service exposes the same port. With default\nvalues, the retriever service and chart-managed NIMs export OTLP metrics to this\nendpoint.

The retriever service also retains its native Prometheus /metrics endpoint.\nEnable serviceMonitor.enabled=true when a Prometheus Operator should scrape\nthat endpoint directly. Direct scraping remains useful for service metrics that\ndo not use OTLP, including the worker-pool metrics used by split-mode\nautoscaling.

After a successful ingestion, allow up to 30 seconds for metric export and\nverify the endpoint by port-forwarding the Collector Service:

-nemo-retriever-otel 8889:8889\nmetric_found=false\nfor attempt in {1..30}; do\n if curl -fsS http://127.0.0.1:8889/metrics | grep -q '^nemo_retriever_'; then\n metric_found=true\n break\n fi\n sleep 1\ndone\ntest "${metric_found}" = true\n\">

kubectl port-forward svc/<release>-nemo-retriever-otel 8889:8889\nmetric_found=false\nfor attempt in {1..30}; do\n  if curl -fsS http://127.0.0.1:8889/metrics | grep -q '^nemo_retriever_'; then\n    metric_found=true\n    break\n  fi\n  sleep 1\ndone\ntest \"${metric_found}\" = true\n

Set topology.otel.ports.prometheus to use a different port. The chart updates\nthe Collector listener and Service port together. If the command does not find\na metric after 30 seconds, confirm that topology.otel.enabled and the\nworkload's OpenTelemetry settings are enabled, then inspect the Collector logs.

Common opt-out and override knobs:

topology:\n  zipkin:\n    enabled: false                 # do not deploy chart-owned Zipkin\n    exporter:\n      enabled: false               # keep Zipkin deployed, but do not export traces to it\n      endpoint: http://external-zipkin:9411/api/v2/spans\n\nservice:\n  otel:\n    enabled: true                  # required for the following service overrides to render\n    env:\n      OTEL_METRICS_EXPORTER: none  # retain tracing, but do not export service metrics\n      OTEL_METRIC_EXPORT_INTERVAL: \"10000\" # service metric cadence in milliseconds\n# To opt out of all service OTLP telemetry instead, set `service.otel.enabled: false`.\n\nnimOperator:\n  otel:\n    enabled: true                  # required for the following inherited NIM override to render\n    env:\n      NIM_OTEL_METRICS_EXPORTER: \"console\" # do not send inherited NIM metrics to OTLP\n# To opt out of all inherited NIM OTLP telemetry instead, set `nimOperator.otel.enabled: false`.\n  page_elements:\n    otel:\n      enabled: false               # per-NIM opt-out\n  ocr:\n    otel:\n      env:\n        NIM_OTEL_METRICS_EXPORTER: \"console\" # per-NIM metric-export opt-out\n        TRITON_OTEL_RATE: \"10\"     # per-NIM Triton OTel override\n

Set topology.zipkin.exporter.endpoint when you run your own Zipkin-compatible\ncollector. Set topology.otel.enabled=false to disable the chart-owned collector\nand all chart-rendered collector wiring. Values in service.env override the\nchart-managed service OpenTelemetry environment variables. Existing NIM\ncontainer environment variables take precedence over inherited\nnimOperator.otel.env values, and per-NIM nimOperator.<key>.otel.env values\noverride the inherited NIM values.

OpenShift deployment { #openshift-deployment }

OpenShift install procedures, restricted-v2 / PSA restricted value overrides, prebuilt ffmpeg images, internal registry pull secrets, optional NIM LD_LIBRARY_PATH tuning, and install examples are in OpenShift deployment. Pass -f openshift-restricted.yaml from that guide when you install on OpenShift.

Air-gapped deployment { #air-gapped-deployment }

Refer to Deployment options — Air-gapped and disconnected deployment for overview and workflow. Chart-specific reference for mirroring:

Container images to mirror (chart defaults)

Verify tags on the Git branch or tag you ship (for example main or\nyour release tag). Defaults below match\nvalues.yaml on the current chart.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

tr:nth-child(odd)]:bg-surface-sunken\">

Role	`nimOperator` key	Default image (`repository:tag`)
Retriever service	—	`service.image.repository`:`service.image.tag` (override for production)
Page Elements	`page_elements`	`nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1`
Table Structure	`table_structure`	`nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1`
OCR	`ocr`	`nvcr.io/nim/nvidia/nemotron-ocr-v2:2.0.1`
VL embed	`vlm_embed`	`nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:2.3.0`
VL reranker (optional)	`rerankqa`	`nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:2.3.0`
Nemotron Parse (optional)	`nemotron_parse`	`nvcr.io/nim/nvidia/nemotron-parse-v1.2:1.7.0-variant`
Omni caption or configurable answer VLM (optional)	`nemotron_3_nano_omni_30b_a3b_reasoning`	`nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant`
Answer LLM (optional, Super-49B default)	`answer_llm`	`nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5:2.0.5`
Parakeet ASR (optional)	`audio`	`nvcr.io/nim/nvidia/parakeet-1-1b-ctc-en-us:1.5.0`

GPU SKU support for audio is in Model hardware requirements.

Also mirror images for the vectordb sidecar, Redis, or other subcharts if\nyour values enable them.

Helm values for a private registry

Example overrides (replace placeholders):

helm upgrade --install retriever ./nemo_retriever/helm \\\n  -f my-airgap-values.yaml\n

my-airgap-values.yaml should include at least:

/nemo-retriever-service\n tag: \n pullPolicy: IfNotPresent\n\nimagePullSecrets:\n - name: my-private-registry\n\nngcImagePullSecret:\n create: false\n name: "" # Explicitly empty — clears the default "ngc-secret"\n\nnimOperator:\n page_elements:\n image:\n repository: /nemotron-object-detection\n tag: "2.0.1"\n pullPolicy: IfNotPresent\n # Repeat for table_structure, ocr, vlm_embed, and any optional keys you enable.\n\">

service:\n  image:\n    repository: <PRIVATE_REGISTRY>/nemo-retriever-service\n    tag: <PINNED_TAG>\n    pullPolicy: IfNotPresent\n\nimagePullSecrets:\n  - name: my-private-registry\n\nngcImagePullSecret:\n  create: false\n  name: \"\"   # Explicitly empty — clears the default \"ngc-secret\"\n\nnimOperator:\n  page_elements:\n    image:\n      repository: <PRIVATE_REGISTRY>/nemotron-object-detection\n      tag: \"2.0.1\"\n      pullPolicy: IfNotPresent\n  # Repeat for table_structure, ocr, vlm_embed, and any optional keys you enable.\n

Set nimOperator.<key>.image.pullSecrets to your mirror pull secret\n(for example my-private-registry) when it differs from\nngcImagePullSecret.name. Empty per-NIM pullSecrets inherit the\nchart-wide name.
Leave serviceConfig.nimEndpoints.* empty when operator-managed NIMs\nare in-cluster; set explicit URLs only for external or mirrored services\noutside the chart.
For offline captioning, enable\nnimOperator.nemotron_3_nano_omni_30b_a3b_reasoning and point the pipeline\ncaption endpoint at the in-cluster NIM URL (refer to\nImage captioning).

Mirroring pattern

/nemotron-object-detection:2.0.1\ndocker push /nemotron-object-detection:2.0.1\n\">

docker login nvcr.io -u '$oauthtoken' -p \"$NGC_API_KEY\"\ndocker pull nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1\ndocker tag nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1 \\\n  <PRIVATE_REGISTRY>/nemotron-object-detection:2.0.1\ndocker push <PRIVATE_REGISTRY>/nemotron-object-detection:2.0.1\n

For bulk sync, prefer skopeo or\ncrane.\nRecord repository@sha256:... digests for regulated environments.

Roadmap

External scheduler backend — introduce shared job, queue, lease, and\nSSE state before allowing more than one gateway replica.
NetworkPolicies restricting the service Pod to the NIM Pods\nonly.
Gateway autoscaling on inflight-uploads (currently fixed\ntopology.gateway.replicas) — shared scheduler and SSE ownership must land first.

Validation

The chart is exercised in CI with helm lint and helm template. Run\nlocally:

/tmp/r-op.yaml\n\n# Operator CRDs absent: vectordb has no operator URL to fall back to, so\n# either disable vectordb or supply an explicit embed endpoint.\nhelm template r nemo_retriever/helm \\\n --set serviceConfig.vectordb.enabled=false > /tmp/r.yaml\n# or:\n# helm template r nemo_retriever/helm \\\n# --set serviceConfig.nimEndpoints.embedInvokeUrl=http://embed.svc:8000/v1/embeddings \\\n# > /tmp/r.yaml\n\">

helm lint nemo_retriever/helm\n\n# Operator CRDs present: vectordb resolves vlm_embed via the operator URL.\nhelm template r nemo_retriever/helm \\\n  --api-versions apps.nvidia.com/v1alpha1 > /tmp/r-op.yaml\n\n# Operator CRDs absent: vectordb has no operator URL to fall back to, so\n# either disable vectordb or supply an explicit embed endpoint.\nhelm template r nemo_retriever/helm \\\n  --set serviceConfig.vectordb.enabled=false > /tmp/r.yaml\n#   or:\n# helm template r nemo_retriever/helm \\\n#   --set serviceConfig.nimEndpoints.embedInvokeUrl=http://embed.svc:8000/v1/embeddings \\\n#   > /tmp/r.yaml\n

Both renders should succeed cleanly and parse as valid Kubernetes manifests\n(kubectl apply --dry-run=client -f /tmp/r.yaml). Refer to VectorDB and the\nembed endpoint for why\nhelm template r nemo_retriever/helm without flags is rejected as a\nmisconfiguration.

","resourceVersions",[196,197,198],"resourceVersionsTotal","hasMoreVersions","versionMetadata",{"_178":29,"_179":-5,"_53":83,"_80":-5,"_84":180,"_86":181,"_182":82,"_93":18,"_90":35,"_183":184,"_88":89,"_185":-5,"_186":35,"_187":-5,"_188":35,"_189":-5,"_190":35,"_191":-5,"_192":35,"_193":-5,"_194":35,"_91":195},"versionNotFound","isLoggedIn","latestVersionPath","/orgs/nvidia/nemo-microservices/helm-charts/nemo-retriever/-","version","imageReference","cliDownloadCommand","helmFetchCommand","helm pull https://helm.ngc.nvidia.com/nvidia/nemo-microservices/charts/nemo-retriever-26.5.0.tgz","viewAllVersionsPath","/orgs/nvidia/nemo-microservices/helm-charts/nemo-retriever/26.5.0/version-history","accessCheckResult",{"_177":89},"hasRelatedCollections","context",{"_173":174,"_175":176},[],[],[167,168,169,170,171,172],{"_49":63,"_51":74,"_53":65,"_55":66,"_57":75,"_59":60},{"_49":63,"_51":70,"_53":65,"_55":66,"_57":71,"_59":60},{"_49":63,"_51":68,"_53":65,"_55":66,"_57":69,"_59":60},{"_49":63,"_51":64,"_53":65,"_55":66,"_57":67,"_59":60},{"_49":61,"_51":62,"_53":116,"_55":-5,"_57":-5,"_59":60},{"_49":50,"_51":52,"_53":54,"_55":56,"_57":58,"_59":60},"orgName","nvidia","teamName","nemo-microservices","allowed","versionId","description",["D",1782913538916],["D",1782913539626],"totalSizeInBytes","malwareScanStatus","CLEAN","releaseNotes","hasReleaseNotes","setup","hasSetup","quickStartGuide","hasQuickStartGuide","advanced","hasAdvanced","performance","hasPerformance",[],{"_78":201,"_79":202,"_80":-5,"_90":35,"_88":89,"_91":200,"_186":35},{"_78":95,"_79":95,"_80":-5,"_90":35,"_88":89,"_91":200,"_186":35},{"_78":29,"_79":29,"_80":-5,"_90":35,"_88":89,"_91":199,"_186":35},[],[],"-","Latest (26.08.1)","nvidia/nemo-microservices/nemo-retriever","key","nvidia/nemo-microservices/helm-charts/nemo-retriever","resourceId","nemo-retriever","shortDescription","Helm chart for the NeMo Retriever ingest service and optional NIM microservices","# nemo-retriever Helm chart\n\nA Kubernetes Helm chart for running the **service** mode of\n[`nemo-retriever`](../README.md): a FastAPI document ingestion server that\nstreams uploads through a set of NVIDIA NIM microservices\n(object detection, OCR, VLM embed by default) and exposes\nresult + status APIs over HTTP / SSE.\n\nUse **Helm** (this chart and/or the **additional Library charts** documented in the\n[NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/))\nfor supported NIM and service deployment.\n\nThe chart ships two deployable layers behind feature flags:\n\n- **the service** — always on; one Deployment (standalone) or three\n Deployments (split topology: gateway / realtime / batch), built from\n `Dockerfile --target service`.\n- **the NIMs** — optional, GPU-backed `NIMCache` + `NIMService` custom\n resources (`apiVersion: apps.nvidia.com/v1alpha1`) reconciled by the\n **NVIDIA NIM Operator**. The chart auto-wires the operator-managed\n Service URLs into the retriever-service config when the operator CRDs\n are present in the cluster.\n\n> **NIM Operator prerequisite.** The NIM templates are gated on the\n> `apps.nvidia.com/v1alpha1` API group. Install the NIM Operator before\n> running `helm install`:\n> https://docs.nvidia.com/nim-operator/\n>\n> Without the operator the chart still installs cleanly — every NIMCache /\n> NIMService template short-circuits and the service falls back to\n> external NIM URLs supplied via `serviceConfig.nimEndpoints.*`.\n\n> **Gateway scheduler state is explicitly ephemeral.** Split mode enforces one\n> gateway replica and uses a `Recreate` rollout so independent in-memory\n> schedulers never overlap. A gateway restart, rollout, eviction, or node failure\n> loses every accepted job, queued item, active lease, status record, and SSE\n> catch-up event owned by that process. Drain accepted work before any rollout.\n\n> For behavioral consistency between local HuggingFace deployments and Helm service deployments: \n> `results = ingestor.ingest(...return_results=True)\n> return_results defaults to True. This incurs a significant performance and system memory usage cost. \n> Unless you know explicitly you need to fetch extraction results to the client, you should use:\n> return_results=False\n> If you must return results, you may need to increase pod memory specs to support the increased pod memory usage.\n\n---\n\n## Layout\n\n```\nnemo_retriever/helm/\n├── Chart.yaml\n├── values.yaml\n├── README.md <-- this file\n├── openshift.md <-- OpenShift restricted-v2 install guide\n├── .helmignore\n└── templates/\n ├── _helpers.tpl\n ├── NOTES.txt\n ├── configmap.yaml # renders retriever-service.yaml\n ├── deployment.yaml # the service Deployment(s)\n ├── service.yaml # ClusterIP/NodePort for the service\n ├── ingress.yaml # optional Ingress\n ├── hpa.yaml # optional HorizontalPodAutoscaler\n ├── servicemonitor.yaml # optional Prometheus ServiceMonitor\n ├── serviceaccount.yaml\n ├── pvc.yaml # general persistence PVC\n ├── secrets.yaml # ngc-secret + ngc-api\n └── nims/\n ├── nemotron-page-elements-v3.yaml # NIMCache + NIMService\n ├── nemotron-table-structure-v1.yaml # NIMCache + NIMService\n ├── nemotron-ocr-v2.yaml # NIMCache + NIMService\n ├── llama-nemotron-embed-vl-1b-v2.yaml # NIMCache + NIMService (VLM embed)\n ├── llama-nemotron-rerank-vl-1b-v2.yaml # NIMCache + NIMService (optional; auto-wired when enabled)\n ├── nemotron-parse.yaml # NIMCache + NIMService (optional; not auto-wired)\n ├── nemotron-3-nano-omni-30b-a3b-reasoning.yaml # NIMCache + NIMService (optional; auto-wired when enabled)\n └── audio.yaml # NIMCache + NIMService (optional; not auto-wired)\n```\n\n---\n\n## Quick start\n\n### Persistent storage prerequisite { #persistent-storage-prerequisite }\n\nThe default chart creates **seven** PersistentVolumeClaims: three\nchart-managed service claims and four NIM Operator NIMCache claims for\nthe core NIMs. `helm install` reports `STATUS: deployed` when the\nrelease is rendered, even if every claim stays `Pending`. Confirm a\nworking persistent-volume binding strategy before you install.\n\nRun the following preflight commands:\n\n```bash\nkubectl get storageclass\nkubectl get pv\n```\n\nUse one of the following strategies:\n\n- A default StorageClass (`(default)` in `kubectl get storageclass`)\n backed by a working provisioner.\n- Explicit `storageClass` values for every default claim, each backed\n by a working provisioner or matching persistent volumes.\n- Compatible static persistent volumes, or pre-created claims where\n the chart supports `existingClaim`.\n\nWhen a `storageClass` value is empty, the chart omits\n`storageClassName`. Kubernetes then assigns the default StorageClass\nif one exists. If none exists, the claim binds only to a compatible\nclassless persistent volume that matches the requested size and\n`ReadWriteOnce` access mode.\n\nThe following table lists the default claims for a release named\n`retriever`:\n\n| Example claim name | Default size | Helm value path |\n| --- | --- | --- |\n| `retriever-nemo-retriever-data` | `50Gi` | `persistence.storageClass` |\n| `retriever-nemo-retriever-retriever-results` | `50Gi` | `retrieverResults.storageClass` |\n| `retriever-nemo-retriever-vectordb-data` | `50Gi` | `topology.vectordb.persistence.storageClass` |\n| `nemotron-page-elements-v3-pvc` | `25Gi` | `nimOperator.page_elements.storage.pvc.storageClass` |\n| `nemotron-table-structure-v1-pvc` | `25Gi` | `nimOperator.table_structure.storage.pvc.storageClass` |\n| `nemotron-ocr-v2-pvc` | `25Gi` | `nimOperator.ocr.storage.pvc.storageClass` |\n| `llama-nemotron-embed-vl-1b-v2-pvc` | `50Gi` | `nimOperator.vlm_embed.storage.pvc.storageClass` |\n\nWhen `nims.enabled=false`, the four NIMCache claims are not created.\nThe three chart-managed claims still are, unless you disable those\npersistence blocks. Enabling an optional NIM adds another NIMCache\nclaim for that key.\n\nTo pin a named StorageClass on every default claim, pass the following\n`--set` flags with your cluster class name:\n\n```bash\nhelm install retriever ./nemo_retriever/helm \\\n --set persistence.storageClass= \\\n --set retrieverResults.storageClass= \\\n --set topology.vectordb.persistence.storageClass= \\\n --set nimOperator.page_elements.storage.pvc.storageClass= \\\n --set nimOperator.table_structure.storage.pvc.storageClass= \\\n --set nimOperator.ocr.storage.pvc.storageClass= \\\n --set nimOperator.vlm_embed.storage.pvc.storageClass= \\\n --set ngcImagePullSecret.create=true \\\n --set ngcImagePullSecret.password=$NGC_API_KEY \\\n --set ngcApiSecret.create=true \\\n --set ngcApiSecret.password=$NGC_API_KEY\n```\n\nSet each per-NIM `nimOperator..storage.pvc.storageClass` path.\nThe chart-level `nimOperator.nimCache.pvc.storageClass` value is not\napplied to the four core NIMCache resources.\n\n`persistence.existingClaim` and `retrieverResults.existingClaim` skip\nchart PVC creation and mount the named claim. The VectorDB claim does\nnot have an `existingClaim` path.\n\nIf `helm install` already succeeded and claims stay `Pending`, refer\nto [Helm install succeeds but PersistentVolumeClaims stay Pending](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/troubleshoot.md#helm-pending-pvcs).\n\n### GPU scheduling prerequisite { #gpu-scheduling-prerequisite }\n\nThe [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements)\ntable lists **Total GPUs: 1** for Core Features because the four\ndefault NIMs together use about 4.8 GiB of GPU memory and can\nco-reside on one A10G or better GPU. That figure is VRAM capacity.\nIt is not the number of exclusive Kubernetes GPU requests this chart\nmakes.\n\nA default install creates four `NIMService` resources\n(`page_elements`, `table_structure`, `ocr`, `vlm_embed`). Each\nrenders `spec.resources.limits.nvidia.com/gpu: 1` from\n`nimOperator.nimServiceGpuLimit` (default `1`). On a conventional\ncluster without MIG, time-slicing, or another sharing mechanism, the\nscheduler consumes **four allocatable GPU slots across eligible\nnodes**. Four one-GPU nodes satisfy that topology. A cluster that\nhas only one allocatable GPU schedules only one NIM pod. The other\nthree stay `Pending`.\n\nChoose one of the following before you install:\n\n- Provide four allocatable `nvidia.com/gpu` slots across eligible\n nodes for the default core topology. Each optional NIM adds its\n per-NIM GPU request. Most optional keys use `nimServiceGpuLimit`\n (one GPU). The default `answer_llm` Super-49B resources request\n two physical GPUs.\n- Configure GPU sharing so the cluster advertises at least four\n `nvidia.com/gpu` slots. Time-slicing is the documented sharing\n path. It works on GPUs that do not support Multi-Instance GPU\n (MIG), including A10G.\n\nThis chart does not pack the four NIMs onto a single `nvidia.com/gpu`\nrequest. Sharing is cluster configuration through the\n[NVIDIA GPU Operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html).\nTime-slicing does not isolate GPU memory. Combined VRAM of the\nscheduled NIMs must still fit on the physical GPU.\n\nConfirm allocatable GPU slots across the cluster:\n\n```bash\nkubectl get nodes -o custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\\.com/gpu\n```\n\nSum `GPU` across eligible nodes. A default core install needs four\nslots across the cluster. Four nodes that each report `1` are\nenough. A single node needs a value of `4` or greater only when you\npack all four core NIMs onto one physical GPU with sharing and\nplacement constraints.\n\nThe following GPU Operator time-slicing ConfigMap advertises four\nreplicas per physical GPU. On a node with one physical GPU, that\ncreates four logical slots, which is enough for the four default\nNIMServices. A node with two physical GPUs advertises eight slots.\nThe NVIDIA GPU Operator must already be installed. Apply the\nConfigMap in the GPU Operator namespace (commonly `gpu-operator`)\nbefore `helm install`:\n\n```yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: time-slicing-config-all\ndata:\n any: |-\n version: v1\n flags:\n migStrategy: none\n sharing:\n timeSlicing:\n resources:\n - name: nvidia.com/gpu\n replicas: 4\n```\n\n```bash\nkubectl create -n gpu-operator -f time-slicing-config-all.yaml\nkubectl patch clusterpolicies.nvidia.com/cluster-policy \\\n -n gpu-operator --type merge \\\n -p '{\"spec\": {\"devicePlugin\": {\"config\": {\"name\": \"time-slicing-config-all\", \"default\": \"any\"}}}}'\n```\n\nThat ClusterPolicy patch is a cluster-administrator change.\n`default: \"any\"` applies the four-replica configuration to all\neligible GPU Operator nodes. It can oversubscribe unrelated GPU\nworkloads that share those nodes.\n\nTime-slicing oversubscribes allocatable slots. It does not place\nthe four NIM pods on one physical GPU. The scheduler can spread\nthem across GPUs or nodes. The one-physical-GPU recipe requires\nboth of the following: the target node has one physical GPU and\nadvertises at least four replicas, and all four NIMServices are\npinned to that node. Confirm that node's `Allocatable`\n`nvidia.com/gpu` is `4` or greater, and pin the four core\nNIMServices as shown below.\n\nThe default `answer_llm` Super-49B NIMService is outside that\none-physical-GPU recipe. It requests two GPUs (`nvidia.com/gpu: 2`\nand `NIM_TENSOR_PARALLEL_SIZE=2`). A time-sliced request for more\nthan one GPU does not provide two physical GPUs or proportional\ncompute, so extra time-slice replicas cannot satisfy that\ntensor-parallel requirement. Keep Super-49B on two physical GPUs\nunless you override the slot with a separately validated model and\nprofile. Refer to [Answer generation](#answer-generation-llm).\n\nOn a multi-GPU or multi-node cluster, pin the four core\nNIMServices to a single-GPU node. Set\n`nimOperator..nodeSelector` on `page_elements`,\n`table_structure`, `ocr`, and `vlm_embed`. `nodeSelector`\nconstrains node placement. On a multi-GPU node, it does not\nensure all four pods receive logical replicas from the same\nphysical GPU. This chart does not render pod affinity or\ntopology spread constraints:\n\n```yaml\nnimOperator:\n page_elements:\n nodeSelector:\n kubernetes.io/hostname: \n table_structure:\n nodeSelector:\n kubernetes.io/hostname: \n ocr:\n nodeSelector:\n kubernetes.io/hostname: \n vlm_embed:\n nodeSelector:\n kubernetes.io/hostname: \n```\n\nOn a mixed cluster, omit `devicePlugin.config.default` and label\nonly the target node so other GPU nodes keep exclusive access:\n\n```bash\nkubectl patch clusterpolicies.nvidia.com/cluster-policy \\\n -n gpu-operator --type merge \\\n -p '{\"spec\": {\"devicePlugin\": {\"config\": {\"name\": \"time-slicing-config-all\"}}}}'\nkubectl label node nvidia.com/device-plugin.config=any\n```\n\nIncrease `replicas` if you also enable optional NIMs that request\none GPU each on the same shared GPU and combined VRAM still fits.\nCount each of those Helm GPU requests. Do not increase time-slice\nreplicas to satisfy tensor-parallel GPU counts. For the full\nprocedure, node labels, and limitations, refer to\n[Time-Slicing GPUs in Kubernetes](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html).\nMIG is an advanced GPU Operator configuration outside this chart.\nThe chart does not set a MIG strategy, MIG profile, or MIG\nresource requests. Default NIMServices request `nvidia.com/gpu`.\nA working MIG deployment is GPU-specific and profile-specific and\ncan require per-NIM `resources` overrides. For GPU Operator MIG,\nrefer to\n[GPU Operator with MIG](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-operator-mig.html).\nThe NIM Operator also documents DRA-based sharing. This chart does\nnot render `draResources`; refer to\n[NIM Operator DRA](https://docs.nvidia.com/nim-operator/latest/dra.html)\nif you manage ResourceClaims outside the chart.\n\nIf `helm install` already succeeded and NIM pods stay `Pending` on\n`nvidia.com/gpu`, refer to\n[Core NIM pods stay Pending for GPU](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/troubleshoot.md#helm-pending-gpus).\n\n### 1. Service image { #1-service-image }\n\nThe chart defaults to the image published to NGC:\n\n```\nnvcr.io/nvidia/nemo-microservices/nrl-service:26.5.0\n```\n\nPulling from `nvcr.io` requires an NGC pull secret — either set\n`ngcImagePullSecret.create=true` (see below) or pre-create one in the\nnamespace named `ngc-secret`.\n\nTo run a locally built image instead, build and push it from the repo root,\nthen override `service.image.repository` / `service.image.tag`:\n\n```bash\n# from the repo root:\ndocker build \\\n --target service \\\n --build-arg DOWNLOAD_DEFAULT_TOKENIZER=True \\\n --build-arg RETRIEVER_VERSION= \\\n --build-arg RETRIEVER_RELEASE_TYPE=release \\\n -t /nemo-retriever-service: .\ndocker push /nemo-retriever-service:\n```\n\nAudio and video extraction require the `ffmpeg` and `ffprobe` system\nbinaries inside the service container. The bundled service image can install\nthem at container startup when you set `service.installFfmpeg=true`, which\nsets `INSTALL_FFMPEG=true` for the image entrypoint:\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set service.image.repository=/nemo-retriever-service \\\n --set service.image.tag= \\\n --set service.installFfmpeg=true\n```\n\nDo not also set `INSTALL_FFMPEG` in `service.env`; the chart fails rendering\nwhen both are configured so the rendered Pod does not contain duplicate\nenvironment variables.\n\nWhen `service.installFfmpeg=false` (the default), the service still starts\nnormally and processes PDF, image, text and HTML uploads. Audio / video\nuploads are rejected up-front with **HTTP 501**:\n\n```text\nAudio and video ingestion require FFmpeg in the retriever service\ncontainer, but the following dependencies are missing: ffmpeg, ffprobe.\nRe-deploy the Helm chart with `--set service.installFfmpeg=true` …\n```\n\nThe retriever-service container also logs a `WARNING` at startup when\nFFmpeg is missing so cluster operators can fix the deployment before\nthe first media upload arrives, instead of debugging a Ray worker\ntraceback (`RuntimeError: MediaChunkActor requires media dependencies;\nmissing: ffmpeg, ffprobe`) after the fact. The same WARNING is emitted\non every pod (gateway, realtime, batch) because all roles classify\nuploads — flipping `service.installFfmpeg=true` updates them all.\n\nRuntime installation uses passwordless `sudo` scoped to installing the\n`ffmpeg` package in the service image. The pod must have network egress to the\nUbuntu package repositories, a writable root filesystem, and a security policy\nthat allows sudo/setuid behavior. Do not set\n`service.securityContext.allowPrivilegeEscalation: false` or\n`service.securityContext.readOnlyRootFilesystem: true` for this path.\n\nFor air-gapped or locked-down clusters, see\n[Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment).\nOn a connected staging host you can extend the service image, for example:\n\n```dockerfile\nFROM /nemo-retriever-service:\nUSER root\nRUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \\\n && rm -rf /var/lib/apt/lists/*\nUSER nemo\n```\n\n### 2. Install with external NIM endpoints (operator not required)\n\nComplete the [persistent storage prerequisite](#persistent-storage-prerequisite)\nbefore you install. With `nims.enabled=false`, the chart still creates\nthe three service claims unless you disable those persistence blocks.\n\nIf you already have NIM endpoints reachable from the cluster (e.g. another\nnamespace, or NVIDIA Build), turn the master switch off and supply the\nURLs directly:\n\n```bash\nhelm install retriever ./nemo_retriever/helm \\\n --set nims.enabled=false \\\n --set ngcImagePullSecret.create=true \\\n --set ngcImagePullSecret.password=$NGC_API_KEY \\\n --set ngcApiSecret.create=true \\\n --set ngcApiSecret.password=$NGC_API_KEY \\\n --set serviceConfig.nimEndpoints.pageElementsInvokeUrl=http://page-elements.svc:8000/v1/page-elements \\\n --set serviceConfig.nimEndpoints.tableStructureInvokeUrl=http://table-structure.svc:8000/v1/table-structure \\\n --set serviceConfig.nimEndpoints.ocrInvokeUrl=http://ocr.svc:8000/v1/ocr \\\n --set serviceConfig.nimEndpoints.embedInvokeUrl=http://embed.svc:8000/v1/embeddings\n\n```\n`ngcApiSecret` materialises an `ngc-api` Secret containing both\n`NGC_API_KEY` and `NGC_CLI_API_KEY` keys; the service container reads it\nvia `optional: true` `secretKeyRef`, so the install still succeeds when\nthe secret is absent (useful for fully local NIM endpoints).\n\n### 3. Install with the NIM Operator (in-cluster NIMs)\n\nComplete the [persistent storage prerequisite](#persistent-storage-prerequisite)\nand the [GPU scheduling prerequisite](#gpu-scheduling-prerequisite)\nbefore you install. The default path creates all seven claims and four\nexclusive `nvidia.com/gpu: 1` NIMService requests.\n\nInstall the [NIM Operator](https://docs.nvidia.com/nim-operator/) first so\nthe `NIMCache` / `NIMService` CRDs (`apps.nvidia.com/v1alpha1`) are\nregistered. A plain `helm install` reconciles the four core NIMs\n(`page_elements`, `table_structure`, `ocr`, `vlm_embed`) — every other\nNIM (the VL reranker `rerankqa`, Nemotron Parse, Omni 30B, and the\nParakeet `audio` ASR NIM) is **disabled by default** to honor the\n\"optional and disabled by default\" contract in\n[deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md);\nrefer to [Recommended minimal install](#recommended-minimal-install-2608)\nfor the opt-in `--set` flags that turn any of them on.\n\n```bash\nhelm install retriever ./nemo_retriever/helm \\\n --set ngcImagePullSecret.create=true \\\n --set ngcImagePullSecret.password=$NGC_API_KEY \\\n --set ngcApiSecret.create=true \\\n --set ngcApiSecret.password=$NGC_API_KEY\n```\n\n### Recommended minimal install (26.08.1) { #recommended-minimal-install-26081 }\n\nComplete the [persistent storage prerequisite](#persistent-storage-prerequisite)\nand the [GPU scheduling prerequisite](#gpu-scheduling-prerequisite)\nbefore you install.\n\nDeploy only the four core NIMs that the retriever service auto-wires (`page_elements`, `table_structure`, `ocr`, `vlm_embed`):\n\n```bash\nhelm install retriever ./nemo_retriever/helm \\\n --set ngcImagePullSecret.create=true \\\n --set ngcImagePullSecret.password=$NGC_API_KEY \\\n --set ngcApiSecret.create=true \\\n --set ngcApiSecret.password=$NGC_API_KEY \\\n --set service.image.tag=26.8.1\n```\n\n> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, the generic answer-generation LLM (`answer_llm`, Super-49B defaults), and the Parakeet `audio` ASR NIM are **all off by default** — they only reconcile when you explicitly opt in. Opt-in flags:\n>\n> * VL reranker — `--set nimOperator.rerankqa.enabled=true` (auto-wires `nim_endpoints.rerank_invoke_url` / `rerank_model_name` — refer to [Query-time reranking](#query-time-reranking))\n> * Nemotron Parse — `--set nimOperator.nemotron_parse.enabled=true`\n> * Omni 30B captioner — `--set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true`\n> * Answer generation LLM — `--set nimOperator.answer_llm.enabled=true`\n> * Parakeet ASR — `--set nimOperator.audio.enabled=true` (also set `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` to wire ASR into the service, plus `service.installFfmpeg=true` if your image does not bundle ffmpeg)\n>\n> This matches the \"optional and disabled by default\" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights, loading a large two-GPU LLM, or claiming extra dedicated GPUs on a \"default\" install. Refer to the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs.\n\nThe chart auto-wires the operator-managed in-cluster URLs of the three\n\"core\" NIMs into the service's `nim_endpoints` block:\n\n| key | operator-managed Service | invoke path |\n| --- | ------------------------ | ----------- |\n| `nimOperator.page_elements` | `nemotron-page-elements-v3` | `/v1/page-elements` |\n| `nimOperator.table_structure` | `nemotron-table-structure-v1` | `/v1/table-structure` |\n| `nimOperator.ocr` | `nemotron-ocr-v2` | `/v1/ocr` |\n| `nimOperator.vlm_embed` | `llama-nemotron-embed-vl-1b-v2` | `/v1/embeddings` |\n\n### Query reranking (optional)\n\nThe optional `nimOperator.rerankqa` NIM is not auto-wired into the retriever service. To use `POST /v1/query` with `rerank=true`, enable the NIM and configure the service endpoint explicitly:\n\n```yaml\nnimOperator:\n rerankqa:\n enabled: true\n\nserviceConfig:\n nimEndpoints:\n rerankInvokeUrl: http://llama-nemotron-rerank-vl-1b-v2:8000/v1/ranking\n rerankModelName: nvidia/llama-nemotron-rerank-vl-1b-v2\n```\n\nEnabling `nimOperator.rerankqa.enabled=true` without `serviceConfig.nimEndpoints.rerankInvokeUrl` deploys the NIM but does not enable service query reranking.\n\nTrack operator reconciliation with:\n\n```bash\nkubectl get nimcache,nimservice -n \nkubectl describe nimservice nemotron-object-detection -n \n```\n\nFirst-time NIMCache reconciliation downloads model weights to a PVC. By\ndefault (`nimOperator.nimCache.keepOnUninstall: true`) every **NIMCache**\ncarries `helm.sh/resource-policy: keep` so those downloads survive\n`helm uninstall`. **NIMService** CRs do not use `keep` and are removed by\nHelm on uninstall.\n\n### Why NIM resources still exist after `helm uninstall`\n\n| What you see | Typical cause |\n|--------------|----------------|\n| `NIMCache` + PVC remain | **Expected** when `keepOnUninstall` is true (default). Helm intentionally skips deleting caches so you do not re-pull multi‑GiB weights. |\n| `NIMService` CR remains | **Not expected** on a normal uninstall. Usually an **orphan** from a failed install/upgrade (release never recorded the resource, or the chart renamed a NIM). |\n| Deployments / GPU pods still running | Often the operator workload for a **kept** `NIMCache`, or a stale `NIMService` that Helm did not own. Check `kubectl get nimservice,nimcache -n `. |\n| `nemotron-*-job-*` pods in `Error` | The NIM Operator's **model-download Job** for a `NIMCache` (not the retriever service). Failed cache pulls retry and leave Error pods until the Job or `NIMCache` is deleted. Common after a failed `helm install` when the release is rolled back but `keep` retains the cache CR. |\n| `helm uninstall` appears to do nothing | Release may be missing or failed (`helm list -n -a`). CRs created before a failed install can be left without a release to clean them up. |\n\nTo change a NIM image on a later install or upgrade, delete the kept\n`NIMCache` first. Refer to\n[Changing a NIM image repository or tag](#changing-nim-image-repository-or-tag).\n\n**Full teardown** (dev cluster — deletes caches and PVCs Helm kept):\n\n```bash\nNS=retriever\nREL=nemo-retriever\n\nhelm uninstall \"${REL}\" -n \"${NS}\" 2>/dev/null || true\n\n# Orphans and kept NIMCaches (Helm keep does not block kubectl delete):\nkubectl delete nimservice,nimcache -n \"${NS}\" --all\n# Optional: drop model PVCs if you will re-pull from NGC\nkubectl delete pvc -n \"${NS}\" -l 'app.kubernetes.io/managed-by=nvidia-nim-operator' 2>/dev/null || true\n```\n\n**Dev installs** that should not retain caches on uninstall:\n\n```bash\nhelm upgrade --install \"${REL}\" ./nemo_retriever/helm -n \"${NS}\" \\\n --set nimOperator.nimCache.keepOnUninstall=false \\\n ...\n```\n\n---\n\n## Values reference (highlights)\n\nThe full schema lives in [`values.yaml`](./values.yaml). Below is the\nshort list of knobs you'll touch first.\n\n### Service\n\n| Path | Default | Notes |\n|-------------------------------|------------------------------------|-------|\n| `service.image.repository` | `nvcr.io/nvidia/nemo-microservices/nrl-service` | NGC image; override to pin a different build or use a local registry. |\n| `service.image.tag` | `26.5.0` | Also injected as `RETRIEVER_SERVICE_VERSION` so `/openapi.json` `info.version` matches the running image tag. |\n\n| `service.replicas` | `1` | Keep at 1 because standalone job and scheduler state are process-local. |\n| `service.installFfmpeg` | `false` | Install `ffmpeg`/`ffprobe` at container startup by setting `INSTALL_FFMPEG=true`. Requires network egress, writable root filesystem, and sudo/setuid allowed. Not for air-gapped clusters — use a custom image instead. |\n| `service.resources.requests` | `16 / 16Gi` | Tune in tandem with `serviceConfig.pipeline.*Workers`. |\n| `service.resources.limits` | `96 / 96Gi` | |\n| `service.gpu.enabled` | `false` | The service does **not** need a GPU. |\n\nFor audio and video extraction, set `service.installFfmpeg=true` when your\ncluster allows runtime package installation. **OpenShift restricted-v2** blocks\nthat path — use a prebuilt service image instead; refer to [Audio and video on restricted OpenShift](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift).\nFor air-gapped clusters, refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment).\n\n### Audio and video (Parakeet ASR) { #audio-video-parakeet }\n\nParakeet ASR is disabled by default. The chart default is `nimOperator.audio.enabled=false`. The chart does not auto-wire the in-cluster ASR endpoint when you enable the audio NIM.\n\nTo run self-hosted Parakeet for [audio and video extraction](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md), set both of the following values:\n\n```yaml\nnimOperator:\n audio:\n enabled: true\n\nserviceConfig:\n nimEndpoints:\n audioGrpcEndpoint: audio:50051\n```\n\nEquivalent Helm flags are `--set nimOperator.audio.enabled=true` and `--set serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051`.\n\nEnabling only `nimOperator.audio.enabled=true` renders the Parakeet `NIMCache` and `NIMService`. The ConfigMap still sets `audio_grpc_endpoint` to `null`. The retriever service cannot send ASR traffic until you also set `serviceConfig.nimEndpoints.audioGrpcEndpoint`. Disable other optional NIMs you do not need. Refer to [Recommended minimal install](#recommended-minimal-install-2608).\n\nAfter you set those values, complete the following steps:\n\n1. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)).\n2. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations).\n3. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead.\n\nThe in-cluster gRPC Service name is `audio` on port `50051`. Graph ingest does not read this Helm value. Pass the same endpoint through `ASRParams.audio_endpoints` in Python. Refer to [NIM Operator sub-stack](#nim-operator-sub-stack).\n\n### Health probes\n\nThe service exposes unauthenticated health endpoints for Kubernetes probes:\n\n| Endpoint | Purpose | Default Helm use |\n| --- | --- | --- |\n| `GET /v1/live` | Shallow process liveness. The endpoint does not check worker backends or wait for service readiness. | Startup and liveness probes. In split mode, the realtime and batch `wait-for-gateway` init containers poll this endpoint on the internal gateway startup Service while waiting for the gateway process to start. |\n| `GET /v1/health` | Deep readiness. In split gateway mode, the endpoint checks the realtime and batch workers. It returns HTTP `503` when either required worker is unreachable or returns a non-2xx health response. | Readiness probe on the externally exposed gateway Service. |\n\nIn split topology, the gateway readiness probe uses `/v1/health`, which returns\nHTTP `503` until realtime and batch workers are healthy. Worker Pods cannot start\nuntil their `wait-for-gateway` init container reaches the gateway's shallow\n`/v1/live` endpoint. The externally exposed gateway Service does not publish\nendpoints for an unready Pod, which would deadlock a clean install. The chart\ntherefore renders an additional cluster-internal Service named\n`-nemo-retriever-gateway-startup` with\n`publishNotReadyAddresses: true`. Init containers reach\n`http://-nemo-retriever-gateway-startup:/v1/live`\nthrough that Service so workers can start while the gateway Pod is still\nunready. Client traffic continues to use the readiness-gated gateway Service, so\n`/v1/health` still removes an unhealthy gateway from Service endpoints after\nstartup.\n\nWhen a gateway returns HTTP `503` from `/v1/health`, Kubernetes removes it from\nthe readiness-gated gateway Service endpoints until its required workers are\nready. The response includes backend health details to help diagnose the\nunavailable dependency.\n\n### Service networking\n\n| Path | Default | Notes |\n|------|---------|-------|\n| `networkService.port` | `7670` | Kubernetes Service port. The chart uses this port for generated Service DNS URLs. |\n| `serviceConfig.server.port` | `7670` | Retriever service container listener port. |\n\nYou can set these values independently. When they differ, Kubernetes Services\nlisten on `networkService.port` and route to the container listener on\n`serviceConfig.server.port`.\n\nIn standalone mode, the chart renders one Service named\n`-nemo-retriever`.\n\nIn split topology (`topology.mode: split`), the chart renders four Services:\n\n| Service | Example name (release `retriever`) | Type | Client entrypoint |\n| --- | --- | --- | --- |\n| Gateway | `retriever-nemo-retriever-gateway` | `networkService.type` (default `NodePort`) | Yes |\n| Gateway startup | `retriever-nemo-retriever-gateway-startup` | ClusterIP (cluster-internal only) | No |\n| Realtime | `retriever-nemo-retriever-realtime` | ClusterIP | No |\n| Batch | `retriever-nemo-retriever-batch` | ClusterIP | No |\n\nThe gateway startup Service publishes the gateway Pod address while it is still\nunready so realtime and batch init containers can reach `/v1/live`. It is not a\nclient entrypoint. Refer to [Health probes](#health-probes).\n\n### Service configuration (rendered into `retriever-service.yaml`)\n\n| Path | Default | Notes |\n|---------------------------------------------------|---------|-------|\n| `serviceConfig.server.port` | `7670` | Retriever service container listener port. Refer to [Service networking](#service-networking). |\n| `serviceConfig.pipeline.realtimeWorkers` | `24` | Per-pod realtime worker count. |\n| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. Refer to [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. |\n| `serviceConfig.resources.maxUploadBytes` | `500000000` | Maximum upload file size in bytes; requests exceeding the limit are rejected before buffering. |\n| `serviceConfig.sidecarStore.maxPayloadBytes` | `33554432` | Maximum sidecar metadata upload size in bytes. The service rejects a larger upload with HTTP `413` before buffering the complete payload. This value cannot exceed `serviceConfig.resources.maxUploadBytes`. |\n| `serviceConfig.nimEndpoints.*InvokeUrl` | `\"\"` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, `rerankInvokeUrl` (refer to [Query-time reranking](#query-time-reranking)), and `captionInvokeUrl` (refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b)). |\n| `serviceConfig.nimEndpoints.rerankModelName` | `\"\"` | Model id sent to the remote reranker. Auto-set to `nvidia/llama-nemotron-rerank-vl-1b-v2` whenever a rerank URL is resolved. |\n| `serviceConfig.nimEndpoints.captionModelName` | `\"\"` | Model id sent to the remote VLM. Auto-set to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` whenever a caption URL is resolved. |\n| `serviceConfig.nimEndpoints.rerankInvokeUrl` | `\"\"` | Ranking API URL used by `POST /v1/query` when `rerank=true`. Auto-wired from the optional `rerankqa` NIM when enabled; override to point at a hosted or external ranking endpoint. |\n| `serviceConfig.nimEndpoints.rerankModelName` | `\"\"` | Model ID sent to the ranking API. Auto-set to `nvidia/llama-nemotron-rerank-vl-1b-v2` whenever a rerank URL is resolved; override for a different compatible reranker. |\n| `serviceConfig.nimEndpoints.audioGrpcEndpoint` | `\"\"` | gRPC endpoint for Parakeet ASR. Not auto-wired from `nimOperator.audio`. Set `audio:50051` when you enable the audio NIM. |\n| `serviceConfig.llm.enabled` | `false` | Enables `POST /v1/answer`. Auto-flips to true when `nimOperator.answer_llm` is enabled and the operator URL resolves. |\n| `serviceConfig.llm.apiBase` | `\"\"` | OpenAI-compatible LLM base URL. Explicit value wins; otherwise `answer_llm` opt-in resolves to `http://answer-llm:8000/v1` by default. |\n| `serviceConfig.llm.apiKeySecret.name` | `\"\"` | Optional Secret name for external LLM credentials. Explicit values win; otherwise operator-managed `answer_llm` mounts its `authSecret` as `NEMO_RETRIEVER_LLM_API_KEY` so LiteLLM/OpenAI has a credential value without writing it to the ConfigMap. |\n| `serviceConfig.llm.apiKeySecret.key` | `api_key` | Secret key for external LLM credentials. Operator-managed `answer_llm` uses `NGC_API_KEY` from `nimOperator.answer_llm.authSecret` when no explicit LLM Secret is set. |\n| `serviceConfig.llm.model` | `\"\"` | Optional explicit LiteLLM model id. Leave empty to inherit `nimOperator.answer_llm.model` when using the operator-managed answer LLM; set it for external endpoints. |\n| `serviceConfig.llm.ragSystemPromptPrefix` | `\"\"` | Optional explicit RAG prompt prefix. Leave empty unless an endpoint needs model-specific prompt directives. |\n| `serviceConfig.llm.reasoningEnabled` | `true` | Request-level reasoning toggle for `/v1/answer`. Defaults to true for external OpenAI-compatible providers; set false for Nemotron endpoints that should receive portable no-reasoning controls. |\n| `serviceConfig.agentic.enabled` | `false` | Enables `POST /v1/query` with `agentic=true` and the additive `agentic_query` MCP tool. Not auto-enabled by `nimOperator.answer_llm`. Refer to [Agentic retrieval (self-hosted Super-49B)](#agentic-retrieval-llm). |\n| `serviceConfig.agentic.llmModel` | `\"\"` | Chat model used by the inner agentic retrieval loop. Required when `invokeUrl` is set. Use the NIM-advertised ID (for Super-49B, `nvidia/llama-3.3-nemotron-super-49b-v1.5`), not the LiteLLM `openai/` prefix. |\n| `serviceConfig.agentic.invokeUrl` | `\"\"` | OpenAI-compatible chat completions endpoint used by agentic retrieval. Not auto-populated from `answer_llm`. For the in-cluster Super-49B NIM, set `http://answer-llm:8000/v1/chat/completions`. |\n| `serviceConfig.agentic.requestTimeoutS` | `1800` | Gateway and MCP timeout for the multi-step agentic retrieval call. |\n| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. |\n| `serviceConfig.vectordb.lancedbUri` | `/data/vectordb` | LanceDB on the vectordb Pod's PVC. |\n| `serviceConfig.vectordb.embedModel` | `nvidia/llama-nemotron-embed-vl-1b-v2` | Passed to vectordb + worker `embed_model_name`. |\n| `serviceConfig.vectordb.indexMode` | `hybrid` | Create LanceDB dense-vector and full-text-search indexes. Set to `dense` to create only the dense-vector index. |\n| `serviceConfig.vectordb.embedModelProviderPrefix` | `\"\"` | Optional LiteLLM provider prefix prepended to the remote embed model name. |\n| `serviceConfig.vectordb.writeTimeoutSeconds` | `300` | Rendered as `vectordb.write_timeout_s`. How long a worker waits for the vectordb Pod to acknowledge a record write. A write that is not acknowledged fails the document, so raise this value on slow storage. Refer to [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures). |\n\n### Sidecar metadata in split topology\n\n`ServiceIngestor.vdb_upload()` uploads sidecar metadata to\n`POST /v1/ingest/sidecar` and uses the returned opaque ID in the subsequent\ningest request. The public API retains its time-to-live and\n`consume_on_read` reuse behavior.\n\nSidecar metadata uses the gateway's in-memory store in standalone and split\ntopologies. Configure the maximum allowed payload size as needed:\n\n```yaml\nserviceConfig:\n sidecarStore:\n maxPayloadBytes: 33554432\n```\n\nIn `topology.mode: split`, the chart requires one gateway replica. Upload and\nreference each sidecar while that gateway remains running. An uploaded sidecar\nthat has not yet been admitted is lost if the gateway restarts or is replaced.\n\nAt ingest admission, the gateway authorizes and binds the sidecar to the\nwork attachment before it leases work to a worker. Workers receive the bound\nattachment with their work and do not resolve sidecar IDs. This supports\nindependent realtime and batch pools, worker replicas, and routing decisions\nwithout requiring sidecar replication between workers.\n\n#### VectorDB and the embed endpoint { #vectordb-and-the-embed-endpoint }\n\nThe vectordb Pod's `/v1/query` handler embeds the incoming query text\nbefore searching LanceDB. It needs a NIM embedding endpoint to do that,\nand rendering the Deployment with an empty `--embed-endpoint` produces a\nPod that passes its `/v1/health` probe but answers every `/v1/query`\nrequest with `HTTP 501 No embedding endpoint configured.` — a healthy\ndeployment that silently breaks retrieval.\n\nTo prevent this, the chart now refuses to render\n`deployment-vectordb.yaml` when no embed endpoint can be resolved.\n`helm install` / `helm upgrade --install` fails with a message listing\nthe three supported escape valves:\n\n```\nserviceConfig.vectordb.enabled=true but the embed endpoint could not be\nresolved. Pick one of:\n\n 1. --set serviceConfig.nimEndpoints.embedInvokeUrl=http://:/v1/embeddings\n 2. --set nimOperator.vlm_embed.enabled=true # requires apps.nvidia.com/v1alpha1 CRDs\n 3. --set serviceConfig.vectordb.enabled=false\n```\n\nResolution order matches the rest of the chart (refer to [Mix and match NIM\nsources](#3-install-with-the-nim-operator-in-cluster-nims)):\n\n1. Explicit `serviceConfig.nimEndpoints.embedInvokeUrl` always wins.\n2. Otherwise the operator-managed URL of\n `nimOperator.vlm_embed.nimServiceName` is used, provided\n `nimOperator.vlm_embed.enabled=true` **and** the\n `apps.nvidia.com/v1alpha1` CRDs are installed in the cluster.\n3. Otherwise the chart fails the install.\n\n#### Answer generation (operator-managed LLM) { #answer-generation-llm }\n\nEnable the generic `answer_llm` NIM slot to add service-mode answer\ngeneration on top of the VectorDB query path. The slot defaults to the\nSuper-49B NIM, but the image, model id, service name, resources,\nprofile filter, and environment can be overridden for another\nOpenAI-compatible LLM NIM.\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.answer_llm.enabled=true\n```\n\nWhen the NIM Operator CRDs are present, the chart renders an `answer-llm`\nNIMCache/NIMService by default and writes this block into\n`retriever-service.yaml`:\n\n```yaml\nllm:\n enabled: true\n model: \"openai/nvidia/llama-3.3-nemotron-super-49b-v1.5\"\n api_base: \"http://answer-llm:8000/v1\"\n rag_system_prompt_prefix: null\n reasoning_enabled: true\n```\n\nThe retriever service then exposes `POST /v1/answer`, which calls the\nVectorDB pod's `/v1/query` endpoint for context and sends those chunks to\nthe configured LLM endpoint. This path does not require tool calling.\nThe `answer_llm` NIM is not wired into `serviceConfig.agentic` and is\nnot tool-call ready by default. For agentic retrieval against that NIM,\nrefer to\n[Agentic retrieval (self-hosted Super-49B)](#agentic-retrieval-llm).\nThe `answer_llm` NIM deployment leaves\nreasoning defaults model-neutral; `/v1/answer` controls reasoning per\nrequest. By default, `serviceConfig.llm.reasoningEnabled=true`, so requests\nleave reasoning behavior to the LLM endpoint defaults and avoid sending\nprovider-specific `chat_template_kwargs` to external OpenAI-compatible\nendpoints. Set `serviceConfig.llm.reasoningEnabled=false` for Nemotron\nendpoints that should skip reasoning; the service then adds both `/no_think`\nand `chat_template_kwargs.enable_thinking=false`. The default Super-49B NIMService\nresources request two physical GPUs (`nvidia.com/gpu: 2`) to match the bundled\ntensor-parallel NIM profile. Do not satisfy that count with GPU Operator\ntime-slice replicas. Those two GPUs are in addition to the four core NIMs.\nThe chart NIMCache PVC is `250Gi`. A100 40GB, A10G, L40S, and RTX PRO 4500\nBlackwell are not supported for that default BF16 TP2 profile. Override\n`resources`, `modelProfile`, or `env` for deployments that use a different\nprofile or hardware topology. Refer to\n[Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements)\nin the Support Matrix.\n\nWhen `answer_llm` is enabled and no explicit `serviceConfig.llm.apiKeySecret`\nis set, the service also mounts `nimOperator.answer_llm.authSecret` as\n`NEMO_RETRIEVER_LLM_API_KEY`; OpenAI-compatible clients require a\ncredential value even for in-cluster NIM endpoints, and the key is never\nrendered into the ConfigMap.\n\nFor example, to try Nemotron 3 Nano as the answer LLM on an A100 80GB\nnode, override the operator-managed slot instead of adding a second\nhard-coded LLM service:\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.answer_llm.enabled=true \\\n --set nimOperator.answer_llm.nimServiceName=nemotron-3-nano \\\n --set nimOperator.answer_llm.image.repository=nvcr.io/nim/nvidia/nemotron-3-nano \\\n --set nimOperator.answer_llm.image.tag=1.7.0-variant \\\n --set nimOperator.answer_llm.model=openai/nvidia/nemotron-3-nano-30b-a3b \\\n --set-json nimOperator.answer_llm.modelProfile='{\"profiles\":[\"5f89f01a0af587fd8bae50c611b1f358f92effdb9fb29362e1af0a986e5561c3\"]}' \\\n --set-json nimOperator.answer_llm.resources='{\"limits\":{\"nvidia.com/gpu\":1},\"requests\":{\"nvidia.com/gpu\":1}}' \\\n --set nimOperator.answer_llm.env[0].name=NIM_HTTP_API_PORT \\\n --set-string nimOperator.answer_llm.env[0].value=8000 \\\n --set nimOperator.answer_llm.env[1].name=NIM_SERVED_MODEL_NAME \\\n --set-string nimOperator.answer_llm.env[1].value=nvidia/nemotron-3-nano-30b-a3b \\\n --set nimOperator.answer_llm.env[2].name=NIM_TENSOR_PARALLEL_SIZE \\\n --set-string nimOperator.answer_llm.env[2].value=1\n```\n\nUse the repository and tag available in your NGC environment; staging\nregistries can use the same override shape with `nvstaging` image names\nor tags. `nimOperator.answer_llm.model` is the LiteLLM model id used by\nthe retriever service; for an OpenAI-compatible in-cluster NIM, keep the\n`openai/` prefix there and set `NIM_SERVED_MODEL_NAME` to the raw model\nname advertised by the NIM. Replace the default Super-49B `modelProfile`,\n`resources`, and `env` when the target model requires a different\nGPU/profile setup. Leaving `modelProfile` empty preserves NIM\nOperator auto-discovery, but for Nano it can cache every advertised\nprofile on first reconciliation; pin a known-compatible profile when you\nknow the target GPU topology.\n\n`serviceConfig.llm.apiBase` and `serviceConfig.llm.model` can be set\nexplicitly to point `/v1/answer` at an external OpenAI-compatible LLM\ninstead of deploying an answer LLM in-cluster. For external credentials,\ncreate a Kubernetes Secret and set `serviceConfig.llm.apiKeySecret.name`\nplus `serviceConfig.llm.apiKeySecret.key`; Helm mounts the Secret as an\nenvironment variable instead of writing the key into the ConfigMap.\n\n`nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning` deploys Omni for\nimage captioning only. It does not enable `/v1/answer`. Omni\n(`nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`) is a supported\nconfigurable VLM-capable answer-generation backend. Use one of the\nfollowing:\n\n- Override the generic `answer_llm` slot with the Omni image\n `nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant`,\n set `nimOperator.answer_llm.model` to\n `openai/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`, set\n `NIM_SERVED_MODEL_NAME` to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`,\n and size `resources` plus `NIM_TENSOR_PARALLEL_SIZE` from the Omni rows in the\n [Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements)\n (one GPU on 80 GB or better, two GPUs on L40S). Leave `modelProfile`\n empty for NIM Operator auto-discovery, or pin a profile for your GPU.\n- If the Omni caption NIM is already in the cluster, reuse it for\n `/v1/answer` without deploying Super-49B:\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true \\\n --set serviceConfig.llm.enabled=true \\\n --set serviceConfig.llm.apiBase=http://nemotron-3-nano-omni-30b-a3b-reasoning:8000/v1 \\\n --set serviceConfig.llm.model=openai/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning\n```\n\nEnabling caption Omni and the default Super-49B `answer_llm` as separate\nNIMServices adds their GPU and disk requirements. Reusing the caption\nOmni endpoint for `/v1/answer` does not add a second Omni GPU or cache.\n\n#### Agentic retrieval (self-hosted Super-49B) { #agentic-retrieval-llm }\n\n`nimOperator.answer_llm.enabled=true` deploys Super-49B and auto-wires\nit only to `serviceConfig.llm` for `POST /v1/answer`. That answer path\nsends a plain text-generation request and does not require tool\ncalling. `serviceConfig.agentic` is a separate block. The chart does\nnot populate it from `answer_llm`.\n\nThe default Super-49B NIM starts with\n`NIM_PASSTHROUGH_ARGS=--disable-custom-all-reduce`. Agentic retrieval\nsends OpenAI-style tool-call messages with `tool_choice=auto`. A\nself-hosted vLLM-backed Super-49B NIM rejects those requests with\nHTTP 400 unless you also pass `--enable-auto-tool-choice` and\n`--tool-call-parser llama3_json`.\n\nYou can reuse the same Super-49B NIM for agentic retrieval after you\nadd those arguments. `POST /v1/answer` continues to work. This gap\ndoes not apply to NVIDIA-hosted Build endpoints.\n\nIf you set `nimOperator.answer_llm.env` in a values file, include\nthe full list. Change only the passthrough value:\n\n```yaml\nnimOperator:\n answer_llm:\n enabled: true\n env:\n - name: NIM_HTTP_API_PORT\n value: \"8000\"\n - name: NIM_TENSOR_PARALLEL_SIZE\n value: \"2\"\n - name: NIM_PASSTHROUGH_ARGS\n value: \"--disable-custom-all-reduce --enable-auto-tool-choice --tool-call-parser llama3_json\"\n - name: NCCL_IB_DISABLE\n value: \"1\"\n - name: NCCL_P2P_DISABLE\n value: \"1\"\n\nserviceConfig:\n agentic:\n enabled: true\n llmModel: nvidia/llama-3.3-nemotron-super-49b-v1.5\n invokeUrl: http://answer-llm:8000/v1/chat/completions\n```\n\nEquivalent `--set` form when you do not use a values file.\nHelm `--set` replaces the `env` list, so include every Super-49B\nenvironment entry and change only the `NIM_PASSTHROUGH_ARGS` value:\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.answer_llm.enabled=true \\\n --set nimOperator.answer_llm.env[0].name=NIM_HTTP_API_PORT \\\n --set-string nimOperator.answer_llm.env[0].value=8000 \\\n --set nimOperator.answer_llm.env[1].name=NIM_TENSOR_PARALLEL_SIZE \\\n --set-string nimOperator.answer_llm.env[1].value=2 \\\n --set nimOperator.answer_llm.env[2].name=NIM_PASSTHROUGH_ARGS \\\n --set-string nimOperator.answer_llm.env[2].value=\"--disable-custom-all-reduce --enable-auto-tool-choice --tool-call-parser llama3_json\" \\\n --set nimOperator.answer_llm.env[3].name=NCCL_IB_DISABLE \\\n --set-string nimOperator.answer_llm.env[3].value=1 \\\n --set nimOperator.answer_llm.env[4].name=NCCL_P2P_DISABLE \\\n --set-string nimOperator.answer_llm.env[4].value=1 \\\n --set serviceConfig.agentic.enabled=true \\\n --set serviceConfig.agentic.llmModel=nvidia/llama-3.3-nemotron-super-49b-v1.5 \\\n --set serviceConfig.agentic.invokeUrl=http://answer-llm:8000/v1/chat/completions\n```\n\n`serviceConfig.agentic.llmModel` is the model ID advertised by the\nNIM, not the LiteLLM `openai/` prefix used by\n`serviceConfig.llm.model`. Change `invokeUrl` if you override\n`nimOperator.answer_llm.nimServiceName`.\n\nAfter the NIM is Ready, confirm the passthrough arguments:\n\n```bash\nkubectl exec -n deploy/answer-llm -- printenv NIM_PASSTHROUGH_ARGS\n```\n\nThe value must include `--enable-auto-tool-choice` and\n`--tool-call-parser llama3_json`. For one-shot CLI use, port-forward\n`service/answer-llm` and point `--agentic-invoke-url` at\n`http://localhost:9000/v1/chat/completions`. For the CLI command,\nservice request, and MCP notes, refer to\n[Self-hosted Helm Super-49B](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/workflow-agentic-retrieval.md#self-hosted-helm-super-49b).\n\nFor other self-hosted OpenAI-compatible NIMs, enable automatic tool\nchoice and the parser that model requires. The `llama3_json` parser\nis the verified Super-49B setting.\n\n### NIM Operator sub-stack\n\nEach enabled NIM block under `nimOperator.` renders operator resources\ngated on three conditions ALL holding:\n\n1. The `apps.nvidia.com/v1alpha1` CRDs are installed in the cluster.\n2. The master switch `nims.enabled` is `true`.\n3. The per-NIM `nimOperator..enabled` is `true`.\n\n| Path | Default | Notes |\n|----------------------------------------|---------|-------|\n| `nims.enabled` | `true` | Master switch. Set false to render no NIM resources. |\n| `nimOperator.page_elements.enabled` | `true` | Page Elements 2.0 service; auto-wired to `/v1/page-elements`. |\n| `nimOperator.table_structure.enabled` | `true` | Table Structure 2.0 service; auto-wired to `/v1/table-structure`. |\n| `nimOperator..image` | `nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1` | Both services use the combined image but select distinct models. |\n| `nimOperator.ocr.enabled` | `true` | OCR NIM. |\n| `nimOperator.ocr.image` | `nvcr.io/nim/nvidia/nemotron-ocr-v2:2.0.1` | Default OCR NIM image. |\n| `nimOperator.vlm_embed.enabled` | `true` | Multimodal embedding NIM (also used by the vectordb Pod). |\n| `nimOperator.vlm_embed.nimServiceName` | `llama-nemotron-embed-vl-1b-v2` | NIMService / in-cluster DNS name. |\n| `nimOperator.vlm_embed.image` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:2.3.0` | Default VLM embed NIM image. |\n| `nimOperator.vlm_embed.env` | `NIM_HTTP_API_PORT=8000`, `NIM_TRITON_LOG_VERBOSE=1`, `OMP_NUM_THREADS=1`, `NIM_ENGINE_PRECISION=fp16` | Environment for the default VLM embed NIM. Overrides replace the complete list. |\n| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional). Set `true` to opt in — refer to [Query-time reranking](#query-time-reranking). Default `false` so chart installs honor the \"optional and disabled by default\" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. |\n| `nimOperator.rerankqa.image` | `nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:2.3.0` | Default optional VL reranker NIM image. |\n| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `method=\"nemotron_parse\"`. Default `false` so chart installs honor the \"optional and disabled by default\" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tags follow the [image tag conventions](#image-tag-conventions). |\n| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). This VLM is also a supported configurable `/v1/answer` backend. Enabling this key does not enable `/v1/answer`. Refer to [Answer generation (operator-managed LLM)](#answer-generation-llm). Default `false` so chart installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). |\n| `nimOperator.answer_llm.enabled` | `false` | Generic answer-generation LLM NIM (optional; Super-49B defaults). Set `true` to enable `/v1/answer` — refer to [Answer generation (operator-managed LLM)](#answer-generation-llm). This opt-in does not enable agentic retrieval. Refer to [Agentic retrieval (self-hosted Super-49B)](#agentic-retrieval-llm). Default `false` so installs do not silently claim answer-generation GPUs. |\n| `nimOperator.answer_llm.model` | `openai/nvidia/llama-3.3-nemotron-super-49b-v1.5` | LiteLLM/OpenAI model id inherited by `serviceConfig.llm.model` when the operator-managed answer LLM is enabled and no explicit service model is set. |\n| `nimOperator.answer_llm.ragSystemPromptPrefix` | `\"\"` | Optional prompt prefix inherited by `serviceConfig.llm.ragSystemPromptPrefix` only when explicitly set. Leave empty to keep the operator-managed LLM model-neutral and use `serviceConfig.llm.reasoningEnabled` for request-level reasoning control. |\n| `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. |\n| `nimOperator..image.repository` | `nvcr.io/nim/nvidia/...` | Per-NIM image. |\n| `nimOperator..image.pullSecrets` | `[]` | Per-NIM pull Secret name list. Empty inherits `ngcImagePullSecret.name` (default `ngc-secret`) on every NIMCache / NIMService. Non-empty replaces the chart-wide name for that NIM only. |\n| `nimOperator..authSecret` | `\"\"` | Per-NIM auth Secret name. Empty inherits `ngcApiSecret.name` (default `ngc-api`). Non-empty replaces the chart-wide name for that NIM only. |\n| `nimOperator..storage.pvc.size` | `25Gi` (50Gi for vlm_embed/rerankqa, 100Gi parse, 300Gi VL) | NIMCache PVC size. |\n| `nimOperator..storage.pvc.storageClass` | `\"\"` | Per-NIM NIMCache StorageClass. An empty value renders an empty class on the NIMCache CR, so the operator-created claim uses the cluster default when one exists. Set this path for each enabled NIM. `nimOperator.nimCache.pvc.storageClass` is not applied to per-NIM caches. |\n| `nimOperator..replicas` | `1` | Per-NIMService replica count. |\n| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Four core NIMs therefore request four GPU slots unless the cluster shares GPUs. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions). Refer to [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade) and [GPU scheduling prerequisite](#gpu-scheduling-prerequisite). |\n| `nimOperator..resources` | `{}` | Per-NIM override of the whole `resources` block. Empty uses `nimServiceGpuLimit`; non-empty replaces the chart default (may require `--force-conflicts` on later `helm upgrade`). |\n| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). |\n| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). |\n| `nimOperator..expose.service.port` | `8000` (9000 for audio) | HTTP port. |\n| `nimOperator..expose.service.grpcPort` | `8001` (50051 for audio) | gRPC port. |\n\n> The four \"core\" NIMs (page_elements, table_structure, ocr, vlm_embed)\n> are enabled and auto-wired by default. Optional NIMs stay off until\n> `nimOperator..enabled` is `true`. When you opt in, the chart\n> auto-wires Omni captioning and VL reranking into `nim_endpoints`\n> (refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b) and\n> [Query-time reranking](#query-time-reranking)); other optional NIMs\n> still need an explicit serviceConfig hook (for example\n> `audioGrpcEndpoint` for Parakeet ASR). For minimal installs, prefer the\n> [minimal install](#recommended-minimal-install-2608) overrides.\n\n#### Filtering cached GPU profiles { #filtering-cached-gpu-profiles }\n\nEvery NIMCache the chart renders supports the NIM Operator's\n`spec.source.ngc.model` block, which restricts which model profiles the\ncache job downloads. The chart exposes this through two values:\n\n| Path | Scope | Behaviour |\n| ---- | ----- | --------- |\n| `nimOperator.modelProfile` | Chart-wide | Applied to every NIMCache that doesn't carry its own override. |\n| `nimOperator..modelProfile` | Per-NIM | When non-empty, **REPLACES** the chart-wide default (no merge). |\n\nBoth default to `{}`. With both empty the chart emits no `model:`\nblock and the NIM Operator falls back to its \"cache every profile\napplicable to the detected GPUs\" default — fine on a single-GPU\nlaptop, but on heterogeneous clusters (or any cluster with ≥ 3 NIMs)\nthis wastes tens of GiB of PVC storage, NGC bandwidth, and cache-job\nruntime.\n\nThe mapping is rendered verbatim under `spec.source.ngc.model`, so the\nshape lines up 1:1 with the [NIMCache CRD](https://docs.nvidia.com/nim-operator/latest/reference-nimcache.html).\nTwo filter dimensions are supported (use whichever fits your cluster;\n`gpus` is the common case):\n\n```yaml\nnimOperator:\n modelProfile:\n gpus:\n # NIMCache only downloads profiles compatible with at least one\n # of these GPU selectors. Each selector is {ids: [...], product: ...}.\n - ids: [\"26B5\"] # PCI device ID(s)\n product: \"NVIDIA-H100-80GB-HBM3\" # NVIDIA marketing name\n # profiles:\n # # Alternative: list of exact profile UUIDs from `ngc registry\n # # model list-profiles /:`.\n # - \"11111111-2222-3333-4444-555555555555\"\n```\n\nEquivalent overrides via `--set`:\n\n```bash\n# Homogeneous H100 80 GB cluster — every NIMCache only pulls the H100 profile:\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set 'nimOperator.modelProfile.gpus[0].ids[0]=26B5' \\\n --set 'nimOperator.modelProfile.gpus[0].product=NVIDIA-H100-80GB-HBM3'\n\n# Restrict only the page_elements NIMCache to a specific profile UUID, leave the rest alone:\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set 'nimOperator.page_elements.modelProfile.profiles[0]=11111111-2222-3333-4444-555555555555'\n\n# Chart-wide H100 default plus a per-NIM override (the override REPLACES the global; it does NOT merge):\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set 'nimOperator.modelProfile.gpus[0].product=NVIDIA-H100-80GB-HBM3' \\\n --set 'nimOperator.vlm_embed.modelProfile.profiles[0]=22222222-3333-4444-5555-666666666666'\n```\n\nTips:\n\n- Run `ngc registry model list-profiles nvcr.io/nim/nvidia/:` to enumerate the available profiles for any chart-pinned NIM image and pick the smallest profile that matches your GPU.\n- Filter mismatches surface as `NIMCache` events such as `NoCompatibleProfile`; check with `kubectl describe nimcache `.\n- The chart's defaults (`{}`) preserve operator behaviour, so adding `modelProfile` is a strict opt-in — existing releases keep working unchanged.\n\n#### Image tag conventions { #image-tag-conventions }\n\nEvery NIM in this chart pins an exact NGC image tag in `values.yaml`\n— there is no `:latest` floating reference. Two tag families show up:\n\n| Family | Example | Meaning |\n| ------ | ------- | ------- |\n| Plain semver | `nemotron-object-detection:2.0.1` | A standard NIM release, identical bytes on every pull. Used by the four core NIMs and the reranker / ASR NIMs. |\n| `-variant` | `nemotron-parse-v1.2:1.7.0-variant`, `nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant` | The Nemotron Parse and Nemotron 3 Nano Omni 30B builds that ship per-GPU TensorRT engine variants the NIM Operator selects from at reconciliation time (refer to the Omni and Parse rows in the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table). The `-variant` suffix is the NGC tag that ships alongside this chart and matches footnote ³ of the support matrix. |\n\nFor air-gapped mirror pipelines: mirror the *exact* tag — both the\nplain semver and the `-variant` form — and do not substitute `:latest`.\nSubstituting `:latest` would pin to a moving target that may not match\nthe engine plans the NIM Operator profile expects for a given GPU.\n\nIf you want a different NIM build on a **new** install, override the tag\nexplicitly:\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true \\\n --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.image.tag=\n```\n\nand validate against the same release of the retriever service before\nproduction rollout.\n\nIf a `NIMCache` for that NIM already exists, do not change\n`image.repository` or `image.tag` with `helm upgrade` alone.\nThe NIM Operator rejects in-place `modelPuller` updates.\nDelete the cache first, then upgrade. Refer to\n[Changing a NIM image repository or tag](#changing-nim-image-repository-or-tag).\n\n**Charts and captioning.** Charts and infographics use **Page Elements, Table Structure**\nand **ocr**. For image\ncaptioning, set `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` — refer to\n[Image captioning (Omni 30B)](#image-captioning-omni-30b) for the\nchart-side wiring and\n[Image captioning](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning)\nfor the product matrix.\n\n#### Changing a NIM image repository or tag { #changing-nim-image-repository-or-tag }\n\nEach chart NIM renders a `NIMCache` whose `metadata.name` stays the same\nacross image changes. `spec.source.ngc.modelPuller` is the concatenation\nof `nimOperator..image.repository` and `nimOperator..image.tag`.\nThe NIM Operator `NIMCache` CRD (including 3.1.2) marks `modelPuller`\nimmutable. Kubernetes rejects an update with a message similar to:\n\n```text\nmodelPuller is an immutable field. Please create a new NIMCache resource instead when you want to change this container.\n```\n\nA `helm upgrade` that only changes the repository or tag fails\non the existing object. Other release resources can already have been\napplied before that rejection, which leaves the release partially upgraded.\n\nDo not rename `nimOperator..nimServiceName` as a workaround.\nA new name creates a second cache and Service DNS label. The previous\n`NIMCache` remains, especially when `nimOperator.nimCache.keepOnUninstall`\nis `true` (the default).\n\nThe following table lists default `NIMCache` names for the core NIMs.\nOptional NIMs follow the same immutable-`modelPuller` rule.\n\n| Helm key | Default `NIMCache` name |\n| --- | --- |\n| `nimOperator.page_elements` | `nemotron-page-elements-v3` |\n| `nimOperator.table_structure` | `nemotron-table-structure-v1` |\n| `nimOperator.ocr` | `nimOperator.ocr.nimServiceName` (`nemotron-ocr-v2`) |\n| `nimOperator.vlm_embed` | `nimOperator.vlm_embed.nimServiceName` (`llama-nemotron-embed-vl-1b-v2`) |\n\n**Before you change a repository or tag** on an existing release,\ncomplete the following steps for every NIM whose image changes.\nThe affected NIM is unavailable while the operator re-caches weights.\n\n1. Drain ingest traffic that depends on that NIM.\n2. Confirm the live `modelPuller` value differs from the new\n `repository:tag`:\n\n ```bash\n NS=retriever\n CACHE=nemotron-page-elements-v3\n\n kubectl get nimcache \"${CACHE}\" -n \"${NS}\" \\\n -o jsonpath='{.metadata.name}{\" \"}{.spec.source.ngc.modelPuller}{\"\\n\"}'\n ```\n\n Compare that `modelPuller` value with\n `:`.\n3. Delete the `NIMCache`. Helm `keep` annotations do not block\n `kubectl delete`:\n\n ```bash\n kubectl delete nimcache \"${CACHE}\" -n \"${NS}\"\n ```\n\n4. If the operator-created PVC remains, delete it so the new image\n re-pulls weights. List operator PVCs, then delete the claim for\n that cache. Default claim names use a `-pvc` suffix, for example\n `nemotron-page-elements-v3-pvc`. Confirm the name from the list\n before you delete it. Refer to\n [Persistent storage prerequisite](#persistent-storage-prerequisite).\n\n ```bash\n kubectl get pvc -n \"${NS}\" -l 'app.kubernetes.io/managed-by=nvidia-nim-operator'\n kubectl delete pvc \"${CACHE}-pvc\" -n \"${NS}\"\n ```\n\n5. Run `helm upgrade` with the new `image.repository` or `image.tag`.\n Helm creates a new `NIMCache` with the updated `modelPuller`.\n6. Wait until the new cache is ready before you send traffic:\n\n ```bash\n kubectl get nimcache \"${CACHE}\" -n \"${NS}\"\n kubectl get nimservice \"${CACHE}\" -n \"${NS}\"\n kubectl describe nimcache \"${CACHE}\" -n \"${NS}\"\n ```\n\nRepeat those steps for each NIM whose repository or tag changes,\nincluding chart default tag bumps between releases.\n\n`helm uninstall` does not remove kept `NIMCache` objects when\n`keepOnUninstall` is `true`. A later install or upgrade with a\ndifferent image still requires this delete and re-cache sequence.\n\n**If `helm upgrade` already failed** with the immutable-`modelPuller`\nmessage, delete the existing `NIMCache` and its PVC, then re-run the\nsame upgrade so Helm creates the cache instead of patching it.\n\nChanging `service.image.repository` or `service.image.tag` does not\nuse `NIMCache` and is not subject to this rule.\n\n#### Query-time reranking { #query-time-reranking }\n\nThe VL reranker NIM (`llama-nemotron-rerank-vl-1b-v2`) backs\n`POST /v1/query` with `rerank=true`. When you enable it,\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.rerankqa.enabled=true \\\n ...\n```\n\nthe chart auto-wires two fields into the rendered\n`retriever-service.yaml` ConfigMap (including every split-topology\nrole ConfigMap):\n\n```yaml\nnim_endpoints:\n rerank_invoke_url: \"http://llama-nemotron-rerank-vl-1b-v2:8000/v1/ranking\"\n rerank_model_name: \"nvidia/llama-nemotron-rerank-vl-1b-v2\"\n```\n\nWithout those fields the gateway returns\n`HTTP 400 Reranking is not configured` even when the Rerank NIM Pod is\nReady.\n\nResolution order mirrors every other NIM endpoint (see the\n[NIM Operator sub-stack](#nim-operator-sub-stack) section):\n\n1. Explicit `serviceConfig.nimEndpoints.rerankInvokeUrl` always wins\n (use this to point at a hosted or external ranking endpoint).\n2. Otherwise the operator-managed URL of\n `llama-nemotron-rerank-vl-1b-v2` is used, provided\n `nimOperator.rerankqa.enabled=true` **and** the\n `apps.nvidia.com/v1alpha1` CRDs are installed.\n3. Otherwise `rerank_invoke_url` stays `null` and query-time reranking\n stays disabled.\n\n`serviceConfig.nimEndpoints.rerankModelName` follows the same order —\nit defaults to the canonical VL reranker model id\n(`nvidia/llama-nemotron-rerank-vl-1b-v2`) whenever the chart resolves any\nrerank URL. Override only when pointing at a different ranking SKU.\n\n\n#### Image captioning (Omni 30B) { #image-captioning-omni-30b }\n\nThe Nemotron 3 Nano Omni VLM is the canonical image-caption NIM for\nthis chart. When you enable it,\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n --set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true \\\n ...\n```\n\nthe chart now auto-wires two fields into the rendered\n`retriever-service.yaml` ConfigMap:\n\n```yaml\nnim_endpoints:\n caption_invoke_url: \"http://nemotron-3-nano-omni-30b-a3b-reasoning:8000/v1/chat/completions\"\n caption_model_name: \"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning\"\n```\n\nThe service derives `caption_enabled=true` from a non-null\n`caption_invoke_url`, so the ingestion pipeline routes caption work to\nthe in-cluster Omni Pod with no manual ConfigMap edits.\n\nResolution order mirrors every other NIM endpoint (see the\n[NIM Operator sub-stack](#nim-operator-sub-stack) section):\n\n1. Explicit `serviceConfig.nimEndpoints.captionInvokeUrl` always wins\n (use this to point at a hosted endpoint, e.g.\n `https://integrate.api.nvidia.com/v1/chat/completions`).\n2. Otherwise the operator-managed URL of\n `nemotron-3-nano-omni-30b-a3b-reasoning` is used, provided\n `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true`\n **and** the `apps.nvidia.com/v1alpha1` CRDs are installed.\n3. Otherwise `caption_invoke_url` stays `null` and the caption stage\n is disabled.\n\n`serviceConfig.nimEndpoints.captionModelName` follows the same order —\nit defaults to the canonical Omni remote model id\n(`nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`, matching\n`nemo_retriever.common.modality.caption.model_profiles.OMNI_REMOTE_MODEL_ID`) whenever\nthe chart resolves any caption URL. Override only when pointing at a\ndifferent VLM SKU.\n\n#### GPU limits and `helm upgrade` { #gpu-limits-and-helm-upgrade }\n\nThe chart defaults to **`nimOperator.nimServiceGpuLimit: 1`**, which\nrenders `spec.resources.limits.nvidia.com/gpu: 1` on every NIMService\nunless a per-NIM `resources` map overrides it. A default core install\ntherefore requests **four GPU slots** (one per enabled NIMService).\nSetting that per-NIM limit is required on NIM Operator **v3.1.2**\n(and other versions tested on A100/H100): when the chart omits the\n`resources` block entirely, the operator often **does not** populate\nGPU limits from the model profile, and NIM pods start without GPU\naccess (`The NVIDIA Driver was not detected`).\n\nThis per-NIM `1` request is not the support-matrix VRAM figure of\none A10G for all four core models combined. On a conventional\ncluster, plan for four allocatable GPUs across eligible nodes or\nconfigure sharing.\nRefer to [GPU scheduling prerequisite](#gpu-scheduling-prerequisite).\n\n**Trade-off:** Helm and the NIM Operator may both server-side-apply\n`spec.resources.limits.nvidia.com/gpu`. A later `helm upgrade --install`\ncan then fail with:\n\n```\nError: UPGRADE FAILED: conflict occurred while applying object\n / apps.nvidia.com/v1alpha1, Kind=NIMService:\n Apply failed with 1 conflict:\n conflict with \"manager\" using apps.nvidia.com/v1alpha1:\n .spec.resources.limits.nvidia.com/gpu\n```\n\n**Operator-only mode** (omit GPU limits from Helm — only if your NIM\nOperator version reliably reconciles them):\n\n```yaml\nnimOperator:\n nimServiceGpuLimit: null\n```\n\n**If upgrades hit SSA conflicts** after the operator has reconciled GPU\nlimits, use one of:\n\n1. `helm upgrade --install … --force-conflicts --server-side`\n2. `kubectl -n edit nimservice ` to set GPU limits outside Helm\n\nTo pin a non-default GPU count chart-wide, set `nimServiceGpuLimit: 2`\n(or set per-NIM `resources.limits.nvidia.com/gpu`).\n\nA failed upgrade that reports an immutable `modelPuller` field is a\ndifferent class of error. Refer to\n[Changing a NIM image repository or tag](#changing-nim-image-repository-or-tag).\n\n### OCR NIM configuration { #ocr-nim-configuration }\n\nThe core OCR NIM is configured under [`nimOperator.ocr`](./values.yaml) (the `ocr:`\nblock). Confirm `image.repository` and `image.tag` before you upgrade.\nIf either value changes on an existing release, delete the OCR `NIMCache`\nbefore you upgrade. Refer to\n[Changing a NIM image repository or tag](#changing-nim-image-repository-or-tag).\n\n| Path | Role |\n|------|------|\n| `nimOperator.nimCache.keepOnUninstall` | When `true`, NIMCache CRs survive `helm uninstall` (`helm.sh/resource-policy: keep`). NIMService CRs are always removed. Set `false` for dev clusters that should fully tear down on uninstall. |\n| `nimOperator.ocr.enabled` | Reconcile the OCR `NIMService` |\n| `nimOperator.ocr.image.repository` | NIM image (default `nvcr.io/nim/nvidia/nemotron-ocr-v2`) |\n| `nimOperator.ocr.image.tag` | Pin the image tag for reproducible upgrades |\n\nOverride the auto-wired in-cluster URL with `serviceConfig.nimEndpoints.ocrInvokeUrl`\nwhen the OCR service runs outside the operator sub-stack.\n\n### Persistence\n\nComplete the [persistent storage prerequisite](#persistent-storage-prerequisite)\nbefore a default install. Empty `storageClass` values omit\n`storageClassName` and rely on a default StorageClass or a compatible\nclassless persistent volume.\n\n| Path | Default | Notes |\n|----------------------------|-------------------------------|-------|\n| `persistence.enabled` | `true` | Mount the pre-existing general PVC for logs and other non-scheduler uses. |\n| `persistence.size` | `50Gi` | |\n| `persistence.accessModes` | `[ReadWriteOnce]` | Access mode for the general PVC. |\n| `persistence.storageClass` | `\"\"` | Use cluster default unless set. Use `\"-\"` to disable a `storageClassName`. |\n| `persistence.existingClaim` | `\"\"` | When set, skip PVC creation and mount this claim. |\n| `persistence.mountPath` | `/var/lib/nemo-retriever` | General persistent files only; scheduler state and payloads are never stored here. |\n| `retrieverResults.enabled` | `true` | Create the results PVC unless `existingClaim` is set. |\n| `retrieverResults.storageClass` | `\"\"` | Use cluster default unless set. Use `\"-\"` to disable a `storageClassName`. |\n| `retrieverResults.existingClaim` | `\"\"` | When set, skip PVC creation and mount this claim. |\n| `topology.vectordb.persistence.enabled` | `true` | Create the VectorDB PVC when `serviceConfig.vectordb.enabled` is `true`. |\n| `topology.vectordb.persistence.storageClass` | `\"\"` | Use cluster default unless set. Use `\"-\"` to disable a `storageClassName`. No `existingClaim` path. |\n\nThe gateway enforces active-lease budgets independently of worker replicas.\n`serviceConfig.workQueue.maxActiveLeases.realtime` defaults to `8` and\n`.batch` defaults to `48`; treat these as explicit downstream-capacity budgets,\nnot values inferred from pod or NIM counts.\n\n#### Scheduler loss boundary and upgrade from durable releases\n\nThe work spool always uses `serviceConfig.workQueue.spoolDirectory` under the\ngateway `/tmp` `emptyDir`; `persistence.enabled` does not change scheduler\nbehavior. During one gateway lifetime, FIFO order, lease caps, delivery attempts,\ngenerations, stale-lease rejection, worker recovery after lease expiry, and\nqueued-plus-active demand metrics remain available.\n\nReplacing the gateway loses accepted jobs, queued payloads, active leases, job\nstatus history, and SSE catch-up state. After replacement, status and event\nrequests for old jobs return not found, and old callbacks and heartbeats return\n`409`. Clients must create a new job and submit the documents again. Worker loss\nremains recoverable through lease expiry only while the same gateway process is\nalive. Public ingest and worker HTTP wire formats are unchanged.\n\nBefore upgrading from a release with durable scheduler checkpoints, drain the\ngateway. The ephemeral implementation neither reads nor automatically deletes\n`gateway-state.sqlite3` or payload files left under an older PVC. After rollback\nto that release is no longer required, operators may manually remove its old\n`work-queue` directory from the general PVC. The removed\n`work_queue.persistence_enabled` key was internal and unreleased; delete it from\ncustom service configuration files.\n\n### Secrets\n\n| Path | Default | Notes |\n|-----------------------------------|----------------|-------|\n| `ngcImagePullSecret.create` | `false` | Chart-managed dockerconfigjson Secret. |\n| `ngcImagePullSecret.name` | `ngc-secret` | Name referenced by every Pod and, when per-NIM `image.pullSecrets` is empty, by every NIMCache / NIMService. |\n| `ngcImagePullSecret.password` | `\"\"` | NGC API key. |\n| `ngcApiSecret.create` | `false` | Chart-managed Opaque Secret. |\n| `ngcApiSecret.name` | `ngc-api` | Name referenced by NIMCache/NIMService `authSecret` when per-NIM `authSecret` is empty. |\n| `ngcApiSecret.password` | `\"\"` | NGC API key (populates `NGC_API_KEY` + `NGC_CLI_API_KEY`). |\n| `imagePullSecrets` | `[]` | Extra pre-existing pull secrets appended to every Pod. |\n| `serviceConfig.vectordb.internalAuth.enabled` | `false` | Enable the Secret-backed credential for VectorDB traffic and restricted gateway-to-worker handoffs. |\n| `serviceConfig.vectordb.internalAuth.existingSecret.name` | `\"\"` | Existing Secret shared by Retriever and VectorDB pods. |\n| `serviceConfig.auth.scopeTokenSecret.name` | `\"\"` | Existing Secret containing the public scope-token JSON file. |\n| `serviceConfig.auth.enabled` | `false` | Require bearer authentication for the public gateway. |\n| `serviceConfig.auth.allowInsecureInlineApiToken` | `false` | Explicit development-only gate for ConfigMap-backed `apiToken`. |\n\n### Optional features\n\n| Feature | Toggle | Default |\n|-------------------|---------------------------------|---------|\n| Ingress | `ingress.enabled` | `true` |\n| Autoscaling (HPA) | `autoscaling.enabled` | `false` (max=1 anyway) |\n| ServiceMonitor | `serviceMonitor.enabled` | `false` |\n\n---\n\n## Configuration recipes\n\n### Mount a custom retriever-service.yaml verbatim\n\nThe chart renders `retriever-service.yaml` from structured values so you\nshouldn't normally need to ship a verbatim file. If you really want to,\nmount one via `service.extraVolumes` + `service.extraVolumeMounts` at\n`/etc/nemo-retriever/retriever-service.yaml` (which silently overrides the\nchart-managed ConfigMap because `subPath` mounts win).\n\n### Use externally managed Secrets\n\n```yaml\nngcImagePullSecret:\n create: false # don't render; reference an existing Secret\n name: my-org-ngc-pull\nngcApiSecret:\n create: false\n name: my-org-ngc-api\n```\n\nThe chart will skip Secret creation. Make sure `my-org-ngc-pull` exists\nas `kubernetes.io/dockerconfigjson` and `my-org-ngc-api` as `Opaque` with\nan `NGC_API_KEY` key, in the release namespace. Retriever Pods and every\nrendered NIMCache / NIMService inherit those names unless you set a\nnon-empty per-NIM `image.pullSecrets` or `authSecret` override.\n\nProtect the public gateway and internal service calls with separate\npre-existing Secrets:\n\n```yaml\nserviceConfig:\n auth:\n scopeTokenSecret:\n name: nrl-public-auth\n key: scope-tokens.json\n enabled: true\n vectordb:\n internalAuth:\n enabled: true\n existingSecret:\n name: nrl-internal-vdb-auth\n key: token\n```\n\n`nrl-public-auth` must contain a JSON document such as\n`{\"tokens\":[{\"token\":\"\",\"scopes\":[\"workspace-123\"]}]}` under the\nconfigured key. `nrl-internal-vdb-auth` must contain a distinct, high-entropy\ncredential. In split topology, the public token file mounts only on the\ngateway. The gateway authenticates public requests, then uses the internal\ncredential for restricted worker handoffs and pull-worker claims of\n`/v1/internal/work/claim`. Workers do not receive or validate the public\nbearer token, so `serviceConfig.auth.scopeTokenSecret.name` also requires\n`serviceConfig.vectordb.internalAuth.enabled=true`. Internal authentication\nis opt-in for local compatibility; enable it for production deployments. When\nenabled, a missing Secret or key prevents the pods from starting instead of\nfalling back to unauthenticated VectorDB access. Inline\n`serviceConfig.auth.apiToken` is rejected unless\n`allowInsecureInlineApiToken=true`, and must never be used for production.\n\n### Disable one NIM and supply an external URL for it\n\n```yaml\nnimOperator:\n vlm_embed:\n enabled: false # don't deploy the embed NIM in-cluster\n\nserviceConfig:\n nimEndpoints:\n embedInvokeUrl: https://integrate.api.nvidia.com/v1/embeddings\n```\n\nThe chart's resolution order is **explicit URL → operator-managed URL →\nempty**, so per-endpoint overrides Just Work.\n\n### Roll the service after editing values\n\nThe `Deployment` carries a `checksum/config` annotation derived from the\nConfigMap, so `helm upgrade` automatically rolls the pod when any\n`serviceConfig.*` value changes.\n\n---\n\n## Timeouts and alleviating ingest failures\n\nBatch ingest fans out extract and embed work to remote NIM HTTP endpoints.\nUnder heavy parallelism a single slow or overloaded NIM can cause timeouts,\nand a worker process crash can surface as many simultaneous `failed`\ndocument callbacks even though only one root cause occurred.\n\n### What the chart configures\n\n| Layer | Default | Where it is set |\n|-------|---------|-----------------|\n| Remote embed HTTP calls | **600 s** (10 min) | Service image (`EmbedParams.request_timeout_s`); not a Helm value today. |\n| Gateway → realtime/batch proxy | **300 s** | Rendered `gateway.timeout_s` in `retriever-service.yaml` (split topology). |\n| Worker → vectordb record write | **300 s** | `serviceConfig.vectordb.writeTimeoutSeconds`, rendered as `vectordb.write_timeout_s` in `retriever-service.yaml`. |\n| VLM embed model name | `serviceConfig.vectordb.embedModel` | Also copied into worker `nim_endpoints.embed_model_name` in the ConfigMap. |\n\nSymptoms to look for in pod logs:\n\n- `Embedding error occurred: timed out` or `httpx.ReadTimeout` on the **batch** pod.\n- `Batch process pool broken (worker crash)` followed by many\n `BrokenProcessPool` failures on other in-flight documents.\n- Embed NIM pod messages such as `failed to allocate pinned system memory`\n (GPU pressure from too many concurrent `/v1/embeddings` requests).\n- `Failed to POST records to vectordb for ` on the **batch** or\n **realtime** pod, followed by a `failed` document. The vectordb Pod did not\n acknowledge the record write in time.\n\nThe **gateway** pod usually only logs `status=failed` callbacks; diagnose on\n**batch** (and **realtime** for page-sized uploads), plus the embed NIM pod.\n\n### Recommended mitigations\n\n**1. Lower batch worker concurrency (first step).**\n\nThe default `serviceConfig.pipeline.batchWorkers` is `48`, which can saturate\na single in-cluster VLM embed NIM. If you see embed timeouts or pool crashes,\nreduce batch parallelism to **16** and redeploy:\n\n```bash\nhelm upgrade retriever ./nemo_retriever/helm \\\n --reuse-values \\\n --set serviceConfig.pipeline.batchWorkers=16\n```\n\nYou can tune further (for example `8` on small GPU nodes), but **16** is a\nreasonable starting point when moving off the default. Realtime workers\n(`realtimeWorkers`, default `24`) are less likely to overload embed NIMs\nbecause they handle smaller units of work; adjust them only if realtime\ningest shows the same timeout pattern.\n\n**2. Confirm embed wiring.**\n\nEnsure `nim_endpoints.embed_model_name` in the mounted config matches the\nVLM embed NIM SKU (`serviceConfig.vectordb.embedModel`, default\n`nvidia/llama-nemotron-embed-vl-1b-v2`). A model mismatch produces\nHTTP 404 on `/v1/embeddings`, not a timeout, but is worth ruling out when\ndebugging failed ingests.\n\n**3. Retry failed documents.**\n\nFailures caused by a one-time pool restart are often transient. After lowering\n`batchWorkers` and rolling the batch Deployment, resubmit documents that\nfailed with `rows=0`.\n\n**4. Scale or isolate the embed NIM.**\n\nIf timeouts persist at `batchWorkers: 16`, add embed NIM replicas (when your\ncluster has GPU capacity), point `serviceConfig.nimEndpoints.embedInvokeUrl`\nat an external embed endpoint, or temporarily disable optional NIMs on\ndev clusters to free GPU memory for `vlm_embed`.\n\n**5. Client and ingress timeouts.**\n\nLong batch jobs may exceed the gateway proxy timeout (300 s) or an Ingress\n`proxy-read-timeout`. Increase ingress annotations if clients disconnect\nwhile workers are still processing; see the commented example on\n`ingress.annotations` in `values.yaml`.\n\n**6. Raise the vectordb write timeout on slow storage.**\n\nA worker fails the document when the vectordb Pod does not acknowledge its\nrecord write:\n\n```text\nVectorDB write failed for report.pdf: . The extracted rows are not\nconfirmed durable, so the document is not queryable.\n```\n\nThat acknowledgement covers the row commit plus the index maintenance that\nfollows it, so a large batch on slow storage can exceed the default 300 s.\nRaise the timeout and redeploy:\n\n```bash\nhelm upgrade retriever ./nemo_retriever/helm \\\n --reuse-values \\\n --set serviceConfig.vectordb.writeTimeoutSeconds=900\n```\n\nConcurrent writes to one vectordb Pod do not block each other, so a write\ntimeout points at storage throughput or table size rather than write\nserialization. Refer to\n[LanceDB index creation fails during concurrent Helm ingestion](https://docs.nvidia.com/nemo/retriever/latest/extraction/troubleshoot/#lancedb-concurrent-index-creation).\n\n---\n\n## Queue-depth autoscaling (split mode)\n\nIn `topology.mode: split` deployments the realtime and batch worker\npods scale horizontally based on the gateway's **central outstanding demand** and\n**95th-percentile processing latency**. Demand is queued records plus active leases from the gateway while\nlatency comes from workers; both publishers are always on (see\n`nemo_retriever_work_queue_demand` in\n[`prometheus.py`](../src/nemo_retriever/service/services/prometheus.py)).\nThe only choice you have to make is **how the metrics get from\nPrometheus into the Kubernetes HPA**.\n\n### Why queue depth (and not CPU)\n\nCPU-based HPA reacts to *the pod that has already saturated its work*.\nFor an ingest pipeline that fans out to remote NIM endpoints, the work\nspends most of its time blocked on HTTP — CPU stays low even when the\nqueue is full. Queue depth measures *demand to be served*, which is\nwhat we actually want to scale on. A 95th-percentile-latency signal\nrides alongside to catch the inverse case (a single hot pod whose\nqueue is shallow but whose per-item processing has stalled).\n\n### Backend choices\n\nThe chart's `autoscaling.queueDepth.backend` controls which path is\nwired up. All three options leave the metrics publisher untouched:\n\n| backend | When to pick it | Cluster prerequisite |\n|------------------------|------------------------------------------------------------------|-----------------------------------|\n| `prometheus-adapter` *(default)* | Production. One adapter feeds HPA + Grafana + future autoscalers. | Prometheus Operator + `prometheus-community/prometheus-adapter`. |\n| `cpu` | Bootstrap / dev cluster without Prometheus. | None — built-in. |\n| `keda` | Already standardised on KEDA org-wide. | KEDA operator (you install + apply your own `ScaledObject`). |\n\nThe chart-recommended path is `prometheus-adapter`. The reasoning is\ndocumented in `values.yaml`; in short, it keeps a single Prometheus as\nthe source of truth, supports HPA's multi-metric arithmetic-mean\nevaluation out of the box, and doesn't force the chart to bundle new\nCRDs.\n\n### Wiring up prometheus-adapter (recommended)\n\nThe chart renders a ConfigMap named\n`-nemo-retriever-prom-adapter-rules` containing PromQL rules\nfor the External Metrics API. You point your existing\nprometheus-adapter at it:\n\n```bash\nhelm upgrade prometheus-adapter prometheus-community/prometheus-adapter \\\n --namespace monitoring \\\n --reuse-values \\\n --set rules.existing=-nemo-retriever-prom-adapter-rules\n```\n\nThen verify both metrics show up in the External Metrics API:\n\n```bash\nkubectl get --raw \\\n \"/apis/external.metrics.k8s.io/v1beta1/namespaces/$NS/nemo_retriever_gateway_work_queue_backlog?labelSelector=pool%3Drealtime\" \\\n | jq .\n```\n\nOnce that returns a non-empty `items` array, the HPAs rendered by this\nchart will start consuming them. The HPA annotation\n`nemo-retriever.nvidia.com/hpa-signals` documents the active set per\nHPA, e.g. `queueRatio=true latencyP95=true cpu=false`.\n\n### CPU fallback (no Prometheus required)\n\nSet `autoscaling.queueDepth.backend: cpu` and enable the CPU metric\nunder each role:\n\n```yaml\nautoscaling:\n queueDepth:\n backend: cpu\ntopology:\n realtime:\n hpa:\n metrics:\n queueBacklog: { enabled: false }\n processingLatencyP95: { enabled: false }\n cpu: { enabled: true, targetUtilizationPercentage: 60 }\n batch:\n hpa:\n metrics:\n queueBacklog: { enabled: false }\n processingLatencyP95: { enabled: false }\n cpu: { enabled: true, targetUtilizationPercentage: 80 }\n```\n\nThe legacy `topology..hpa.targetCPUUtilizationPercentage` field\nstill works and behaves as an alias for the `metrics.cpu` block.\n\n### KEDA path\n\nSet `autoscaling.queueDepth.backend: keda` and disable the chart-managed\nHPAs:\n\n```yaml\nautoscaling:\n queueDepth: { backend: keda }\ntopology:\n realtime: { hpa: { enabled: false } }\n batch: { hpa: { enabled: false } }\n```\n\nThen apply your own `ScaledObject` — example for the realtime pool:\n\n```yaml\napiVersion: keda.sh/v1alpha1\nkind: ScaledObject\nmetadata:\n name: nemo-retriever-realtime\nspec:\n scaleTargetRef:\n name: nemo-retriever-realtime\n minReplicaCount: 2\n maxReplicaCount: 8\n cooldownPeriod: 300\n triggers:\n - type: prometheus\n metadata:\n serverAddress: http://prometheus.monitoring.svc:9090\n metricName: nemo_retriever_pool_queue_depth_ratio\n threshold: \"0.5\"\n query: |\n avg by (pool) (\n nemo_retriever_pool_queue_depth{pool=\"realtime\"}\n /\n on(pool, instance) group_left()\n nemo_retriever_pool_max_queue_size{pool=\"realtime\"}\n )\n - type: prometheus\n metadata:\n serverAddress: http://prometheus.monitoring.svc:9090\n metricName: nemo_retriever_pool_processing_duration_p95\n threshold: \"30\"\n query: |\n histogram_quantile(\n 0.95,\n sum by (le, pool) (\n rate(nemo_retriever_pool_processing_duration_seconds_bucket{pool=\"realtime\"}[2m])\n )\n )\n```\n\nKEDA's biggest win is **scale-from-zero**, which we don't use today —\nboth `minReplicas` defaults are ≥ 1 because the realtime pod is on the\nhot path for SSE consumers. If you do want scale-from-zero (e.g. a\nnightly batch-only job tenant), KEDA is the right tool and this is the\nescape hatch.\n\n### Tuning the thresholds\n\nPer-role tuning lives under `topology..hpa.metrics`:\n\n```yaml\ntopology:\n realtime:\n hpa:\n metrics:\n queueBacklog: { enabled: true, target: \"24\" }\n processingLatencyP95: { enabled: true, targetSeconds: \"30\" }\n batch:\n hpa:\n metrics:\n queueBacklog: { enabled: true, target: \"48\" }\n processingLatencyP95: { enabled: true, targetSeconds: \"120\" }\n```\n\nThe demand `target` is outstanding documents per replica because the HPA uses\n`type: AverageValue`. When migrating an existing values file, rename\n`metrics.queueDepthRatio` to `metrics.queueBacklog`, choose a document-count\ntarget near the role's execution-slot count, and rename\n`prometheusAdapter.queueDepthRatioMetric` to `queueBacklogMetric`. Both legacy\nkeys remain aliases for this release: legacy enable/disable values win, legacy\nmetric names name the new demand metric, and fractional ratio targets are\nreplaced by the role's backlog-count default with an HPA annotation and NOTES\nwarning. The aliases are scheduled for removal in the following release.\n\n### Verifying it scales\n\n```bash\n# Cause realtime pressure (anything that submits to /v1/ingest/job/.../page).\n# Then watch the HPA decide:\nkubectl get hpa -w\n\n# And watch the active signals on each HPA:\nkubectl get hpa -realtime -o jsonpath='{.metadata.annotations.nemo-retriever\\.nvidia\\.com/hpa-signals}'\n```\n\nThe dashboard's *Worker Pool Capacity* card on the **Overview** page\nmirrors the same signal Prometheus is seeing, so it's a quick eyeball\nsanity check before opening Grafana.\n\n---\n\n## Tracing and Zipkin\n\nHelm installs the chart-owned OpenTelemetry Collector and Zipkin backend on by\ndefault. This is intentional: the legacy 26.1.2 Helm chart shipped with a\nmanaged Zipkin deployment enabled, so the new chart keeps a default trace\nbackend available for functional parity. OTLP trace and metric export is also\nenabled by default for retriever service pods and chart-managed NIMs:\n\n```yaml\ntopology:\n otel:\n enabled: true\n zipkin:\n enabled: true\n\nservice:\n otel:\n enabled: true\n\nnimOperator:\n otel:\n enabled: true\n```\n\nBecause Zipkin is chart-owned by default, an upgrade with default values can\ncreate a Zipkin Deployment and Service. Set `topology.zipkin.enabled=false`\nbefore upgrading if your deployment uses an external backend or should not run\nchart-owned Zipkin.\n\nWith default values, retriever service pods and chart-managed NIMs emit OTLP to\nthe chart's OpenTelemetry Collector. The Collector exports traces to the\nchart-owned Zipkin service and exposes received metrics in Prometheus format.\nThe chart configures a 5-second metric export interval. Set\n`service.otel.enabled=false` or `nimOperator.otel.enabled=false` to opt out by\nsurface. Open a job and read the Zipkin lookup key from either the JSON body or\nthe `x-trace-id` response header:\n\n```bash\nkubectl port-forward svc/tracing-smoke-nemo-retriever 7670:80\n\ncurl -s -D headers.txt -o job.json \\\n -X POST http://localhost:7670/v1/ingest/job \\\n -H 'content-type: application/json' \\\n -d '{\"expected_documents\":1}'\n\nTRACE_ID=$(jq -r .trace_id job.json)\ngrep -i x-trace-id headers.txt\n```\n\nPort-forward Zipkin and query the trace directly:\n\n```bash\nkubectl port-forward svc/tracing-smoke-nemo-retriever-zipkin 9411:9411\ncurl \"http://localhost:9411/api/v2/trace/${TRACE_ID}\"\n```\n\n### Prometheus metrics from the OpenTelemetry Collector\n\nWhen `topology.otel.enabled=true`, the chart-owned OpenTelemetry Collector\nexposes metrics received through OTLP in Prometheus format. The endpoint uses\n`topology.otel.ports.prometheus`, which defaults to port `8889`. The\nchart-owned OpenTelemetry Collector Service exposes the same port. With default\nvalues, the retriever service and chart-managed NIMs export OTLP metrics to this\nendpoint.\n\nThe retriever service also retains its native Prometheus `/metrics` endpoint.\nEnable `serviceMonitor.enabled=true` when a Prometheus Operator should scrape\nthat endpoint directly. Direct scraping remains useful for service metrics that\ndo not use OTLP, including the worker-pool metrics used by split-mode\nautoscaling.\n\nAfter a successful ingestion, allow up to 30 seconds for metric export and\nverify the endpoint by port-forwarding the Collector Service:\n\n```bash\nkubectl port-forward svc/-nemo-retriever-otel 8889:8889\nmetric_found=false\nfor attempt in {1..30}; do\n if curl -fsS http://127.0.0.1:8889/metrics | grep -q '^nemo_retriever_'; then\n metric_found=true\n break\n fi\n sleep 1\ndone\ntest \"${metric_found}\" = true\n```\n\nSet `topology.otel.ports.prometheus` to use a different port. The chart updates\nthe Collector listener and Service port together. If the command does not find\na metric after 30 seconds, confirm that `topology.otel.enabled` and the\nworkload's OpenTelemetry settings are enabled, then inspect the Collector logs.\n\nCommon opt-out and override knobs:\n\n```yaml\ntopology:\n zipkin:\n enabled: false # do not deploy chart-owned Zipkin\n exporter:\n enabled: false # keep Zipkin deployed, but do not export traces to it\n endpoint: http://external-zipkin:9411/api/v2/spans\n\nservice:\n otel:\n enabled: true # required for the following service overrides to render\n env:\n OTEL_METRICS_EXPORTER: none # retain tracing, but do not export service metrics\n OTEL_METRIC_EXPORT_INTERVAL: \"10000\" # service metric cadence in milliseconds\n# To opt out of all service OTLP telemetry instead, set `service.otel.enabled: false`.\n\nnimOperator:\n otel:\n enabled: true # required for the following inherited NIM override to render\n env:\n NIM_OTEL_METRICS_EXPORTER: \"console\" # do not send inherited NIM metrics to OTLP\n# To opt out of all inherited NIM OTLP telemetry instead, set `nimOperator.otel.enabled: false`.\n page_elements:\n otel:\n enabled: false # per-NIM opt-out\n ocr:\n otel:\n env:\n NIM_OTEL_METRICS_EXPORTER: \"console\" # per-NIM metric-export opt-out\n TRITON_OTEL_RATE: \"10\" # per-NIM Triton OTel override\n```\n\nSet `topology.zipkin.exporter.endpoint` when you run your own Zipkin-compatible\ncollector. Set `topology.otel.enabled=false` to disable the chart-owned collector\nand all chart-rendered collector wiring. Values in `service.env` override the\nchart-managed service OpenTelemetry environment variables. Existing NIM\ncontainer environment variables take precedence over inherited\n`nimOperator.otel.env` values, and per-NIM `nimOperator..otel.env` values\noverride the inherited NIM values.\n\n---\n\n## OpenShift deployment { #openshift-deployment }\n\nOpenShift install procedures, **restricted-v2** / PSA **restricted** value overrides, prebuilt `ffmpeg` images, internal registry pull secrets, optional NIM `LD_LIBRARY_PATH` tuning, and install examples are in **[OpenShift deployment](./openshift.md)**. Pass `-f openshift-restricted.yaml` from that guide when you install on OpenShift.\n\n---\n\n## Air-gapped deployment { #air-gapped-deployment }\n\nRefer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring:\n\n### Container images to mirror (chart defaults)\n\nVerify tags on the Git branch or tag you ship (for example `main` or\nyour release tag). Defaults below match\n[`values.yaml`](./values.yaml) on the current chart.\n\n| Role | `nimOperator` key | Default image (`repository:tag`) |\n|------|-------------------|----------------------------------|\n| Retriever service | — | `service.image.repository`:`service.image.tag` (override for production) |\n| Page Elements | `page_elements` | `nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1` |\n| Table Structure | `table_structure` | `nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1` |\n| OCR | `ocr` | `nvcr.io/nim/nvidia/nemotron-ocr-v2:2.0.1` |\n| VL embed | `vlm_embed` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:2.3.0` |\n| VL reranker (optional) | `rerankqa` | `nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:2.3.0` |\n| Nemotron Parse (optional) | `nemotron_parse` | `nvcr.io/nim/nvidia/nemotron-parse-v1.2:1.7.0-variant` |\n| Omni caption or configurable answer VLM (optional) | `nemotron_3_nano_omni_30b_a3b_reasoning` | `nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant` |\n| Answer LLM (optional, Super-49B default) | `answer_llm` | `nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5:2.0.5` |\n| Parakeet ASR (optional) | `audio` | `nvcr.io/nim/nvidia/parakeet-1-1b-ctc-en-us:1.5.0` |\n\nGPU SKU support for `audio` is in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements).\n\nAlso mirror images for the vectordb sidecar, Redis, or other subcharts if\nyour values enable them.\n\n### Helm values for a private registry\n\nExample overrides (replace placeholders):\n\n```bash\nhelm upgrade --install retriever ./nemo_retriever/helm \\\n -f my-airgap-values.yaml\n```\n\n`my-airgap-values.yaml` should include at least:\n\n```yaml\nservice:\n image:\n repository: /nemo-retriever-service\n tag: \n pullPolicy: IfNotPresent\n\nimagePullSecrets:\n - name: my-private-registry\n\nngcImagePullSecret:\n create: false\n name: \"\" # Explicitly empty — clears the default \"ngc-secret\"\n\nnimOperator:\n page_elements:\n image:\n repository: /nemotron-object-detection\n tag: \"2.0.1\"\n pullPolicy: IfNotPresent\n # Repeat for table_structure, ocr, vlm_embed, and any optional keys you enable.\n```\n\n- Set `nimOperator..image.pullSecrets` to your mirror pull secret\n (for example `my-private-registry`) when it differs from\n `ngcImagePullSecret.name`. Empty per-NIM `pullSecrets` inherit the\n chart-wide name.\n- Leave `serviceConfig.nimEndpoints.*` empty when operator-managed NIMs\n are in-cluster; set explicit URLs only for external or mirrored services\n outside the chart.\n- For **offline captioning**, enable\n `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning` and point the pipeline\n caption endpoint at the in-cluster NIM URL (refer to\n [Image captioning](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning)).\n\n### Mirroring pattern\n\n```bash\ndocker login nvcr.io -u '$oauthtoken' -p \"$NGC_API_KEY\"\ndocker pull nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1\ndocker tag nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1 \\\n /nemotron-object-detection:2.0.1\ndocker push /nemotron-object-detection:2.0.1\n```\n\nFor bulk sync, prefer [skopeo](https://github.com/containers/skopeo) or\n[crane](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md).\nRecord `repository@sha256:...` digests for regulated environments.\n\n---\n\n## Roadmap\n\n1. **External scheduler backend** — introduce shared job, queue, lease, and\n SSE state before allowing more than one gateway replica.\n2. **NetworkPolicies** restricting the service Pod to the NIM Pods\n only.\n3. **Gateway autoscaling** on inflight-uploads (currently fixed\n `topology.gateway.replicas`) — shared scheduler and SSE ownership must land first.\n\n---\n\n## Validation\n\nThe chart is exercised in CI with `helm lint` and `helm template`. Run\nlocally:\n\n```bash\nhelm lint nemo_retriever/helm\n\n# Operator CRDs present: vectordb resolves vlm_embed via the operator URL.\nhelm template r nemo_retriever/helm \\\n --api-versions apps.nvidia.com/v1alpha1 > /tmp/r-op.yaml\n\n# Operator CRDs absent: vectordb has no operator URL to fall back to, so\n# either disable vectordb or supply an explicit embed endpoint.\nhelm template r nemo_retriever/helm \\\n --set serviceConfig.vectordb.enabled=false > /tmp/r.yaml\n# or:\n# helm template r nemo_retriever/helm \\\n# --set serviceConfig.nimEndpoints.embedInvokeUrl=http://embed.svc:8000/v1/embeddings \\\n# > /tmp/r.yaml\n```\n\nBoth renders should succeed cleanly and parse as valid Kubernetes manifests\n(`kubectl apply --dry-run=client -f /tmp/r.yaml`). Refer to [VectorDB and the\nembed endpoint](#vectordb-and-the-embed-endpoint) for why\n`helm template r nemo_retriever/helm` without flags is rejected as a\nmisconfiguration.","displayName","NVIDIA NeMo Retriever Helm Chart",["D",1787683747054],"publisher","NVIDIA","latestVersion","collectionCounts","imageUrl","/api/image/nvidia/nemo-microservices/helm-charts/nemo-retriever","rawImageUrl","https://developer-blogs.nvidia.com/wp-content/uploads/2024/03/nemo-retriever-graphic.png","canGuestDownload","latestVersionSizeInBytes","licenseTerms","isMultinodeEnabled","isMultiArchitecture","bias","explainability","privacy","safetyAndSecurity","featureLabels",[],"displayLabels",[242],"policyLabels",[],"systemLabels",[],"products",[],"hasNimMetadata",{"_211":243,"_244":243,"_245":246,"_247":248},"NSPECT-AF3Z-BZUP","value","filter",{"_249":248,"_244":243},"category","label","field",{"_121":1228,"_123":1229},"ngcApps",[1181],"csaApiBaseUrl","https://www.nvidia.com/ai-agents/csa-agent/dcsm","datadogRum",{"_1162":1163,"_1164":1165,"_1166":1167,"_1168":1169,"_1170":261,"_152":287,"_1171":1172,"_1173":60,"_1174":89,"_1175":89,"_1176":1177,"_1178":89,"_1179":1180},"amplitude",{"_1159":1160,"_801":-7,"_1161":35},"disableJs","environment","prod","featureFlagsBlob","JTdCJTIydGVzdEZsYWclMjIlM0FmYWxzZSUyQyUyMnNob3dTZWFyY2hMYWJlbFNldEZpbHRlcnMlMjIlM0FmYWxzZSUyQyUyMmVuYWJsZUNzYVdpZGdldCUyMiUzQWZhbHNlJTJDJTIyZW5hYmxlUHJvZHVjdEdyb3VwaW5nQnlTdWJzY3JpcHRpb24lMjIlM0FmYWxzZSUyQyUyMmNvbnRhaW5lclNjYW5uaW5nUmVkZXNpZ25NMiUyMiUzQWZhbHNlJTdE","featureFlagsGuarded","isDevLikeEnv","isPlaywright","isMobileDevice","categorizedOrgs",{"_1153":1154,"_1155":1156,"_1157":1158},"currentOrgContext","currentContextRoles",[],"enterpriseOrgs",[],"manageableExternalUserGroups",[],"ngcApiOrigin","ngcHelmOrigin","origin","https://catalog.ngc.nvidia.com","usesMocks","theme","year",2026,"now",1787727738694,"1.12.0","defaultSearchSuggestions",{"_1041":368,"_1042":1043,"_1044":1045,"_1046":1047},"announcements",[],"consentState",{"_325":1031,"_1032":89,"_1033":35,"_1034":35,"_1035":1036},"consentConfig",{"_307":308,"_309":310,"_311":312,"_313":314,"_315":316,"_317":318,"_319":320,"_321":322,"_323":324,"_325":326},"showContactAdmin",1787727738695,[],[],[301,302,303,304,305,306],{"_49":63,"_51":74,"_53":116,"_55":-5,"_57":-5,"_59":60},{"_49":63,"_51":70,"_53":116,"_55":-5,"_57":-5,"_59":60},{"_49":63,"_51":68,"_53":116,"_55":-5,"_57":-5,"_59":60},{"_49":63,"_51":64,"_53":116,"_55":-5,"_57":-5,"_59":60},{"_49":61,"_51":62,"_53":116,"_55":-5,"_57":-5,"_59":60},{"_49":50,"_51":52,"_53":116,"_55":-5,"_57":-5,"_59":60},"mainInfoText","We and our third-party partners (including social media, advertising, and analytics partners) use cookies and other tracking technologies to collect, store, monitor, and process certain information about you when you visit our website. The information collected might relate to you, your preferences, or your device. We use that information to make the site work, analyze performance and traffic on our website, provide a more personalized web experience, and assist in our marketing efforts.

Under certain privacy laws, you have the right to direct us not to \"sell\" or \"share\" your personal information for targeted advertising. To opt-out of the \"sale\" and \"sharing\" of personal information through cookies, you must opt-out of optional cookies using the toggles below. To opt out of the \"sale\" and \"sharing\" of data collected by other means (e.g., online forms) you must also update your data sharing preferences through the NVIDIA Preference Center.

Click on the different category headings below to find out more and change the settings according to your preference. You cannot opt out of Required Cookies as they are deployed to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, etc.). By clicking \"Save and Accept\" or \"Decline All\" at the bottom, you consent to the use of cookies and other tools as described in our Cookie Policy in accordance with your settings and accept our Terms of Service (which contains important waivers). For more information about our privacy practices, please see our Privacy Policy.","mainText","Cookie Settings","alertNoticeText",{"_1023":1024,"_1025":1026,"_1027":1028,"_1029":1030},"bannerAcceptLabel","Accept All","bannerRejectLabel","Reject Optional","bannerSettingsLabel","Manage Settings","preferenceSaveLabel","Save Choices","preferenceRejectLabel","Decline All","alwaysActiveText","Always Active","groups",[327,328,329,330,331],{"_332":713,"_79":714,"_179":715,"_336":89,"_337":18,"_339":716,"_341":717},{"_332":616,"_79":617,"_179":618,"_336":35,"_337":58,"_339":619,"_341":620},{"_332":523,"_79":524,"_179":525,"_336":35,"_337":526,"_339":527,"_341":528},{"_332":343,"_79":344,"_179":345,"_336":35,"_337":346,"_339":347,"_341":348},{"_332":333,"_79":334,"_179":335,"_336":35,"_337":338,"_339":340,"_341":342},"optanonGroupId","C0005","Social Media Cookies","These cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.","alwaysActive","order",11,"cookies",[],"hosts",[],"C0004","Advertising Cookies","These cookies record your visit to our websites, the pages you have visited and the links you have followed to influence the advertisements that you see on other websites. These cookies and the information they collect may be managed by other companies, including our advertising partners, and may be used to build a profile of your interests and show you relevant advertising on other sites. We and our advertising partners will use this information to make our websites and the advertising displayed on it, more relevant to your interests.",5,[465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486],[349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365],{"_366":460,"_211":460,"_179":368,"_339":461},{"_366":456,"_211":456,"_179":368,"_339":457},{"_366":451,"_211":451,"_179":368,"_339":452},{"_366":447,"_211":447,"_179":368,"_339":448},{"_366":442,"_211":442,"_179":368,"_339":443},{"_366":437,"_211":437,"_179":368,"_339":438},{"_366":429,"_211":429,"_179":368,"_339":430},{"_366":421,"_211":421,"_179":368,"_339":422},{"_366":416,"_211":416,"_179":368,"_339":417},{"_366":411,"_211":411,"_179":368,"_339":412},{"_366":406,"_211":406,"_179":368,"_339":407},{"_366":401,"_211":401,"_179":368,"_339":402},{"_366":396,"_211":396,"_179":368,"_339":397},{"_366":389,"_211":389,"_179":368,"_339":390},{"_366":384,"_211":384,"_179":368,"_339":385},{"_366":377,"_211":377,"_179":368,"_339":378},{"_366":367,"_211":367,"_179":368,"_339":369},"hostName","amazon-adsystem.com","",[370,371],{"_79":376,"_373":367,"_374":35,"_179":375},{"_79":372,"_373":367,"_374":35,"_179":375},"ad-privacy","host","isSession","This domain is owned by online retailer Amazon and is used as part of its affiliate marketing programme.","ad-id","doubleclick.net",[379,380],{"_79":383,"_373":377,"_374":35,"_179":382},{"_79":381,"_373":377,"_374":35,"_179":382},"IDE","This domain is owned by Doubleclick (Google). The main business activity is: Doubleclick is Googles real time bidding advertising exchange","test_cookie","bing.com",[386],{"_79":387,"_373":384,"_374":35,"_179":388},"MUID","This domain is owned by Mircosoft - it is the site for the search engine Bing.","assets.adobedtm.com",[391,392],{"_79":395,"_373":389,"_374":35,"_179":394},{"_79":393,"_373":389,"_374":35,"_179":394},"kndctr_F207D74D549850760A4C98C6_AdobeOrg_*","If you know anything about this host and its uses please add your contributions here. ","kndctr_F207D74D549850760A4C98C6_AdobeOrg_consent","tiktok.com",[398],{"_79":399,"_373":396,"_374":35,"_179":400},"_ttp","TikTok is a social media platform owned by ByteDance, known for short-form video sharing. The service is used to create and share videos, and it utilizes cookies for various purposes including advertising and analytics.","t.co",[403],{"_79":404,"_373":401,"_374":35,"_179":405},"muc_ads","This domain is owned by Twitter. It is used as a URL shortening service, which allows Twitter to collect data on how links are shared.","snowplow.com",[408],{"_79":409,"_373":406,"_374":35,"_179":410},"sp","Testing","google.com",[413],{"_79":414,"_373":411,"_374":35,"_179":415},"NID","This domain is owned by Google Inc. Although Google is primarily known as a search engine, the company provides a diverse range of products and services. Its main source of revenue however is advertising. Google tracks users extensively both through its own products and sites, and the numerous technologies embedded into many millions of websites around the world. It uses the data gathered from most of these services to profile the interests of web users and sell advertising space to organisations based on such interest profiles as well as aligning adverts to the content on the pages where its customer's adverts appear.","linkedin.com",[418],{"_79":419,"_373":416,"_374":35,"_179":420},"bcookie","This domain is owned by LinkedIn, the business networking platform. It typically acts as a third party host where website owners have placed one of its content sharing buttons in their pages, although its content and services can be embedded in other ways. Although such buttons add functionality to the website they are on, cookies are set regardless of whether or not the visitor has an active Linkedin profile, or agreed to their terms and conditions. For this reason it is classified as a primarily tracking/targeting domain.","twitter.com",[423,424],{"_79":427,"_373":421,"_374":35,"_179":428},{"_79":425,"_373":421,"_374":35,"_179":426},"personalization_id","This domain is owned by Twitter. The main business activity is: Social Networking Services. Where twitter acts as a third party host, it collects data through a range of plug-ins and integrations, that is primarily used for tracking and targeting.","guest_*","This domain is owned by X (formerly Twitter). The main business activity is: Social Networking Services. Where X (formerly Twitter) acts as a third party host, it collects data through a range of plug-ins and integrations, that is primarily used for tracking and targeting.","youtube.com",[431,432],{"_79":435,"_373":429,"_374":35,"_179":436},{"_79":433,"_373":429,"_374":35,"_179":434},"VISITOR_INFO1_LIVE","This cookie is used as a unique identifier to track viewing of videos","__Secure-*","YouTube is a Google owned platform for hosting and sharing videos. YouTube collects user data through videos embedded in websites, which is aggregated with profile data from other Google services in order to display targeted advertising to web visitors across a broad range of their own and other websites.","demdex.net",[439],{"_79":440,"_373":437,"_374":35,"_179":441},"demdex","This cookie helps Adobe Audience Manger perform basic functions such as visitor identification, ID synchronization, segmentation, modeling, reporting, etc.","im-apps.net",[444],{"_79":445,"_373":442,"_374":35,"_179":446},"receive-cookie-deprecation","This domain is owned by im-apps.net which provides information and resources about various instant messaging (IM) applications to help users make informed decisions based on their communication needs.","influ2.com",[449],{"_79":450,"_373":447,"_374":35,"_179":410},"R","6sc.co",[453],{"_79":454,"_373":451,"_374":35,"_179":455},"6suuid","This domain is associated with the company 6Sense Insights, Inc., which provides predictive analytics and account engagement services. It is primarily used for tracking and targeting in marketing automation and advertising.","hushly.com",[458],{"_79":459,"_373":456,"_374":35,"_179":410},"_hly_gvid","t.contentsquare.net",[462],{"_79":463,"_373":460,"_374":89,"_179":464},"_cs_*","This is a session cookie. This is a pattern type cookie where a random 13-digit number is appended to the prefix _cs_.",{"_79":521,"_373":488,"_374":35,"_179":522},{"_79":519,"_373":488,"_374":35,"_179":520},{"_79":517,"_373":488,"_374":35,"_179":518},{"_79":399,"_373":488,"_374":35,"_179":516},{"_79":493,"_373":488,"_374":89,"_179":495},{"_79":515,"_373":488,"_374":35,"_179":368},{"_79":514,"_373":491,"_374":35,"_179":368},{"_79":512,"_373":488,"_374":35,"_179":513},{"_79":510,"_373":488,"_374":35,"_179":511},{"_79":509,"_373":488,"_374":35,"_179":368},{"_79":507,"_373":488,"_374":35,"_179":508},{"_79":493,"_373":506,"_374":89,"_179":495},{"_79":504,"_373":491,"_374":35,"_179":505},{"_79":504,"_373":494,"_374":35,"_179":505},{"_79":463,"_373":488,"_374":35,"_179":464},{"_79":502,"_373":488,"_374":35,"_179":503},{"_79":409,"_373":500,"_374":35,"_179":501},{"_79":498,"_373":488,"_374":35,"_179":499},{"_79":496,"_373":488,"_374":35,"_179":497},{"_79":493,"_373":494,"_374":89,"_179":495},{"_79":490,"_373":491,"_374":35,"_179":492},{"_79":487,"_373":488,"_374":89,"_179":489},"AMCVS_xxxxxAdobeOrg","nvidia.com","This is a pattern type cookie name associated with Adobe Marketing Cloud. It stores a unique visitor identifier, and uses an organisation identifier to allow a company to track users across their domains and services.This is a pattern type cookie name associated with Adobe Marketing Cloud. It stores a unique visitor identifier, and uses an organisation identifier.","_rdt_pn","www.nvidia.com","This cookie name is associated with www.mind.org.uk which is used for marketing purposes.","AMP_*","build.nvidia.com","Supports Amplitude product analytics.","_gcl_au","Used by Google AdSense for experimenting with advertisement efficiency across websites using their services","_mkto_trk","This cookie is associated with an email marketing service provided by Marketo. This tracking cookie allows a website to link visitor behaviour to the recipient of an email marketing campaign, to measure campaign effectiveness.","investor.nvidia.com","This cookie is associated with Eventbrite and is used to deliver content tailored to the end user's interests and improve content creation. This cookie is also used for event-booking purposes.","_gat_gtag_UA_140825383_2","Google Analytics","_rdt_uuid","This cookie is set by Reddit and is used for tracking the views of embedded videos and performance of the advertisements.","catalog.ngc.nvidia.com","_uetsid","This cookie is used by Bing to determine what ads should be shown that may be relevant to the end user perusing the site. ","kndctr_F207D74D549850760A4C98C6_AdobeOrg_bc_session_id","ttcsid_XXXXXXXXXXXXXXXXXXXXCategory","This cookie name is associated with TikTok which is used to track your activity on the platform, including how you interact with their embedded services on other websites.This cookie is associated with TikTok. It helps track your activity on the platform, including how you interact with their embedded services on other websites.","_fbp","Used by Facebook to deliver a series of advertisement products such as real time bidding from third party advertisers","_zitok","_twpid","This cookie is typically associated with the use of Taptap Digital, a company specializing in digital advertising. It is used for tracking and ad personalization purposes.","_tt_enable_cookie","This cookie name is associated with TikTok which is used to collect data about behavior and purchases on the website and to measure the effect of advertising. It is also used to track website activity to help optimize advertising efforts.","AMCV_xxxxxAdobeOrg","This cookie is used to identify a unique visitor for tracking purposes by Adobe Experience Cloud.This is a pattern type cookie name associated with Adobe Marketing Cloud. It stores a unique visitor identifier, and uses an organisation identifier to allow a company to track users across their domains and services.","ttcsid","This cookie name is associated with TikTok which is used to enable TikTok Ads session tracking by assigning a unique session ID to users who click on TikTok ads, allowing for the attribution of user behavior and conversions within the same session.This cookie is associated with TikTok. It helps track your activity on the platform, including how you interact with their embedded services on other websites.","C0003","Personalization Cookies","These cookies collect data about how you have interacted with our website to help us improve your web experience, such as which pages you have visited. These cookies may store a unique ID so that our system will remember you when you return. They may be set by us or by third party providers whose services we have added to our pages. These cookies enable us to provide enhanced website functionality and personalization as well as make the marketing messages we send to you more relevant to your interests. If you do not allow these cookies, then some or all of these services may not function properly.",4,[558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582],[529,530,531,532,533],{"_366":442,"_211":442,"_179":368,"_339":553},{"_366":416,"_211":416,"_179":368,"_339":548},{"_366":543,"_211":543,"_179":368,"_339":544},{"_366":538,"_211":538,"_179":368,"_339":539},{"_366":534,"_211":534,"_179":368,"_339":535},"nvidia.partners",[536],{"_79":537,"_373":534,"_374":35,"_179":410},"edge","app.hushly.com",[540],{"_79":541,"_373":538,"_374":35,"_179":542},"*CORS","This domain is associated with Hushly, a platform that provides lead conversion solutions for content marketing by using personalization and content engagement strategies.","github.com",[545],{"_79":546,"_373":543,"_374":35,"_179":547},"_octo","This domain is owned by Github. The main business activity is: Source code repository and collaboration platform",[549,550],{"_79":552,"_373":416,"_374":35,"_179":420},{"_79":551,"_373":416,"_374":35,"_179":420},"lidc","li_gc",[554,555],{"_79":557,"_373":442,"_374":35,"_179":446},{"_79":556,"_373":442,"_374":35,"_179":446},"imid_*","imid",{"_79":583,"_373":491,"_374":35,"_179":585},{"_79":614,"_373":488,"_374":35,"_179":615},{"_79":583,"_373":613,"_374":35,"_179":585},{"_79":611,"_373":488,"_374":89,"_179":612},{"_79":609,"_373":488,"_374":35,"_179":610},{"_79":608,"_373":591,"_374":89,"_179":368},{"_79":583,"_373":587,"_374":35,"_179":585},{"_79":606,"_373":488,"_374":35,"_179":607},{"_79":583,"_373":494,"_374":35,"_179":585},{"_79":594,"_373":491,"_374":35,"_179":368},{"_79":583,"_373":605,"_374":35,"_179":585},{"_79":603,"_373":488,"_374":35,"_179":604},{"_79":600,"_373":601,"_374":35,"_179":602},{"_79":598,"_373":599,"_374":35,"_179":368},{"_79":583,"_373":595,"_374":35,"_179":585},{"_79":597,"_373":488,"_374":35,"_179":368},{"_79":583,"_373":596,"_374":35,"_179":585},{"_79":594,"_373":595,"_374":35,"_179":368},{"_79":593,"_373":591,"_374":89,"_179":368},{"_79":583,"_373":592,"_374":35,"_179":585},{"_79":583,"_373":591,"_374":35,"_179":585},{"_79":583,"_373":590,"_374":35,"_179":585},{"_79":589,"_373":488,"_374":89,"_179":585},{"_79":586,"_373":587,"_374":89,"_179":588},{"_79":583,"_373":584,"_374":35,"_179":585},"_gd_*","info.nvidia.com","This cookie contains no identifying information and is used for diagnostic purposes by Impact Radius, a service we use to track signup commissions for our affiliate program. We have categorized it under Functionality because it is necessary for the operation of our affiliate program.","nv_country_code","developer.nvidia.com","This cookie is likely used to store the country code of the visitor to provide localized content and possibly for legal compliance with regional data privacy laws.","_gdxxxxxxxxxxxxx","nvidianews.nvidia.com","docs.api.nvidia.com","research.nvidia.com","readme_library","wcmmode","ai-podcast.nvidia.com","resources.nvidia.com","geo_locator","openedx-language-preference","learn.nvidia.com","doxygen_width","raytracing-docs.nvidia.com","This cookie is likely used to remember the width preferences of a sidebar or interface element within a documentation platform like Doxygen.","_sfid_eb78","This cookie is categorized as Functionality Cookie.","crowdsource.nvidia.com","mbox","Cookie set by Adobe SiteCatalyst analytics software - used for measuring the performance of page content using A/B split testing","readme_language","vid","This cookie name 'vid' is used for tracking the search request made by the user on the 'Find a practitioner' page.","at_check","Used by Adobe Target to check if cookies are enabled/supported on the browser","blogs.nvidia.com","nvHeadless","This cookie is likely used to track if a user is using the site in a 'headless' non-GUI mode, which could be for accessibility reasons or for automated processes.","C0002","Performance Cookies","These cookies are used to provide quantitative measures of our website visitors, such as the number of times you visit, time on page, your mouse movements, scrolling, clicks and keystroke activity on the websites; other browsing, search, or product research behavior; and what brought you to our site. These cookies may store a unique ID so that our system will remember you when you return. Information collected with these cookies is used to measure and find ways to improve website performance.",[641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669],[621,622,623,624],{"_366":636,"_211":636,"_179":368,"_339":637},{"_366":631,"_211":631,"_179":368,"_339":632},{"_366":429,"_211":429,"_179":368,"_339":628},{"_366":543,"_211":543,"_179":368,"_339":625},[626],{"_79":627,"_373":543,"_374":89,"_179":547},"_gh_sess",[629],{"_79":630,"_373":429,"_374":89,"_179":436},"YSC","bizible.com",[633],{"_79":634,"_373":631,"_374":35,"_179":635},"_BUID","Bizible is a marketing attribution software that tracks the performance of marketing efforts and connects them to revenue, primarily used for B2B environments.","a.usbrowserspeed.com",[638],{"_79":639,"_373":636,"_374":35,"_179":640},"tuid","This domain is owned by Ookla which provides internet speed testing services through its Speedtest application.",{"_79":711,"_373":488,"_374":35,"_179":712},{"_79":696,"_373":710,"_374":89,"_179":697},{"_79":679,"_373":491,"_374":35,"_179":604},{"_79":679,"_373":592,"_374":35,"_179":604},{"_79":708,"_373":488,"_374":35,"_179":709},{"_79":696,"_373":494,"_374":89,"_179":697},{"_79":706,"_373":488,"_374":35,"_179":707},{"_79":696,"_373":491,"_374":89,"_179":697},{"_79":704,"_373":591,"_374":35,"_179":705},{"_79":696,"_373":613,"_374":89,"_179":697},{"_79":676,"_373":491,"_374":35,"_179":678},{"_79":702,"_373":488,"_374":89,"_179":703},{"_79":700,"_373":488,"_374":35,"_179":701},{"_79":698,"_373":488,"_374":89,"_179":699},{"_79":696,"_373":587,"_374":89,"_179":697},{"_79":694,"_373":488,"_374":35,"_179":695},{"_79":692,"_373":488,"_374":35,"_179":693},{"_79":690,"_373":488,"_374":89,"_179":691},{"_79":688,"_373":596,"_374":35,"_179":689},{"_79":676,"_373":587,"_374":35,"_179":678},{"_79":686,"_373":488,"_374":35,"_179":687},{"_79":684,"_373":488,"_374":89,"_179":685},{"_79":682,"_373":488,"_374":35,"_179":683},{"_79":680,"_373":488,"_374":35,"_179":681},{"_79":679,"_373":488,"_374":35,"_179":604},{"_79":676,"_373":677,"_374":35,"_179":678},{"_79":674,"_373":488,"_374":35,"_179":675},{"_79":672,"_373":488,"_374":89,"_179":673},{"_79":670,"_373":488,"_374":35,"_179":671},"_ga_xxxxxxxxxx","Used by Google Analytics to identify and track an individual session with your device.","s_ppvl","This cookie is set by Adobe Analytics to store information on the percentage of the page displayed to the end user.","coveo_visitorId","This cookie name is associated with Coveo which is used to store a unique user ID for marketing purposes and to track the number of distinct users who submitted a query or clicked an item, counting them only once even if they did the same query or clicked the same item over the course of many visits.","_dd_s","marketplace.nvidia.com","Datadog Cookie used to group all events generated from a unique user session across multiple pages. It contains the current session ID, whether the session is excluded due to sampling, and the expiration date of the session. The cookie is extended for an extra 15 minutes every time the user interacts with the website, up to the maximum user session duration (4 hours).","nvweb_A","_ga","This cookie name is associated with Google Universal Analytics - which is a significant update to Google's more commonly used analytics service. This cookie is used to distinguish unique users by assigning a randomly generated number as a client identifier. It is included in each page request in a site and used to calculate visitor, session and campaign data for the sites analytics reports. By default it is set to expire after 2 years, although this is customisable by website owners.","_uetvid","This is a cookie utilised by Microsoft Bing Ads and is a tracking cookie. It allows us to engage with a user that has previously visited our website.","tp","This cookie is associated with Adobe Analytics site tracking. It registers if the website's scroll-depth detection is active. This function remembers how far the user has scrolled on the current session across the website's sub-pages.","_hly_vid","This cookie _hly_vid is used to record website and server performance with each visit to the site. The data is aggregated and analyzed; therefore, it remains anonymous upon use.","_session_id","This cookie creates an interim session ID used as an in-session user ID.\n","s_cc","This cookie is associated with the Adobe Site Catalyst. It determines whether cookies are enabled in the web browser.","s_ecid","Adobe Analytics cookie containing the the Adobe Experience Cloud 1st party visitor id if the Analytics AMCV visitor id is not found.","_hjSessionUser_xxxxxx","Hotjar cookie that is set when a user first lands on a page with the Hotjar script. It is used to persist the Hotjar User ID, unique to that site on the browser. This ensures that behavior in subsequent visits to the same site will be attributed to the same user ID.","_hjHasCachedUserAttributes","This cookie name is associated with Hotjar which is used to know whether the data set in _hjUserAttributes Local Storage item is up to date or not.","s_ppv","This cookie is associated with Adobe Analytics site tracking and is used to measure a user's scroll activity to see how much of a page they view before moving to another page.","_gid","This cookie name is associated with Google Universal Analytics. This appears to be a new cookie and as of Spring 2017 no information is available from Google. It appears to store and update a unique value for each page visited.","_hjTLDTest","When the Hotjar script executes we try to determine the most generic cookie path we should use, instead of the page hostname. This is done so that cookies can be shared across subdomains (where applicable). To determine this, we try to store the _hjTLDTest cookie for different URL substring alternatives until it fails. After this check, the cookie is removed.","ekfls-2025-03-27","This cookie name is associated with readme.io which is used to help maintain data for the API reference support page.","s_getNewRepeat","This cookie is used to determine whether a visitor is a new visitor or a repeat visitor to the site, and is used for analytical purposes.","_hly_sid","This cookie is likely associated with website analytics, as the prefix '_hly_' suggests a connection to Holistics, a business intelligence and analytics platform.","docs.nvidia.com","_hjSession_xxxxxx","A cookie that holds the current session data. This ensues that subsequent requests within the session window will be attributed to the same Hotjar session.","C0001","Required Cookies","These cookies enable core functionality such as security, network management, and accessibility. These cookies are required for the site to function and cannot be turned off.",[858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930],[718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742],{"_366":855,"_211":855,"_179":368,"_339":856},{"_366":849,"_211":849,"_179":368,"_339":850},{"_366":846,"_211":846,"_179":368,"_339":847},{"_366":834,"_211":834,"_179":368,"_339":835},{"_366":831,"_211":831,"_179":368,"_339":832},{"_366":828,"_211":828,"_179":368,"_339":829},{"_366":823,"_211":823,"_179":368,"_339":824},{"_366":429,"_211":429,"_179":368,"_339":820},{"_366":817,"_211":817,"_179":368,"_339":818},{"_366":421,"_211":421,"_179":368,"_339":815},{"_366":416,"_211":416,"_179":368,"_339":813},{"_366":808,"_211":808,"_179":368,"_339":809},{"_366":803,"_211":803,"_179":368,"_339":804},{"_366":798,"_211":798,"_179":368,"_339":799},{"_366":795,"_211":795,"_179":368,"_339":796},{"_366":543,"_211":543,"_179":368,"_339":792},{"_366":401,"_211":401,"_179":368,"_339":789},{"_366":538,"_211":538,"_179":368,"_339":782},{"_366":534,"_211":534,"_179":368,"_339":776},{"_366":770,"_211":770,"_179":368,"_339":771},{"_366":765,"_211":765,"_179":368,"_339":766},{"_366":760,"_211":760,"_179":368,"_339":761},{"_366":756,"_211":756,"_179":368,"_339":757},{"_366":751,"_211":751,"_179":368,"_339":752},{"_366":743,"_211":743,"_179":368,"_339":744},"polyfill.io",[745,746],{"_79":749,"_373":743,"_374":35,"_179":750},{"_79":747,"_373":743,"_374":35,"_179":748},"cf_ob_info","This cookie is associated with sites using CloudFlare. It is used to improve page load times and to override any security restrictions based on the visitors IP address. It does not contain any user identification information.","cf_use_ob","This cookie is associated with sites using CloudFlare. It is used to improve page load times and to override any security restrictions based on the visitors IP address. It does not contain any user identification information. ","www.recaptcha.net",[753],{"_79":754,"_373":751,"_374":35,"_179":755},"_GRECAPTCHA","This domain is used by Google reCAPTCHA, which provides spam protection services to websites by detecting if a user is a human or a bot.","api2.hcaptcha.com",[758],{"_79":759,"_373":756,"_374":35,"_179":410},"__cflb","instagram.com",[762],{"_79":763,"_373":760,"_374":35,"_179":764},"csrftoken","Instagram is a social networking service owned by Facebook, Inc. It's designed for sharing photos and videos from a smartphone. The service also uses cookies for various purposes, including analytics, personalization, and advertising.","readme.io",[767],{"_79":768,"_373":765,"_374":35,"_179":769},"__cf_bm","The __cf_bm cookie is a cookie necessary to support Cloudflare Bot Management, currently in private beta. As part of our bot management service, this cookie helps manage incoming traffic that matches criteria associated with bots.","nvidiacorp.us-5.evergage.com",[772,773],{"_79":775,"_373":770,"_374":35,"_179":410},{"_79":774,"_373":770,"_374":35,"_179":410},"AWSALBTG","AWSALBTGCORS",[777,778],{"_79":780,"_373":534,"_374":35,"_179":781},{"_79":779,"_373":534,"_374":35,"_179":410},"ak_bmsc","bm_mi","This domain is likely associated with NVIDIA's partner program, which could involve marketing, sales, and support services for partners.",[783,784,785],{"_79":788,"_373":538,"_374":35,"_179":410},{"_79":787,"_373":538,"_374":89,"_179":410},{"_79":786,"_373":538,"_374":35,"_179":410},"AWSALBCORS","JSESSIONID","AWSALB",[790],{"_79":768,"_373":401,"_374":35,"_179":791},"This is a CloudFoundry cookie",[793],{"_79":794,"_373":543,"_374":35,"_179":547},"logged_in","hcaptcha.com",[797],{"_79":768,"_373":795,"_374":35,"_179":791},"tracking.intentsify.io",[800],{"_79":801,"_373":798,"_374":35,"_179":802},"userId","Used to track usage of the embedded video player.","static.reo.dev",[805],{"_79":806,"_373":803,"_374":35,"_179":807},"cookietest","Common cookie name could have a number of different origins. Where this is first party and a session cookie, its most likely to do with checking to see if the browser is set to block or allow cookies.","www.datadoghq-browser-agent.com",[810],{"_79":811,"_373":808,"_374":35,"_179":812},"dd_cookie_test_","dd_cookie_test",[814],{"_79":768,"_373":416,"_374":35,"_179":791},[816],{"_79":768,"_373":421,"_374":35,"_179":791},"supabase.co",[819],{"_79":768,"_373":817,"_374":35,"_179":791},[821],{"_79":822,"_373":429,"_374":35,"_179":436},"VISITOR_PRIVACY_METADATA","dosa-prd.azurewebsites.net",[825],{"_79":826,"_373":823,"_374":35,"_179":827},"access_token","This cookie is used to obtain the permissions for users of the portal to allow for API access in order to retrieve information required to perform portal functions.","w.hcaptcha.com",[830],{"_79":768,"_373":828,"_374":35,"_179":791},"cdn.readme.io",[833],{"_79":768,"_373":831,"_374":35,"_179":791},"queue-it.net",[836,837,838,839,840],{"_79":845,"_373":834,"_374":35,"_179":368},{"_79":844,"_373":834,"_374":35,"_179":368},{"_79":843,"_373":834,"_374":35,"_179":368},{"_79":842,"_373":834,"_374":35,"_179":368},{"_79":841,"_373":834,"_374":35,"_179":368},"Queue-it-{queueId}","Queue-it-{dmonvidia }________{eventId}","Queue-it-token","queueitsoftblock_{queueId}","Queue-it-{nvidia}________{eventId}","x.com",[848],{"_79":768,"_373":846,"_374":35,"_179":769},"api.hcaptcha.com",[851,852],{"_79":854,"_373":849,"_374":35,"_179":410},{"_79":759,"_373":849,"_374":35,"_179":853},"This domain is associated with hCaptcha, a service that provides CAPTCHA and reCAPTCHA alternatives for user verification and bot protection.","hmt_id","vimeo.com",[857],{"_79":768,"_373":855,"_374":35,"_179":791},{"_79":991,"_373":488,"_374":35,"_179":1022},{"_79":1020,"_373":584,"_374":89,"_179":1021},{"_79":1013,"_373":982,"_374":89,"_179":1014},{"_79":1019,"_373":677,"_374":89,"_179":368},{"_79":1017,"_373":488,"_374":35,"_179":1018},{"_79":775,"_373":1003,"_374":35,"_179":942},{"_79":950,"_373":1012,"_374":89,"_179":952},{"_79":774,"_373":941,"_374":35,"_179":942},{"_79":953,"_373":1012,"_374":35,"_179":954},{"_79":1015,"_373":488,"_374":89,"_179":1016},{"_79":1013,"_373":989,"_374":89,"_179":1014},{"_79":947,"_373":1012,"_374":35,"_179":949},{"_79":1010,"_373":488,"_374":35,"_179":1011},{"_79":774,"_373":933,"_374":35,"_179":942},{"_79":938,"_373":1009,"_374":35,"_179":940},{"_79":786,"_373":939,"_374":35,"_179":955},{"_79":788,"_373":939,"_374":35,"_179":1006},{"_79":953,"_373":948,"_374":35,"_179":954},{"_79":1007,"_373":587,"_374":35,"_179":1008},{"_79":788,"_373":613,"_374":35,"_179":1006},{"_79":1004,"_373":488,"_374":35,"_179":1005},{"_79":780,"_373":613,"_374":35,"_179":999},{"_79":976,"_373":592,"_374":89,"_179":368},{"_79":774,"_373":1003,"_374":35,"_179":942},{"_79":1001,"_373":941,"_374":89,"_179":1002},{"_79":965,"_373":488,"_374":35,"_179":956},{"_79":938,"_373":488,"_374":35,"_179":940},{"_79":1000,"_373":500,"_374":35,"_179":368},{"_79":780,"_373":488,"_374":35,"_179":999},{"_79":788,"_373":933,"_374":35,"_179":998},{"_79":996,"_373":596,"_374":89,"_179":997},{"_79":994,"_373":488,"_374":89,"_179":995},{"_79":993,"_373":488,"_374":35,"_179":368},{"_79":991,"_373":939,"_374":35,"_179":992},{"_79":990,"_373":500,"_374":35,"_179":368},{"_79":981,"_373":989,"_374":89,"_179":983},{"_79":987,"_373":592,"_374":35,"_179":988},{"_79":985,"_373":677,"_374":35,"_179":986},{"_79":984,"_373":494,"_374":35,"_179":368},{"_79":981,"_373":982,"_374":89,"_179":983},{"_79":979,"_373":948,"_374":89,"_179":980},{"_79":977,"_373":488,"_374":35,"_179":978},{"_79":976,"_373":491,"_374":89,"_179":368},{"_79":947,"_373":951,"_374":35,"_179":949},{"_79":779,"_373":966,"_374":35,"_179":956},{"_79":974,"_373":491,"_374":35,"_179":975},{"_79":972,"_373":488,"_374":35,"_179":973},{"_79":970,"_373":488,"_374":35,"_179":971},{"_79":969,"_373":488,"_374":35,"_179":368},{"_79":811,"_373":491,"_374":35,"_179":968},{"_79":965,"_373":613,"_374":35,"_179":956},{"_79":768,"_373":584,"_374":35,"_179":967},{"_79":965,"_373":966,"_374":35,"_179":956},{"_79":964,"_373":500,"_374":35,"_179":368},{"_79":963,"_373":488,"_374":35,"_179":936},{"_79":961,"_373":948,"_374":35,"_179":962},{"_79":774,"_373":959,"_374":35,"_179":960},{"_79":958,"_373":500,"_374":35,"_179":368},{"_79":957,"_373":488,"_374":35,"_179":368},{"_79":950,"_373":948,"_374":89,"_179":952},{"_79":779,"_373":613,"_374":35,"_179":956},{"_79":786,"_373":613,"_374":35,"_179":955},{"_79":953,"_373":951,"_374":35,"_179":954},{"_79":950,"_373":951,"_374":89,"_179":952},{"_79":947,"_373":948,"_374":35,"_179":949},{"_79":945,"_373":941,"_374":89,"_179":946},{"_79":943,"_373":599,"_374":35,"_179":944},{"_79":775,"_373":941,"_374":35,"_179":942},{"_79":938,"_373":939,"_374":35,"_179":940},{"_79":768,"_373":591,"_374":35,"_179":937},{"_79":935,"_373":488,"_374":35,"_179":936},{"_79":786,"_373":933,"_374":35,"_179":934},{"_79":931,"_373":488,"_374":35,"_179":932},"LoginHints","Keeps users logged in across NVIDIA domains.","api.ngc.nvidia.com","This cookie is managed by AWS and is used for load balancing.","bm_so","This cookie name is used to load the site as quickly as possible and to recognize bots using Akamai.","The __cf_bm cookie is a cookie necessary to support Cloudflare Bot Management, currently in private beta. As part of our bot management service, this cookie helps manage incoming traffic that matches criteria associated with bots.\nThis is a CloudFoundry cookie","OptanonConsent","nvid.nvidia.com","This cookie is set by the cookie compliance solution from OneTrust. It stores information about the categories of cookies the site uses and whether visitors have given or withdrawn consent for the use of each category. This enables site owners to prevent cookies in each category from being set in the users browser, when consent is not given. The cookie has a normal lifespan of one year, so that returning visitors to the site will have their preferences remembered. It contains no information that can identify the site visitor.","events.nvidia.com","This cookie is associated with the Amazon Web Services Elastic Load Balancing functionality for routing client request on the server.Used to attribute commission to affiliates when you arrive at the website from an affiliate referral link. It is set when you click on one of our links and used to let the advertiser and us know the website from which you came.","sessionid","This is a very generic cookie name that may have different purposes on different sites, but generally it will be some kind of anonymous session identifier.","_csrf","This cookie is used to prevent Cross-Site Request Forgery (CSRF) attacks, ensuring that each request made to the server is from the user's browser and is intentional.","LSKey-c$CookieConsentPolicy","enterprise-support.nvidia.com","Used to apply end-user cookie consent preferences set by our client-side utility.Custom- Short Duration. Unique ID of an Enquiry- used to validate access to case, and load exisiting data.\n","renderCtx","academicgrants.nvidia.com","Used to store site parameters in the session for reuse across requests by a single client for functionality and performance reasons.","CookieConsentPolicy","Used to apply end-user cookie consent preferences set by our client-side utility.","This cookie is managed by AWS and is used for load balancing.Used to attribute commission to affiliates when you arrive at the website from an affiliate referral link. It is set when you click on one of our links and used to let the advertiser and us know the website from which you came.","This cookie is associated with Akamai and is used to differentiate between traffic from humans and bots.","nv_co","__uxq412__ses.88c1","learn.learn.nvidia.com","This cookie is associated with the Amazon Web Services Elastic Load Balancing functionality for routing client request on the server.","pctrk","Used to count page views by unauthenticated users against license usage.","bm_s","bpazaws52gukakzc__ctrl0_ctl36_uccaptcha","bm_sv","store.nvidia.com","The __cf_bm cookie is a cookie necessary to support Cloudflare Bot Management, currently in private beta. As part of our bot management service, this cookie helps manage incoming traffic that matches criteria associated with bots.\n","This cookie name is used by DataDog to determine if cookies are supported on the website.dd_cookie_test","AMP_TLDTEST_xxxxxxxx","SESSlsd","This cookie is likely to be a session identifier used to maintain user session state across page requests.","mboxEdgeCluster","This cookie name is associated with Adobe Target which is used to communicate the correct edge cluster to at.js so that Target profiles can stay in sync as users navigate across a site.","_rdt_em","This cookie name is associated with Reddit which is used to manage session state and user interactions.","megaNavCollapsed","_abck","The _abck cookie is associated with Akamai's Bot Manager, which is used to prevent fraud and abuse by safeguarding online interactions and detecting irregularities that could suggest malicious activity, such as bot-driven attacks or hacking attempts.","idccsrf","This cookie name is associated with Salesforce which is used to track CrossSiteRequestForgery validation for certain single sign-on (SSO) flows.","ApplicationGatewayAffinity","partners.nvidia.com","This cookie is used to maintain a user session by the server in a load balanced environment, to ensure the user has a consistent experience.","AMP_TEST_xxxxxxxx","bm_lso","This cookie name is associated with Akamai which is used to load the site as quickly as possible and to recognize bots.","SSESS###########","This cookie is likely to be a session cookie used to maintain an anonymized user session by the server in a secure environment, often seen in sites built with the Drupal content management system.","nvdeveloper.nvidia.com","__uxq412__id.88c1","OptanonAlertBoxClosed","This cookie is set by websites using certain versions of the cookie law compliance solution from OneTrust. It is set after visitors have seen a cookie information notice and in some cases only when they actively close the notice down. It enables the website not to show the message more than once to a user. The cookie has a normal lifespan of one year and contains no personal information.","QueueITAccepted-SDFrts345E-V3_{eventId}","TEST_AMCV_COOKIE_WRITE","This cookie is used to test whether cookies are enabled in the browser. It is a temporary cookie used to determine if the browser accepts cookies.","XSRF-TOKEN","This cookie is written to help with site security in preventing Cross-Site Request Forgery attacks.This cookie helps with site security in preventing Cross-Site Request Forgery attacks.","AWS ELB application load balancer","This cookie is associated with Akamai. It is used to analyse traffic to determine if it is automated traffic generated by bots or a human user.","bpazaws52gukakzc__ctrl0_ctl78_list_ctl09_systemcaptcha","PHPSESSID","PHP session cookie associated with embedded content from this domain.\n","base.manager.learn.nvidia.com","bm_ss","This cookie is used to help protect websites from automated bot traffic.","AWS ELB application load balancerUsed to attribute commission to affiliates when you arrive at the website from an affiliate referral link. It is set when you click on one of our links and used to let the advertiser and us know the website from which you came.","_devzone_session","This cookie is likely used to maintain a user session on the website by the server. It is common for session identifiers to be used to keep users logged in and track their interaction with the site during a session.","docs.ngc.nvidia.com","bm_sz","This cookie is used by Akamai to optimize site performance and security. It is typically associated with the delivery of web content and services.","vcalliance.nvidia.com","ASP.NET_SessionId","General purpose platform session cookie, used by sites written with Microsoft .NET based technologies. Usually used to maintain an anonymised user session by the server.","com.adobe.alloy.getTld","This cookie name is associated with the Adobe WebSDK which is used to determine the top-most domain that can accept cookies.","SESSls","This cookie is likely to be a session identifier, used to maintain a user's session state across page requests.","c_code","BIGipServerx","This cookie name is associated with the BIG-IP product suite from company F5. Usually associated with managing sessions on load balanced servers, to ensure user requests are routed consistently to the correct server. The common root is BIGipServer most commonly followed by a domain name, usually the one that it is hosted on, but not always.","This cookie is set by websites using certain versions of the cookie law compliance solution from OneTrust. It is set after visitors have seen a cookie information notice and in some cases only when they actively close the notice down. It enables the website not to show the message more than once to a user. The cookie has a one year lifespan and contains no personal information.","default","

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking \"Accept All\", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on \"Manage Settings.\" By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.

","gpc","

We have detected the Global Privacy Control (GPC) signal and recorded your rejection of all optional cookies on this site for this browser. You can manage your cookie settings by clicking on \"Manage Settings\". Please see our Cookie Policy for more information. To opt out of non-cookie personal information \"sales\" / \"sharing\" for targeted advertising purposes, please visit the NVIDIA Preference Center. Please see our Privacy Policy for more information on our privacy practices.

","gpcLoggedIn","

We have detected the Global Privacy Control Signal (GPC) and have opted you out of all optional cookies on this browser. You can manage your cookie settings by clicking on \"Manage Settings\". Please see our Cookie Policy for more information. We have also opted you out of \"sharing\"/\"sales\" of personal information outside of cookies. You can manage these settings in the NVIDIA NVIDIA Preference Center. Please see our Privacy Policy for more information.

","gpcLoggedInOverridden","

We have detected the Global Privacy Control Signal (GPC) and have opted you out of all optional cookies on this browser. You can manage your cookie settings by clicking on \"Manage Settings\". Please see our Cookie Policy for more information. We have also opted you out of \"sharing\"/\"sales\" of personal information outside of cookies which overrides at least one of your previous settings. You can manage them in the NVIDIA Preference Center. Please see our Privacy Policy for more information.

",{"_713":18,"_616":60,"_523":60,"_343":60,"_333":60},"hasConsented","gpcApplied","showBanner","geolocation",{"_1037":1038,"_1039":1040},"country","US","state","OR","query","categories",[1060,1061,1062,1063,1064],"optionUrls",[1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059],"total",12,"/orgs/nim/deepseek-ai/containers/deepseek-r1/-","/orgs/nim/nvidia/containers/llama-3.1-nemotron-70b-instruct/-","/orgs/nvidia/-/containers/pytorch/-","/orgs/nvidia/omniverse/collections/kit/-","/orgs/nvidia/-/collections/deepstream_sdk/-","/orgs/nvidia/omniverse/collections/kit-appstreaming-collection/-","/orgs/nvidia/research/models/stylegan3/-","/orgs/nvidia/tao/models/peoplenet/-","/orgs/nvidia/tao/models/trafficcamnet/-","/orgs/nvidia/riva/resources/riva_quickstart/-","/orgs/nvidia/-/helm-charts/gpu-operator/-","/orgs/nvidia/-/helm-charts/k8s-nim-operator/-",{"_21":1137,"_1065":1138,"_1067":60},{"_21":1102,"_1065":1103,"_1067":58},{"_21":1085,"_1065":1086,"_1067":1087},{"_21":1077,"_1065":1078,"_1067":1079},{"_21":22,"_1065":1066,"_1067":20},"items",[1068,1069],"startIndex",{"_21":22,"_211":1074,"_173":174,"_175":-7,"_206":1075,"_214":215,"_239":1076,"_1073":1058},{"_21":22,"_211":1070,"_173":174,"_175":-7,"_206":1071,"_214":215,"_239":1072,"_1073":1059},"NVIDIA NIM Operator","k8s-nim-operator",[],"url","GPU Operator","gpu-operator",[],"resources",[1080],9,{"_21":1077,"_211":1081,"_173":174,"_175":1082,"_206":1083,"_214":215,"_239":1084,"_1073":1057},"Riva Skills Quick Start","riva","riva_quickstart",[],"models",[1088,1089,1090],6,{"_21":1085,"_211":1098,"_173":174,"_175":1099,"_206":1100,"_214":215,"_239":1101,"_1073":1054},{"_21":1085,"_211":1095,"_173":174,"_175":1092,"_206":1096,"_214":215,"_239":1097,"_1073":1055},{"_21":1085,"_211":1091,"_173":174,"_175":1092,"_206":1093,"_214":215,"_239":1094,"_1073":1056},"TrafficCamNet","tao","trafficcamnet",[],"PeopleNet","peoplenet",[],"StyleGAN3 pretrained models","research","stylegan3",[],"collections",[1104,1105,1106],{"_21":1102,"_211":1134,"_173":174,"_175":1108,"_206":1135,"_214":215,"_239":1136,"_1073":1051},{"_21":1102,"_211":1131,"_173":174,"_175":-7,"_206":1132,"_214":215,"_239":1133,"_1073":1052},{"_21":1102,"_211":1107,"_173":174,"_175":1108,"_206":1109,"_214":215,"_239":1110,"_1073":1053},"Omniverse Kit App Streaming","omniverse","kit-appstreaming-collection",[1111,1112,1113],{"_79":1125,"_211":1126,"_208":1127,"_218":1128,"_1118":1129},{"_79":1108,"_211":1121,"_208":1122,"_218":1117,"_1118":1123},{"_79":1114,"_211":1115,"_208":1116,"_218":1117,"_1118":1119},"omniverse-dev","NVIDIA Developer Program","Join the Developer Program for access to free tools, support, and tech resources.","https://store-images.s-microsoft.com/image/apps.33936.6f1bcf42-55d0-45d1-b4b6-f08c68a50bbe.e91ca707-db64-49d6-9761-4781590a9685.7ba64fc5-7c44-4864-a5e6-7bde05126935","supportedFeatures",[1120],"UNIFIED_CATALOG","NVIDIA Omniverse","NVIDIA Omniverse is a platform of APIs, services, and software development kits (SDKs) that enable developers to build generative AI-enabled tools and applications.",[1120,1124],"SHAREABLE_TO_USER_GROUPS","nv-ai-enterprise","NVIDIA AI Enterprise","Accelerate your AI agent development","https://assets.ngc.nvidia.com/products/catalog/images/product-NVAIE.jpg",[1120,1124,1130],"NGC_CONSOLE","DeepStream SDK","deepstream_sdk",[],"Omniverse Kit (FB)","kit",[1111,1112,1113],"containers",[1139,1140,1141],{"_21":1137,"_211":1149,"_173":1146,"_175":1150,"_206":1151,"_214":215,"_239":1152,"_1073":1048},{"_21":1137,"_211":1145,"_173":1146,"_175":174,"_206":1147,"_214":215,"_239":1148,"_1073":1049},{"_21":1137,"_211":1142,"_173":174,"_175":-7,"_206":1143,"_214":215,"_239":1144,"_1073":1050},"PyTorch","pytorch",[],"Llama-3.1-Nemotron-70B-Instruct","nim","llama-3.1-nemotron-70b-instruct",[],"DeepSeek-R1","deepseek-ai","deepseek-r1",[],"ownerOrgs",[],"memberOrgs",[],"externalOrgs",[],"apiKey","2b44f86fea50e90ffd2db29ee2028419","consentGranted","applicationId","4c857e4f-5ffa-45fa-b75e-af773c22f0ad","clientToken","pubb6927db33dae60ba5031a186cde0242a","site","datadoghq.com","service","catalog-ui","env","sessionSampleRate",100,"sessionReplaySampleRate","compressIntakeRequests","storeContextsAcrossPages","defaultPrivacyLevel","mask-user-input","trackViewsManually","trackingConsent","not-granted",{"_1182":1183,"_1184":280,"_1185":35,"_1186":1187,"_1065":1188,"_1189":1190,"_1191":1183,"_1192":35,"_1193":35,"_1194":35,"_1195":1196},"appName","ngc-catalog","appOrigin","hideOnPrivateOrg","icon","ObjectsModules2",[1197,1198,1199,1200,1201,1202,1203],"menuIcon","editor-swatches","productName","requiresAllScopes","requiresLogin","requiresOrgLevelScopesOnly","title","Catalog",{"_1195":1225,"_51":1226,"_1189":1227},{"_1218":1219,"_1195":1220,"_51":1221,"_1191":1125},{"_1195":1216,"_51":1217,"_1189":1190},{"_1195":1213,"_51":1214,"_1189":1215},{"_1195":1210,"_51":1211,"_1189":1212},{"_1195":1207,"_51":1208,"_1189":1209},{"_1195":1204,"_51":1205,"_1189":1206},"Resources","/resources","misc-library","Models","/models","misc-neural-network","Helm Charts","/helm-charts","misc-ship-wheel","Containers","/containers","3d-cube","Collections","/collections","access",[1222],"Enterprise Catalog","/enterprise",[1223,1224],"NVAIE","READ","Explore Catalog","/","common-home",{"_126":127,"_128":129,"_130":131,"_132":133,"_134":135,"_136":137},{"_125":35}]

nemo-retriever Helm chart

Layout

Quick start

Persistent storage prerequisite { #persistent-storage-prerequisite }

GPU scheduling prerequisite { #gpu-scheduling-prerequisite }

1. Service image { #1-service-image }

2. Install with external NIM endpoints (operator not required)

3. Install with the NIM Operator (in-cluster NIMs)

Recommended minimal install (26.08.1) { #recommended-minimal-install-26081 }

Query reranking (optional)

Why NIM resources still exist after helm uninstall

Values reference (highlights)

Service

Audio and video (Parakeet ASR) { #audio-video-parakeet }

Health probes

Service networking

Service configuration (rendered into retriever-service.yaml)

Sidecar metadata in split topology

VectorDB and the embed endpoint { #vectordb-and-the-embed-endpoint }

Answer generation (operator-managed LLM) { #answer-generation-llm }

Agentic retrieval (self-hosted Super-49B) { #agentic-retrieval-llm }

NIM Operator sub-stack

Filtering cached GPU profiles { #filtering-cached-gpu-profiles }

Image tag conventions { #image-tag-conventions }

Changing a NIM image repository or tag { #changing-nim-image-repository-or-tag }

Query-time reranking { #query-time-reranking }

Image captioning (Omni 30B) { #image-captioning-omni-30b }

GPU limits and helm upgrade { #gpu-limits-and-helm-upgrade }

OCR NIM configuration { #ocr-nim-configuration }

Persistence

Scheduler loss boundary and upgrade from durable releases

Secrets

Optional features

Configuration recipes

Mount a custom retriever-service.yaml verbatim

Use externally managed Secrets

Disable one NIM and supply an external URL for it

Roll the service after editing values

Timeouts and alleviating ingest failures

What the chart configures

Recommended mitigations

Queue-depth autoscaling (split mode)

Why queue depth (and not CPU)

Backend choices

Wiring up prometheus-adapter (recommended)

CPU fallback (no Prometheus required)

KEDA path

Tuning the thresholds

Verifying it scales

Tracing and Zipkin

Prometheus metrics from the OpenTelemetry Collector

OpenShift deployment { #openshift-deployment }

Air-gapped deployment { #air-gapped-deployment }

Container images to mirror (chart defaults)

Helm values for a private registry

Mirroring pattern

Roadmap

Validation

Why NIM resources still exist after `helm uninstall`

Service configuration (rendered into `retriever-service.yaml`)

GPU limits and `helm upgrade` { #gpu-limits-and-helm-upgrade }