NVCF LLM API Gateway

The NVCF LLM API Gateway provides an OpenAI-compatible entry point for large language model workloads running on NVIDIA Cloud Functions Self-Managed.

Features

• Accepts supported OpenAI-compatible inference requests.
• Integrates authentication, request handling, and LLM routing with NVCF control-plane services.
• Supports streaming inference workflows in Kubernetes-based self-managed deployments.

Deployment and Support

Deploy this container through the NVCF Self-Managed stack and its LLM API Gateway Helm chart. See the NVIDIA Cloud Functions documentation. Support is available through the NVIDIA Developer Forums.

License

Use of this software is governed by the NVIDIA Cloud Agreement and applicable third-party notices.