USD Search API EKS Cluster Terraform Module

This Terraform module provisions a fully-configured Amazon Elastic Kubernetes Service (EKS) cluster with supporting infrastructure, ready for deploying USD Search API.

Prerequisites

Terraform v1.5.7 or higher
AWS Account with sufficient quota for EKS, VPC and EC2 instances (CPU and GPU)
AWS CLI configured with appropriate access
An existing Route 53 hosted zone for DNS management

Features

EKS Cluster: Provisions an EKS cluster with managed node groups.
VPC Configuration: Creates a custom VPC with public and private subnets, NAT gateways, and necessary routing.
Deploys Essential Kubernetes Applications and resources:
- NGINX Ingress Controller: Manages inbound traffic to Kubernetes services.
- Prometheus Stack: Provides monitoring and alerting. Compatible with scrape configuration that is shipped with USD Search API.
- Grafana: Integrated with Prometheus for visualizing metrics and data. Compatible with dashboards that are shipped with USD Search API.
- Karpenter: Automatically scales cluster based on workload demands, with support for both general-purpose and GPU-accelerated nodes.
- NVIDIA GPU Operator: Manages GPU resources and provides GPU drivers and monitoring.
- AWS EBS and EFS CSI Drivers: Enables dynamic storage provisioning using AWS EBS and EFS.
- Kubernetes Priority Class: Configures priority classes for workloads.

Download

Generate your NGC helm and container registry API Key. See onboarding guide here. This step could be skipped if the API Key was generated before.

Fetch the latest resource from the NGC registry by using the Download button in the top right corner of the page and selecting the preferred download method. Below you can find a sample command that uses NGC CLI for downloading:

ngc registry resource download-version "nvidia/usdsearch/usdsearch_terraform:1.0.0"

Usage example

Inside the downloaded resource - create a new file main.tf in the root of your project and add the following content:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  // Use S3 as a backend for storing Terraform state.
  // For local development, you can comment out this block and use the default local backend.
  backend "s3" {
    bucket = "my-usd-search-tfstate"  
    key    = "usdsearch"
    region = "us-east-1"
  }
}

provider "aws" {
  region = "us-east-2"
}

module "usdsearch_cluster" {
  source = "./usdsearch_terraform"
  
  cluster_name                     = "my-usd-search-cluster"
  environment                      = "dev"
  azs                              = ["us-east-2a", "us-east-2b", "us-east-2c"]
  default_node_group_capacity_type = "SPOT"
  // Set the number of GPUs available on the cluster
  renderer_node_pool_gpu_limit     = 4
  // Optional: Create DNS record for the ingress controller. By default it creates a wildcard record for `route53_record_name` to later make USD Search API accessible on subdomains (via ingress, configured in the helm chart). If such functionality is not desired, please set ``create_route53_record`` to ``false``.
  create_route53_record            = true
  route53_zone_id                  = "your-route53-zone-id"
  route53_record_name              = "my-usd-search-cluster.example.com"
  // Optional: Enable Grafana Ingress. Note that it uses default grafana credentials. For production deployments, it is required to restrict access to Grafana.
  enable_grafana_ingress           = true
  grafana_ingress_host             = "grafana.my-usd-search-cluster.example.com"
}

Update the configuration according to your desired settings. Please refer to usdsearch_terraform/variables.tf for more information about the variables.
Initialize terraform modules by running:

terraform init

Provision the infrastructure by running:

terraform apply

Setup kubectl for use with the new cluster:

aws eks update-kubeconfig --region us-east-2 --name my-usd-search-cluster