DOCA HBN

Introduction

The host-based networking (HBN) solution packages a set of network functions inside a container which, itself, is packaged as a service pod to be run on the DPU.

At the core of HBN is the Linux networking DPU acceleration driver. Netlink to DOCA daemon, or nl2docad, implements the DPU acceleration driver. nl2docad seamlessly accelerates Linux networking using DPU hardware programming APIs.

The driver mirrors the Linux kernel routing and bridging tables into the DPU hardware by discovering the configured Linux networking objects using the Linux Netlink API. Dynamic network flows, as learned by the Linux kernel networking stack, are also programmed by the driver into DPU hardware by listening to Linux kernel networking events.

Installation and Getting Started

All preparation steps are listed under DOCA's Container Deployment User Guide.

Note: The DOCA Service container is configured for K8S-based deployment, hence the use of the docker pull command is discouraged.

Preparation steps for the DOCA Service

HBN Preparation Script

The preparation steps require executing the DPU HBN setup script, hbn-dpu-setup.sh located in <resource>/scripts/doca_hbn/2.1.0/hbn-dpu-setup.sh.

Note: The entire scripts/doca_hbn folder will be downloaded as part of the NGC Resource as explained in the previous steps (See "Installation and Getting Started").

Run DPU HBN setup script, hbn-dpu-setup.sh, on the DPU (Arm-side) as a root user. This script:

1. Sets the DPU to embedded mode.
2. Removes default OVS configuration and OVS bridges.
3. Enables IPv4/IPv6 kernel forwarding.
4. Sets up huge tables for DPDK.

After running the script, users likely must power cycle the host server and the DPU. Power cycling the host server is only necessary if the DPU was not already in embedded mode. The script output will tell if power cycle is needed.

chmod +x hbn-dpu-setup.sh
./hbn-dpu-setup.sh

Add Network Configuration File

Add network interfaces and FRR configuration files on the DPU to achieve the desired configuration:

• /etc/network/interfaces
• /etc/frr/frr.conf
• /etc/frr/daemons

Adjusting the .yaml configuration

The .yaml configuration for our container is doca_hbn.yaml:

wget https://api.ngc.nvidia.com/v2/resources/nvidia/doca/doca_container_configs/versions/2.6.0v1/files/configs/2.6.0/doca_hbn.yaml

Note: The file is also stored with the rest of the .yaml configurations as were pulled from NGC in the previous steps (See "Installation and Getting Started").

Spawning the container

Simply copy the updated doca_hbn.yaml file to the /etc/kubelet.d directory. Kubelet will automatically pull the container image from NGC, and spawn a pod executing the container. The DOCA HBN service will start executing right away.

# View currently active pods, and their IDs (it might take up to 20 seconds for the pod to start)
crictl pods

# View currently active containers, and their IDs
crictl ps

# Examine logs of a given container
crictl logs 

# Examine kubelet logs, in case something didn't work as expected
journalctl -u kubelet

Please refer to the documentation for more information.

Verify Container is Running

Inspect container to verify the services are running.

1. Log into the HBN container. Run:

   crictl exec -it $(crictl ps | grep hbn | awk '{print $1;}') bash
   

2. Verify frr, nl2doca, neighmgr services are running. Run:

   (hbn-container) $ supervisorctl status frr
   (hbn-container) $ supervisorctl status nl2doca
   (hbn-container) $ supervisorctl status neighmgr
   

   You may look at various logs under /var/log/.

3. It is possible to make changes to /etc/network/interfaces or /etc/frr/frr.conf and reload interface configuration using the following commands:

   (hbn-container) $ vim /etc/network/interfaces
   (hbn-container) $ ifreload -a
   

4. You may, for example, restart frr using the following command:

   (hbn-container) $ supervisorctl restart frr
   

Documentation

The DOCA HBN Service Guide is available here.

License & EULA

DOCA is licensed under the NVIDIA DOCA License. By pulling and using the container, you accept the terms and conditions of this license.

Technical Support

Use the NVIDIA Developers forum for questions regarding this software.