Linux / amd64
DOCA Telemetry Service (DTS) runs inside of its own Kubernetes pod on the BlueField and collects data from built-in providers and external telemetry applications. The following providers are available (sysfs is enabled by default):
Aggregation providers to collect data from other applications via TCP:
Additional telemetry applications written on top of DOCA API can send the service data over IPC (shared memory).
The providers that require additional permissions, which cannot be given by DTS, can be executed via Doca Privileged Executer (DPE) server. Data collected in DPE server will be consumed by DTS as standard providers data. For details please refer to documentation.
Collected data is written/exported according to the configuration. The following oprions are available:
Telemetry agent can export the data via Prometheus (pull) or use Fluent bit (push). The Prometheus endpoint is bound to port 9100 and can be enabled using the config file. Data collected by several Telemetry Agents from their BlueFields can be aggregated by a Telemetry Agent that runs on a separate host. This setup requires InfiniBand port configuration.
Note: starting from DTS version 1.9.0-doca1.3.0, shared memory is mounted under "/dev/shm/telemetry/". Telemetry applications should be mounted to the same folder.
Note: DTS version 1.9.0-doca1.3.0 is not backward compatible with previous DOCA API versions. Note: DTS version 1.11.0-doca1.5.0 is not backward compatible with previous DOCA API versions.
All preparation steps are listed under DOCA's Container Deployment User Guide.
Note: The DOCA Service container is configured for K8S-based deployment, hence the use of the
docker pull command is discouraged.
The .yaml configuration for our container is doca_telemetry.yaml:
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/doca/doca_container_configs/versions/2.2.0v1/files/configs/2.2.0/doca_telemetry.yaml
The yaml file allows generating DTS configuration from scratch, and overwriting Fluent-Bit export configuration.
Note: The file is also stored with the rest of the .yaml configurations as were pulled from NGC in the previous steps (See "Installation and Getting Started").
In case Fluent Bit forwarding should be enabled, please add the destination host and port to the "command" found in the initContainers section:
command: ["/bin/bash/", "-c", "/usr/bin/telemetry-init.sh && /usr/bin/enable-forward-to-morpheus.sh -i=127.0.0.1 -p=24224"]
Please note that the host and port shown above are just an example.
Simply copy the updated
doca_telemetry.yaml file to the
/etc/kubelet.d directory. Kubelet will automatically pull the container image from NGC, and spawn a pod executing the container. The DOCA Telemetry Service application will start executing right away.
# View currently active pods, and their IDs (it might take up to 20 seconds for the pod to start) crictl pods # View currently active containers, and their IDs crictl ps # Examine logs of a given container crictl logs # Examine kubelet logs, in case something didn't work as expected journalctl -u kubelet
Please refer to the documentation for more information.
Note: Host deployment is available starting from DTS version 1.13.0-doca2.0.2
x86_64 host version runs as docker container. Please refer to section Host Deployment of documentation for configuration and run commands.
The DOCA Telemetry Service guide is available here.
DOCA is licensed under the NVIDIA DOCA License. By pulling and using the container, you accept the terms and conditions of this license.
Use the NVIDIA Developers forum for questions regarding this Software.