How to Run a Reference Pipeline using Local Input Files

Download & Install the Pipeline from NGC

Refer to the Setup section to install this Pipeline.

NGC ID for the clara pull pipeline command: clara_hqs_pipeline
Pipeline file: hqs-pipeline.yaml

Record the pipeline ID during pipeline installation, as it will be used in a later section to trigger a job with input files.

Trigger Pipeline

A pipeline can be triggered via Clara CLI. Follow these steps to trigger a pipeline:

Open a terminal and go to the directory which contains the reference pipeline.

cd ~/.clara/pipelines/clara_hqs_pipeline

Trigger the pipeline using the pipeline ID in the previous section:

clara create jobs -n clarahqs -p <pipeline ID>

When the above step has successfully completed, you should see the following result in the terminal (the Job and Payload UUID may be different).

JOB_ID: daa5aea74d1446a8a24b9d3990bfef67
PAYLOAD_ID: 79aa5500e6134cf888f1d6d89f2e249e

Note down this job ID and payload ID as they would be used in a later section to access output files.

Then, start the job:

clara start job -j <JOB ID>

Check Job Status and Download/View Payloads in Clara Console/Dashboard

With Clara Orchestrator (API version >= 0.4.0)

Go to the Clara Management Console using a web browser: The URL is: <IP of the machine>:32002
Go to the JOBS view. You should see a job with a name that includes the name of the job you specified
You can inspect the status of each operator inside that job

Please check Clara Management Console to see the web application in detail.

With Argo Orchestrator (API version <= 0.3.0)

Go to the Clara dashboard UI using a web browser: The URL is <machine-IP>:8000.
- You should see a job that includes the name of the job you specified (clarahqs-xxxxx).
- You can inspect the status of each operator inside that job.
- Viewing the UI status for Pipeline Jobs executed using the Clara orchestrator option is currently not available. See the orchestrator documentation for more details.
Click the clara-hqs stage and then click LOGS. You should get something similar to the logs below:

:::::::::::::::::::::::Clara Hardware Qualification Suite:::::::::::::::::::::::
NV_DRIVER_VERSION                   : 440.33.01
GPU:
  ATTACHED_GPU(s)                  : 1
  gpu0:
     product_name                  : Quadro GV100
     gpu_virtualization            : None
     PCIe_gen (current/max)        : 3/3
     link_width (current/max)      : 16x/16x
     ecc_errors                    : N/A
     gpu_temperature               : 43 C
     max_clocks:
        graphics_clock             : 1912 MHz
        sm_clock                   : 1912 MHz
        mem_clock                  : 850 MHz
        video_clock                : 1717 MHz
CPU:
  cores                            : 12
  model_name                       : Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz
MEMORY:
  total                            : 62G
  available                        : 19G
STORAGE:
  disk0                            : 931.5G SSD 860
  partitions:
     /dev/sda1:
        file_type                  : ext4
        size                       : 916G
        used                       : 58G
        avail                      : 812G
STORAGE_TEST:
  write_speed                      : 331 MB/s
  read_speed                       : 5.2 GB/s
GPU_TEST:
  bandwidthTest:
     host_to_device                : 12.3 GB/s
     device_to_host                : 13.1 GB/s
  concurrentKernels:
     time_taken                    : 0.012s
  convolutionSeparable:
     throughput                    : 26416.1900 MPixels/sec
     time                          : 0.00036 s
  transpose:
     transpose_simple_copy:
        throughput                 : 479.8362 GB/s
        time                       : 0.01628 ms
     transpose_optimized:
        throughput                 : 484.3772 GB/s
        time                       : 0.01613 ms
  sortingNetworks:
     throughput                    : 454.1256 MElements/s
     time                          : 0.00231 s

Download and Verify the Outputs

Once the job has succeeded, you can download the output payload using the clara download command.

The output payload files are located under the /operators/\<operator name defined in the pipeline definition\>/ folder. Since the triggered pipeline had an AI operator name of clara-hqs, you can use the following command to download output files:

clara download <Job ID>:/operators/clara-hqs/*

Then, you can see the output hardware metrics in JSON format using the following command:

cat hardware_metrics.json

If you are working with the server machine, the payload folder is directly accessible in the default payload folder /clara/payloads/<Payload ID>/.