NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. NAMD uses the popular molecular graphics program VMD for simulation setup and trajectory analysis, but is also file-compatible with AMBER, CHARMM, and X-PLOR.
The following requirements must be met before running the NGC NAMD container:
The ApoA1 benchmark consists of 92,224 atoms and has been a standard NAMD cross-platform benchmark for years. Follow the steps below to use the APOA1 input dataset to test the NGC NAMD container.
Download the APOA1 dataset to your current directory:
wget -O - https://gitlab.com/NVHPC/ngc-examples/raw/master/namd/3.0/get_apoa1.sh | bash
Take a moment to inspect the shell script above. In particular, it injects the CUDASOAintegrate on
in the configuration file, which enables the NAMD 3.0 GPU-resident mode code path.
Replace {input_file}
in the examples below with the path to the apoa1 namd input file:
/host_pwd/apoa1/apoa1_nve_cuda_soa.namd
Several NAMD images are available, depending on your needs. Set the following environment variable which will be used in the example below.
export NAMD_TAG={TAG}
Where {TAG}
is 3.0-beta2
or any other tag previously posted on NGC.
Set the executable name depending on the chosen tag.
export NAMD_EXE=namd3
3.0-beta2
3.0-alpha11
or specify both GPU-resident mode (CUDASOAintegrate on
) and device migration (DeviceMigration on
).NGC supports the Docker runtime through the nvidia-docker plugin.
DOCKER="nvidia-docker run -it --rm -v $(pwd):/host_pwd nvcr.io/hpc/namd:${NAMD_TAG}"
DOCKER="nvidia-docker run -it --rm -v $(pwd):/host_pwd --device=/dev/infiniband --cap-add=IPC_LOCK --net=host nvcr.io/hpc/namd:${NAMD_TAG}"
Launch NAMD with 1 CPU thread, utilizing 1 GPU (simplest way for NAMD versions >= 3.0), on your local machine or single node:
${DOCKER} ${NAMD_EXE} +p1 +devices 0 +setcpuaffinity {input_file}
The +p
argument is used to specify the number of cores to be used, and +devices
specify the GPUs used. To use 2 CPU threads and 2 GPUs:
${DOCKER} ${NAMD_EXE} +p2 +devices 0,1 +setcpuaffinity {input_file}
An example shell script demonstrating this mode is available for 3.0-alpha11 tag.
In addition, for very large systems, multi-node simulations, or Pascal GPUs, it is recommanded to run NAMD 2.x.
In NAMD 3, this can be achieved by setting CUDASOAintegrate off
or simply not setting it in the configuration file.
The input file /host_pwd/apoa1/apoa1_nve_cuda.namd
(note the lack of _soa
in file name) in the APOA1 dataset can be used to test this:
${DOCKER} ${NAMD_EXE} +ppn $(nproc) +setcpuaffinity +idlepoll {input_file}
The nproc
command is used to specify all available CPU cores should be used. Depending on system setup manually specifying the number of PE's may yield better performance.
The NGC NAMD container provides native Singularity runtime support.
Save the NGC NAMD container as a local Singularity image file based upon if you're targeting a single workstation or a multi-node cluster.
singularity build ${NAMD_TAG}.sif docker://nvcr.io/hpc/namd:${NAMD_TAG}`
The container is now saved in the current directory as ${NAMD_TAG}.sif
Define the SINGULARITY
command alias.
SINGULARITY="$(which singularity) exec --nv -B $(pwd):/host_pwd ${NAMD_TAG}.sif"
Launch NAMD with 1 CPU thread, utilizing 1 GPU (simplest way for NAMD versions >= 3.0), on your local machine or single node:
${SINGULARITY} ${NAMD_EXE} +p1 +devices 0 +setcpuaffinity {input_file}
The +p
argument is used to specify the number of cores to be used, and +devices
specify the GPUs used. To use 2 CPU threads and 2 GPUs:
${SINGULARITY} ${NAMD_EXE} +p2 +devices 0,1 +setcpuaffinity {input_file}
An example shell script demonstrating this mode is available for 3.0-alpha11 tag.
In addition, for very large systems, multi-node simulations, or Pascal GPUs, it is recommanded to run NAMD 2.x.
In NAMD 3, this can be achieved by setting CUDASOAintegrate off
or simply not setting it in the configuration file.
The input file /host_pwd/apoa1/apoa1_nve_cuda.namd
(note the lack of _soa
in file name) in the APOA1 dataset can be used to test this:
${SINGULARITY} ${NAMD_EXE} +ppn $(nproc) +setcpuaffinity +idlepoll {input_file}
The nproc
command is used to specify all available CPU cores should be used. Depending on system setup manually specifying the number of PE's may yield better performance.
There is currently a bug in Singularity 3.1.x and 3.2.x causing the LD_LIBRARY_PATH
to be incorrectly set within the container environment.
As a workaround The LD_LIBRARY_PATH
must be unset before invoking Singularity:
$ LD_LIBRARY_PATH="" singularity exec ...
NVIDIA Base Command Platform (BCP) offers a ready-to-use cloud-hosted solution that manages the end-to-end lifecycle of development, workflows, and resource management. Before running the commands below, install and configure the ngc cli, more information can be found here.
Note: apoa1_nve_cuda_soa.namd needs to be modified to remove the outputName
parameter due to the nature of the mounted read-only dataset directory.
Upload the apoa1
dataset using the command below
ngc dataset upload --source ./apoa1/ --desc "NAMD dataset" namd_dataset
Single node on a single GPU running the apoa1
dataset
ngc batch run --name "NAMD_single_gpu" --priority NORMAL --order 50 --preempt RUNONCE --min-timeslice 0s --total-runtime 0s --ace <your-ace> --instance dgxa100.80g.1.norm --commandline "namd3 +p1 +devices 0 +setcpuaffinity --outputName /results/namd_output /work/apoa1_nve_cuda_soa.namd" --result /results/ --image "hpc/namd:${NAMD_TAG}" --org <your-org> --datasetid <datasetid>:/work/