Kipoi is an API and a repository of trained models for application to genomics. The repository contains over 2000 different models, covering canonical predictive tasks in transcriptional and post-transcriptional gene regulation.
More information about Kipoi is available on the Kipoi webpage.
Before running the NGC Kipoi container, please ensure that your system meets the following requirements.
A Kipoi executable optimized for your system hardware will be chosen automatically at runtime.
Kipoi provides models and example data through its API. To run Kipoi with the example data for the DeepSEA/predict
model:
Use the Kipoi container to download the DeepSEA/predict
example data:
Docker:
nvidia-docker run --rm -it --net host -v $(pwd):/host_pwd kipoi:v0.6.8-keras2 kipoi get-example DeepSEA/predict -o /host_pwd/example
Singularity:
$ singularity build kipoi_v0.6.8-keras2.simg docker://nvcr.io/hpc/kipoi:v0.6.8-keras2
$ singularity run --nv -B $(pwd):/host_pwd kipoi:v0.6.8_keras2.simg /bin/bash -c "source activate kipoi-py3-keras2 kipoi get-example DeepSEA/predict -o /host_pwd/example"
Use the Kipoi container to run the DeepSEA/predict
example:
Docker:
$ nvidia-docker run --rm -it --net host -v $(pwd):/host_pwd kipoi:v0.6.8-keras2 kipoi predict DeepSEA/predict --dataloader_args='{"intervals_file": "/host_pwd/example/intervals_file", "fasta_file": "/host_pwd/example/fasta_file"}' -o '/host_pwd/predict.tsv'
Singularity:
$ singularity run --nv -B $(pwd):/host_pwd kipoi_v0.6.8-keras2.simg /bin/bash -c "source activate kipoi-py3-keras2 && kipoi predict DeepSEA/predict --dataloader_args='{"intervals_file": "/host_pwd/example/intervals_file", "fasta_file": "/host_pwd/example/fasta_file"}' -o '/host_pwd/predict.tsv'"
More detailed instructions on using the NGC Microvolution container in Docker and Singularity can be found below.
NGC supports the Docker runtime through the nvidia-docker plugin.
To simplify the examples below, define the following command alias. This may be set as an environment variable in a shell or batch script.
DOCKER
will be used to launch processes within the NGC Microvolution container using the nvidia-docker
runtime:
$ export DOCKER="nvidia-docker run --rm -it --net host -v $(pwd):/host_pwd nvcr.io/hpc/kipoi:v0.6.8-keras2"
Where:
DOCKER
: alias used to store the base Docker commandrun
: specifies the mode of execution--rm
: makes the container instance ephemeral (does not save on exit)-it
: runs the container in an interactive tty shell--net host
: allows the container to use the host network devices (necessary to connect to the Kipoi API server)-v $(pwd):/host_pwd
: bind mounts the current working directory into the container at /host_pwd
nvcr.io/hpc/kipoi:v0.6.8-keras2
: URI of the latest NGC Kipoi containerThis mode of running is suitable for interactive execution from a local workstation containing one or more GPUs. There are no requirements other than those stated in the System Requirements section.
To launch Kipoi via nvidia-docker, use:
$ ${DOCKER} kipoi predict <model_name> --dataloader_args='{"intervals_file": "</path/to/intervals_file>", "fasta_file": "</path/to/fasta_file>"}' -o '</path/to/output_file.tsv>' -n <thread_count> --batch_size <batch_size>
Where:
DOCKER
: alias used to store the base Docker command<model_name>
: the name of the Kipoi model to use</path/to/intervals_file>
: path to the intervals input file for Kipoi</path/to/fasta_file>
: path to the fasta input file for Kipoi</path/to/output_file.tsv>
: path of a file for Kipoi to write prediction results into<thread_count>
: specify the number of threads for Kipoi to use at runtime<batch_size>
: specify the number of input intervals for Kipoi to process simultaneouslyTo start an interactive shell within the container environment, run /bin/bash
in the container:
$ ${DOCKER} /bin/bash
Where:
DOCKER
: alias used to store the base Docker command/bin/bash
: starts an interactive bash shellTo run Kipoi while using the interactive shell:
$ ${DOCKER} kipoi predict <model_name> --dataloader_args='{"intervals_file": "</path/to/intervals_file>", "fasta_file": "</path/to/fasta_file>"}' -o '</path/to/output_file.tsv>' -n <thread_count> --batch_size <batch_size>
Where:
DOCKER
: alias used to store the base Docker command<model_name>
: the name of the Kipoi model to use</path/to/intervals_file>
: path to the intervals input file for Kipoi</path/to/fasta_file>
: path to the fasta input file for Kipoi</path/to/output_file.tsv>
: path of a file for Kipoi to write prediction results into<thread_count>
: specify the number of threads for Kipoi to use at runtime<batch_size>
: specify the number of input intervals for Kipoi to process simultaneouslySave the NGC Kipoi container as a local Singularity image file:
$ singularity build kipoi_v0.6.8-keras2.simg docker://nvcr.io/hpc/kipoi:v0.6.8-keras2
This command saves the container in the current directory as kipoi_v0.6.8-keras2.simg
In order to pull NGC images with singularity
version 2.x and earlier, NGC container registry authentication credentials are required.
To set your NGC container registry authentication credentials:
$ export SINGULARITY_DOCKER_USERNAME='$oauthtoken'
$ export SINGULARITY_DOCKER_PASSWORD=<NVIDIA NGC Cloud Services API key>
More information describing how to obtain and use your NVIDIA NGC Cloud Services API key can be found here.
Once the local Singularity image has been pulled, the following modes of running are supported:
To simplify the examples below, define the following command aliases. These may be set as environment variables in a shell or batch script.
SINGULARITY
will be used to launch processes within the NGC Kipoi container using the Singularity runtime:
$ export SINGULARITY="$(which singularity) run --nv -B $(pwd):/host_pwd kipoi_v0.6.8-keras2.simg"
Where:
run
: specifies mode of execution--nv
: exposes the host GPU to the container-B $(pwd):/host_pwd
: bind mounts the current working directory in the container at /host_pwd
kipoi_v0.6.8-keras2.simg
: path of the saved Singularity image fileThis mode of running is suitable for interactive execution from a local workstation containing one or more GPUs. There are no requirements other than those stated in the System Requirements section.
To launch Kipoi, use:
$ ${SINGULARITY} /bin/bash -c "source activate kipoi-py3-keras1.2 && kipoi predict <model_name> --dataloader_args='{"intervals_file": "</path/to/intervals_file>", "fasta_file": "</path/to/fasta_file>"}' -o '</path/to/output_file.tsv>' -n <thread_count> --batch_size <batch_size>"
Where:
source activate kipoi-py3-keras1.2
: activates the appropriate Conda environment for keras/v1.2 runs of KipoiSINGULARITY
: alias used to store the base Singularity command<model_name>
: the name of the Kipoi model to use</path/to/intervals_file>
: path to the intervals input file for Kipoi</path/to/fasta_file>
: path to the fasta input file for Kipoi</path/to/output_file.tsv>
: path of a file for Kipoi to write prediction results into<thread_count>
: specify the number of threads for Kipoi to use at runtime<batch_size>
: specify the number of input intervals for Kipoi to process simultaneouslyTo invoke an interactive shell, run /bin/bash
within the container:
$ ${SINGULARITY} /bin/bash
While Singularity provides an interactive shell via singularity shell
, this invocation ignores container entrypoint
scripts. Thus, the preferred method to access an interactive shell is via a singularity run
command directed at /bin/bash
.
To run a HOOMD-blue Python script while using the interactive shell:
$ ${SINGULARITY} /bin/bash -c "source activate kipoi-py3-keras1.2 && /bin/bash"
Where:
SINGULARITY
: alias used to store the base Singularity command