ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
NeMo Framework Megatron Backend
Container
NVIDIA
NVIDIA
NeMo Framework Megatron Backend

NVIDIA NeMo™ framework Megatron backend supports pre-training, post-training, and reinforcement learning of LLMs and multi-modal generative AI models with state-of-the-art data processing, model training techniques, and flexible deployment options.

NVIDIA AI Enterprise Supported
LayerLabelCreated
338a1dd4a516a407666cc6c1fc1c9d7b442543641d84083ecf7f6afc37ca365eCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /opt; ExposedPorts 6006/tcp, 8888/tcp
04/09/2024 2:35 AM UTC
61f181c0096c2f18b563f6794d0c519291527b86aa62218498beefc354f6fd6bRUN
git clone https://github.com/NVIDIA/NeMo.git &&
  cd NeMo &&
  git checkout chcui/conversion_precision_on_24.03 &&
  mkdir -p /opt/NeMo/scripts/checkpoint_converters &&
  cp scripts/checkpoint_converters/* /opt/NeMo/scripts/checkpoint_converters &&
  rm /tmp/NeMo -r
04/09/2024 2:35 AM UTC
2ab93c9fafc3bb7b3afcc138ca9ffd24b9f462c45b29d724c270466a5a4b54daWORKDIR
/tmp
04/09/2024 2:35 AM UTC
614882b8a7c29b171d9c69e3c78d86720e1e3c062b4401c2196c976c0470c50dCOPY
llm_inference/changes/megatron_gpt_model.py /opt/NeMo/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py
04/09/2024 2:35 AM UTC
967b917eed22e59d584ebd09adda27203c67f8672cd4312875e5693a513febd0COPY
llm_inference/nemo-fw/scripts/export /opt/NeMo/scripts/export
04/09/2024 2:35 AM UTC
2a5cc763d12261ea463ef09eddd2a42a42ab41c35585ba0d4bd8d3d34aba8717COPY
llm_inference/nemo-fw/scripts/deploy /opt/NeMo/scripts/deploy
04/09/2024 2:35 AM UTC
c59685e5b209e97fb792d6930b7e6c0745b8622f3fc3277f9cbbec7e6c357c36COPY
llm_inference/nemo-fw/nemo/export /opt/NeMo/nemo/export
04/09/2024 2:35 AM UTC
697199f516016cf69c46023d3cd281b4386d359618af928dd42624883170ed28COPY
llm_inference/nemo-fw/nemo/deploy /opt/NeMo/nemo/deploy
04/09/2024 2:35 AM UTC
98dad150f6497217ed3124e08b8b85ec9046bca7ddf46ddeccaa73251e238e3bRUN
ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  ln -sv lib/libnvinfer_plugin_tensorrt_llm.so lib/libnvinfer_plugin_tensorrt_llm.so.9 &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig
03/18/2024 2:59 PM UTC
cefecc0625a195daa7228e49105a1d016076bcc2654f00be8189e916e1e0dcfcRUN
pip install tensorrt_llm*.whl --extra-index-url https://pypi.nvidia.com &&
  rm tensorrt_llm*.whl
03/18/2024 2:59 PM UTC
...