NeMo Framework Megatron Backend

NVIDIA

Container

NVIDIA

NeMo Framework Megatron Backend

NVIDIA NeMo™ framework Megatron backend supports pre-training, post-training, and reinforcement learning of LLMs and multi-modal generative AI models with state-of-the-art data processing, model training techniques, and flexible deployment options.

NVIDIA AI Enterprise Supported

Layer	Label		Created
9e971bd84940b29aadb744dc982b683f1def5fd8a0638ec5beb0bbdcd585c1a3	CONFIG	`Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /workspace; ExposedPorts 6006/tcp, 8888/tcp`	06/27/2024 7:25 AM UTC
c5d787cf851b7f9c715703216654dcc74a62c7fb87b50640ba679a2264fb7715	ARG	`NVIDIA_BUILD_REF`	06/27/2024 7:25 AM UTC
b442755c1f3588f3b9940b93abaf42a97d8d2f4b573c65da5578e18bdbaccea9	LABEL	`com.nvidia.build.id=82611821`	06/27/2024 7:25 AM UTC
19b84797cf8214a954d4b7dce7d4153194db0513ea65e8d559e3aac7c9a26398	ENV	`NVIDIA_BUILD_ID=82611821`	06/27/2024 7:25 AM UTC
5354fd90550c60be520056e76c8ea683319519b37f9e7e320739109788ceac37	ARG	`NVIDIA_BUILD_ID`	06/27/2024 7:25 AM UTC
f7d23240c099762db00c9c6c45b0b5bd8af519fa31a60e25bcf5589dba10596d	RUN	`NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT=810ffae374a2b9cb4b5c5e28eaeca7d7998fca0c TE_COMMIT=a51ff542dcb1f605aa54f9b0e1aaadb132acd53d MCORE_COMMIT=core_r0.7.0 ALIGNER_COMMIT=dev LAUNCHER_COMMIT=24.05rc DATA_CURATOR_COMMIT=e927313dbc602e2310628dec70f9f25490fedb29 NEMO_COMMIT=r2.0.0rc0 chmod -R a+w /workspace`	06/27/2024 7:25 AM UTC
877f8938876a29a619abe4b7152556e5e7fd56606a263a2ef51f9aa943ff8554	WORKDIR	`/workspace`	06/27/2024 7:25 AM UTC
c59c9935570b1419163e9737023bc0b35e8bbe0fff9c5195283c05547ebba36d	RUN	`NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT=810ffae374a2b9cb4b5c5e28eaeca7d7998fca0c TE_COMMIT=a51ff542dcb1f605aa54f9b0e1aaadb132acd53d MCORE_COMMIT=core_r0.7.0 ALIGNER_COMMIT=dev LAUNCHER_COMMIT=24.05rc DATA_CURATOR_COMMIT=e927313dbc602e2310628dec70f9f25490fedb29 NEMO_COMMIT=r2.0.0rc0 pip uninstall -y causal-conv1d && pip install --no-cache-dir 'causal-conv1d==1.2.0.post2'`	06/27/2024 7:25 AM UTC
53698a6a6f1a2a6aab79dcf480a4b9771dca1133db0807b1f96e609a1b17d683	RUN	`NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT=810ffae374a2b9cb4b5c5e28eaeca7d7998fca0c TE_COMMIT=a51ff542dcb1f605aa54f9b0e1aaadb132acd53d MCORE_COMMIT=core_r0.7.0 ALIGNER_COMMIT=dev LAUNCHER_COMMIT=24.05rc DATA_CURATOR_COMMIT=e927313dbc602e2310628dec70f9f25490fedb29 NEMO_COMMIT=r2.0.0rc0 pip uninstall -y jupyterlab aiohttp onnx && pip install --no-cache-dir jupyterlab==3.6.7 aiohttp==3.9.5 onnx==1.16.0`	06/27/2024 7:24 AM UTC
d4c9393e83dd5ddc273d241972454c5fce424f352acbffbacd19e38fdc694cf1	RUN	NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT=810ffae374a2b9cb4b5c5e28eaeca7d7998fca0c TE_COMMIT=a51ff542dcb1f605aa54f9b0e1aaadb132acd53d MCORE_COMMIT=core_r0.7.0 ALIGNER_COMMIT=dev LAUNCHER_COMMIT=24.05rc DATA_CURATOR_COMMIT=e927313dbc602e2310628dec70f9f25490fedb29 NEMO_COMMIT=r2.0.0rc0 python -c "from transformers import CLIPImageProcessor; vit_p=CLIPImageProcessor.from_pretrained('openai/clip-vit-large-patch14'); vit_p_336=CLIPImageProcessor.from_pretrained('openai/clip-vit-large-patch14-336');"	06/27/2024 7:24 AM UTC