NeMo Framework Megatron Backend

NVIDIA

Container

NVIDIA

NeMo Framework Megatron Backend

NVIDIA NeMo™ framework Megatron backend supports pre-training, post-training, and reinforcement learning of LLMs and multi-modal generative AI models with state-of-the-art data processing, model training techniques, and flexible deployment options.

NVIDIA AI Enterprise Supported

Layer	Label		Created
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	LABEL	`com.nvidia.build.ref=`	08/02/2024 11:09 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ARG	`NVIDIA_BUILD_REF`	08/02/2024 11:09 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	LABEL	`com.nvidia.build.id=85286408`	08/02/2024 11:09 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ENV	`NVIDIA_BUILD_ID=85286408`	08/02/2024 11:09 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ARG	`NVIDIA_BUILD_ID`	08/02/2024 11:09 PM UTC
sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1	RUN	`NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT= TE_COMMIT=7d576ed25266a17a7b651f2c12e8498f67e0baea MCORE_COMMIT=core_r0.8.0 ALIGNER_COMMIT=r0.4.0rc0 LAUNCHER_COMMIT= DATA_CURATOR_COMMIT= NEMO_COMMIT=r2.0.0rc1 chmod -R a+w /workspace`	08/02/2024 11:09 PM UTC
sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1	WORKDIR	`/workspace`	08/02/2024 11:09 PM UTC
sha256:fe09dfb13976e7d3e058c4c15fc904f0a6340b9d8a0e47d2a14473acb68034fc	RUN	`NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT= TE_COMMIT=7d576ed25266a17a7b651f2c12e8498f67e0baea MCORE_COMMIT=core_r0.8.0 ALIGNER_COMMIT=r0.4.0rc0 LAUNCHER_COMMIT= DATA_CURATOR_COMMIT= NEMO_COMMIT=r2.0.0rc1 python -c "from transformers import CLIPImageProcessor; vit_p=CLIPImageProcessor.from_pretrained('openai/clip-vit-large-patch14'); vit_p_336=CLIPImageProcessor.from_pretrained('openai/clip-vit-large-patch14-336');"`	08/02/2024 11:09 PM UTC
sha256:049794f6ad95f48e01b88905307b2aed13f5b3ba31aff6585b67fdd1b8fa0c81	RUN	`NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT= TE_COMMIT=7d576ed25266a17a7b651f2c12e8498f67e0baea MCORE_COMMIT=core_r0.8.0 ALIGNER_COMMIT=r0.4.0rc0 LAUNCHER_COMMIT= DATA_CURATOR_COMMIT= NEMO_COMMIT=r2.0.0rc1 python -c "from transformers import AutoTokenizer; vit_l_tokenizer=AutoTokenizer.from_pretrained('openai/clip-vit-large-patch14');"`	08/02/2024 11:09 PM UTC
sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1	RUN	NVIDIA_BIGNLP_VERSION= TRT_VER= CUDA_VER= CUDNN_VER= NCCL_VER= CUBLAS_VER= APEX_COMMIT= TE_COMMIT=7d576ed25266a17a7b651f2c12e8498f67e0baea MCORE_COMMIT=core_r0.8.0 ALIGNER_COMMIT=r0.4.0rc0 LAUNCHER_COMMIT= DATA_CURATOR_COMMIT= NEMO_COMMIT=r2.0.0rc1 python -c "from transformers import AutoTokenizer; tok_llama3_8b=AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-8B'); tok_llama3_70b=AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-70B'); tok_llama3_8b_inst=AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-8B-instruct'); tok_llama3_70b_inst=AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-70B-instruct');"	08/02/2024 11:09 PM UTC