LMI V13 DLC containers release¶
This document will contain the latest releases of our LMI containers for use on SageMaker. For details on any other previous releases, please refer our github release page
Release Notes¶
Key Features¶
LMI Container (vllm, lmi-dist) - Release 11-23-2024¶
- vLLM updated to version 0.6.3.post1
- Support for SageMaker Fast Model Loading: https://aws.amazon.com/blogs/machine-learning/introducing-fast-model-loader-in-sagemaker-inference-accelerate-autoscaling-for-your-large-language-models-llms-part-1/
- Support for Multi-Lora Inference natively on SageMaker: https://aws.amazon.com/blogs/machine-learning/easily-deploy-and-manage-hundreds-of-lora-adapters-with-sagemaker-efficient-multi-adapter-inference/