Skip to content

LMI Backend User Guides

LMI provides backend-specific user guides that cover the following topics:

  • Model Artifact Structure

  • All backends support standard HuggingFace Transformers Pretrained artifacts

  • The TensorRT-LLM and Transformer-NeuronX user guides provide information on compiled model artifact structures

  • Supported Model Architectures

  • Some Model Architectures can only be deployed using specific backends

  • Quick Start Configurations

  • Starter configurations in both and environment variable formats to provide an out-of-the-box solution for that backend

  • Quantization Guide

  • If a backend supports quantization, we describe the different options and how to enable them

  • Advanced Configurations

  • Configurations that are only available with this backend

The available backends and their respective user guides are available below: