Skip to content
Deep Java Library
System Design Guide
Initializing search
deepjavalibrary/djl
Home
Tutorials
Guides
DJL Community
Supported Engines
Extensions
DJL Serving
Large Model Inference
Demos
Deep Java Library
deepjavalibrary/djl
Home
Home
Main
Getting DJL
Quick start
Documentation
Examples
Interactive Development
Contributor Documentation
Contributor Documentation
Main
Setup development environment
Development Guideline
Troubleshooting
DJL dependency management
Add a new model to the DJL model zoo
Add a new dataset to DJL basic datasets
Roadmap
FAQ
Tutorials
Tutorials
Beginner Tutorial
Beginner Tutorial
01 create your first network
02 train your first model
03 image classification with your model
Dive into Deep Learning
rank classification using BERT on Amazon Review
Transfer learning on cifar10
Load your own BERT
Load your own BERT
BERT with MXNet
BERT with PyTorch
Guides
Guides
Models
Models
Model Loading
Model Zoo
Datasets
Datasets
Dataset
Dataset Creation
Inference and Production
Inference and Production
Create a serving ready model
Logging
Metrics
Inference Performance Optimization
Engine Profiler Support
Resource Caches
Memory Management
Computer Vision Utilities
DJL Community
DJL Community
Forums
Community Leaders
Supported Engines
Supported Engines
Overview
PyTorch
PyTorch
Overview
PyTorch Engine
PyTorch NDArray Operators
PyTorch Model Zoo
Import PyTorch Model
Load a PyTorch Model
TensorFlow
TensorFlow
Overview
TensorFlow Engine
TensorFlow Model Zoo
Import TensorFlow Model
Load a TensorFlow Model
Apache MXNet
Apache MXNet
Overview
MXNet Engine
MXNet Model Zoo
Import Gluon Model
Load a MXNet Model
Backend Optimizer for MXNet
Hybrid engines
Hybrid engines
Hybrid engine overview
ONNX Runtime
ONNX Runtime
Overview
Load a ONNX Model
XGBoost
LightGBM
TensorRT
Extensions
Extensions
Android
AWS S3 support
Audio
fastText
Hadoop support
Huggingface Tokenizers
OpenCV
SentencePiece
Spark support
Tablesaw
TimeSeries
DJL Zero
DJL Benchmark
DJL Serving
DJL Serving
Why DJL Serving?
Starting DJL Serving
DJL Serving Inference
DJL Serving Operation Modes
Management Console
Configuration
Configuration
DJL Serving Configuration
Global Configuration
Engine Configuration
Deep Learning Workflows
Model Configuration
DJL Serving Architecture
HTTP API
HTTP API
DJL Serving Inference API
DJL Serving Management API
DJL Serving plugin management
DJL Serving - WorkLoadManager
Large Model Inference
Large Model Inference
Table of Contents
User Guides
User Guides
LMI Backend User Guides
LMI Starting Guide
DeepSpeed Engine User Guide
LMI-Dist Engine User Guide
vLLM Engine User Guide
Transformers-NeuronX Engine in LMI
TensorRT-LLM(TRT-LLM) Engine User Guide
HuggingFace Accelerate User Guide
LMI handlers Inference API Schema
Chat Completions API Schema
Deployment Guides
Deployment Guides
Steps for Deploying models with LMI Containers on AWS SageMaker
Model Artifacts for LMI
Instance Type Selection
Backend Selection
Container and Model Configurations
Deploying your model on a SageMaker Endpoint
Benchmarking your Endpoint
Testing for custom script/entryPoint with LMI
Tutorials
Tutorials
Seq-Scheduler and Max-Sparsity Thresholding
TensorRT-LLM ahead-of-time compilation of models tutorial
TensorRT-LLM manual compilation of models tutorial
LMI NeuronX ahead-of-time compilation of models tutorial
Conceptual Guides
Conceptual Guides
LMI running Engines
SageMaker LMI containers resources
SageMaker LMI containers resources
SageMaker Sample Notebooks for LLM
Demos
Demos
Demos
AWS
AWS
Amazon SageMaker
Amazon SageMaker
Start with SageMaker
SageMaker Notebook
SageMaker Studio
AWS-kinesis-video-streams
Model Serving on AWS BeanStalk EC2
AWS Lambda Serverless Model Serving with DJL
AWS EMR
AWS EMR
Distributed inference
GPU Image Classification
AWS Inferentia
Android
Android
Doodledraw (PyTorch)
Style Transfer (PyTorch)
Face Detection (PyTorch)
MXNet Android Template
EcoSystem
EcoSystem
Java Integrations
Java Integrations
DJL Component in Apache Camel
Run TensorFlow model on GraalVM
Apache Spark Image Classification
Apache Beam CTR Prediction
Apache Flink
Apache Flink
Sentiment Analysis
Sentence Encoding
Apache Kafka Twitter Sentiment Analysis
Quarkus
Quarkus
DJL Extension for Quarkus
Integration without the Extension
Applications
Applications
Footwear Classification
Live Object Detection
Pneumonia Detection
MultiEngine on DJL
Interactive JShell and Block Runner for DJL
Malicious URL Detection
Extensions
Extensions
Visualizing Training with DJL
Interactive JShell and Block Runner for DJL
System Design Guide
¶
TODO