Skip to content

DJL Serving Inference API

There are two types of APIs:

  1. Health check API - Check DJL Serving health status
  2. Predictions API - Make predictions API call to DJL Serving

Health check API

GET /ping DJL Serving supports a ping API that user can check health status:

curl http://localhost:8080/ping

  "health": "healthy!"

Predictions API

POST /predictions/{model_name}

POST /predictions/{model_name}/{version}

For each loaded model, user can make REST call to run the prediction

# Load PyTorch resent18 model:
curl -X POST "http://localhost:8080/models?"

# Download an image for testing
curl -O

curl -X POST http://localhost:8080/predictions/traced_resnet18 -T kitten.jpg


curl -X POST http://localhost:8080/predictions/traced_resnet18 -F "data=@kitten.jpg"

The result was some JSON that told us our image likely held a tabby cat.

    "className": "n02123045 tabby, tabby cat",
    "probability": 0.40216901898384094
    "className": "n02123159 tiger cat",
    "probability": 0.29153719544410706
    "className": "n02124075 Egyptian cat",
    "probability": 0.27031397819519043
    "className": "n02123394 Persian cat",
    "probability": 0.007626921869814396
    "className": "n02127052 lynx, catamount",
    "probability": 0.004957360681146383