Run this notebook online:

PaddleOCR DJL example¶

In this tutorial, we will be using pretrained PaddlePaddle model from PaddleOCR to do Optical character recognition (OCR) from the given image. There are three models involved in this tutorial:

Word detection model: used to detect the word block from the image
Word direction model: used to find if the text needs to rotate
Word recognition model: Used to recognize test from the word block

Import dependencies and classes¶

PaddlePaddle is one of the Deep Engines that requires DJL hybrid mode to run inference. Itself does not contains NDArray operations and needs a supplemental DL framework to help with that. So we import Pytorch DL engine as well in here to do the processing works.

// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/

%maven ai.djl:api:0.22.1
%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.22.1
%maven org.slf4j:slf4j-simple:1.7.32

// second engine to do preprocessing and postprocessing
%maven ai.djl.pytorch:pytorch-engine:0.22.1

import ai.djl.*;
import ai.djl.inference.Predictor;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.modality.cv.output.*;
import ai.djl.modality.cv.util.NDImageUtils;
import ai.djl.ndarray.*;
import ai.djl.ndarray.types.DataType;
import ai.djl.ndarray.types.Shape;
import ai.djl.repository.zoo.*;
import ai.djl.paddlepaddle.zoo.cv.objectdetection.PpWordDetectionTranslator;
import ai.djl.paddlepaddle.zoo.cv.imageclassification.PpWordRotateTranslator;
import ai.djl.paddlepaddle.zoo.cv.wordrecognition.PpWordRecognitionTranslator;
import ai.djl.translate.*;
import java.util.concurrent.ConcurrentHashMap;

the Image¶

Firstly, let's take a look at our sample image, a flight ticket:

String url = "https://resources.djl.ai/images/flight_ticket.jpg";
Image img = ImageFactory.getInstance().fromUrl(url);
img.getWrappedImage();

Word detection model¶

In our word detection model, we load the model exported from PaddleOCR. After that, we can spawn a DJL Predictor from it called detector.

var criteria1 = Criteria.builder()
        .optEngine("PaddlePaddle")
        .setTypes(Image.class, DetectedObjects.class)
        .optModelUrls("https://resources.djl.ai/test-models/paddleOCR/mobile/det_db.zip")
        .optTranslator(new PpWordDetectionTranslator(new ConcurrentHashMap<String, String>()))
        .build();
var detectionModel = criteria1.loadModel();
var detector = detectionModel.newPredictor();

[IJava-executor-0] INFO ai.djl.paddlepaddle.jni.LibUtils - Paddle MKL/GPU requires user to set LD_LIBRARY_PATH=/home/runner/.djl.ai/paddle/2.3.2-cpu-linux-x86_64, the current one is set to: /opt/hostedtoolcache/Python/3.11.3/x64/lib
[IJava-executor-0] INFO ai.djl.pytorch.engine.PtEngine - PyTorch graph executor optimizer is enabled, this may impact your inference latency and throughput. See: https://docs.djl.ai/docs/development/inference_performance_optimization.html#graph-executor-optimization
[IJava-executor-0] INFO ai.djl.pytorch.engine.PtEngine - Number of inter-op threads is 1
[IJava-executor-0] INFO ai.djl.pytorch.engine.PtEngine - Number of intra-op threads is 2

Then, we can detect the word block from it. The original output from the model is a bitmap that marked all word regions. The PpWordDetectionTranslator convert the output bitmap into a rectangle bounded box for us to crop the image.

var detectedObj = detector.predict(img);
Image newImage = img.duplicate();
newImage.drawBoundingBoxes(detectedObj);
newImage.getWrappedImage();