DJL - PyTorch native Library¶
This project builds the JNI layer for Java to call PyTorch C++ APIs.
You can find more information in the
You need to install
cmake and C++ compiler on your machine in order to build
apt-get install -y locales cmake curl unzip software-properties-common
Use the following task to build PyTorch JNI library:
This task will send a Jni library copy to
pytorch-engine model to test locally.
Note: PyTorch C++ library requires CUDA path set in the system.
Use the following task to build pytorch JNI library for GPU:
# compile CUDA 10.2 version of JNI ./gradlew compileJNI -Pcu10
# compile CUDA 11.X version of JNI gradlew compileJNI -Pcu11
Format C++ code¶
It uses clang-format to format the code.
PyTorch native package release¶
Step 1: Build new JNI on top of new libtorch on osx, linux-cpu, linux-gpu, windows¶
- Spin up a EC2 instance for linux, linux-gpu, windows, windows-gpu and cd pytorch/pytorch-native.
- Run ./gradlew compileJNI and resolve all the issues you are facing.
- Raise a PR for the JNI code change and don’t merge it until we have the rest things ready.
Step 2: Check dependencies of each JNI¶
- check the dependencies of each JNI library by
otool -L libdjl_torch.dylibfor osx,
ldd libdjl_torch.sofor linux and
dumpbin /dependents libdjl_torch.dllfor windows.
- Compare all dependency libraries to those in downloadPyTorchNativeLib, if we miss copying new dependencies, correct the script.
- modify the version to desired release version in pytorch/pytorch-native/build.gradle and make sure the URL in the task downloadPyTorchNativeLib point to the right, available URL. Usually the URL that is not for the latest version will have %2Bcpu/cuXXX in the end.
- Make corresponding change on build.sh and build.cmd
- Raise PR for script change and get them merge
Step 3: Upload the new version of libtorch dependencies to S3¶
- Spin up a EC2 instance and
cd pytorch/pytorch-native && ./gradlew dPTNL
cd build/native/liband gzip all dependencies (
gzip -r download)
- Create a new pytorch-X.X.X in ai.djl/publish bucket with djl-prod account.
aws s3 sync build/native/lib s3://djl-ai/publish/pytorch-X.X.X
Step 4: Build new JNI and upload to S3:¶
- Merge the JNI code change.
- Now every script should point to new PyTorch version except integration and example are still using old pytorch-native version
- Trigger Native JNI S3 PyTorch and resolve issues if any
Step 5: Build pytorch-native snapshot¶
- Trigger Native Snapshot PyTorch
- Raise a PR to bump up all PyTorch version to new version and add -SNAPSHOT
- Test integration test , example and pytorch-engine unit test with snapshot pytorch-native
Step 6: Publish pytorch-native to staging¶
- Trigger Native Release PyTorch
- Test integration test, example and pytorch-engine unit test with staging pytorch-native
- Publish to sonatype
- Raise a PR to remove all the -SNAPSHOT