DJL Kafka Sentiment Analysis on Twitter Data¶
This an example to show case how to deploy a deep learning model in Apache Kafka.
We will use DJL's built-in sentiment analysis model based on PyTorch to run analysis on twitter data.
There is a
data.txt extracted from the Kaggle Twitter Sentiment Analysis Dataset.
This demo will make predictions on whether the tweet is positive or negative.
- DJL requires JDK 8+, you can follow the quick start guide.
- Kafka installed, download from website or use
brew install kafka
Steps to run¶
1. start zookeeper:¶
2. start kafka server:¶
3. create test topic¶
kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic twitter-data
4. download data¶
5. pipe twitter data into producer¶
kafka-console-producer --broker-list localhost:9092 --topic twitter-data < data.txt
6. run prediction inside consumer¶
content: @Frumph I'd hug you, too! Poor Frumph..... prediction: [ class: "Positive", probability: 0.99894 class: "Negative", probability: 0.00105 ] content: Andre Riue on neighbours..what has the world come to...internets down lol prediction: [ class: "Negative", probability: 0.97309 class: "Positive", probability: 0.02690 ] content: Looks like rain today, bet it buckets down as soon as I step outside front door, always the way !!!!, downhill all the way from today prediction: [ class: "Negative", probability: 0.99624 class: "Positive", probability: 0.00375 ]