Mini Analytics is a small scale project I've done as an internship introduction project.
This project can be used to search Twitter for user written Lucene queries.
To install this project on your system you need Maven and Docker. If you want to fetch tweets from Twitter you additionally need a Twitter Developer Account (optional).
To download the project just use git clone https://github.com/Max-Leopold/mini-analytics.git to clone the project onto your machine.
You can configure the project to only use your desired sources. Currently available sources are:
In order to start the project you have to specify which sources you want to use. To do this, open the mini-analytics/docker/sources.txt file.
Add the sources you want to scrape to the end of first line, separated by one whitespace.
Example services.txt for all services:
zoo1 kafka1 kafka2 kafka3 postgres solr1 solr2 create-collection mention-storer mentions-generator Twitter Reddit
If you want to start the project and include the Twitter Module you have to specify your Twitter Developer Keys in the mini-analtics/twitterpuller/src/main/resources/twitter4j.properties file
oauth.consumerKey= your consumer key
oauth.consumerSecret= your consumer secret
oauth.accessToken= your access token
oauth.accessTokenSecret= your access token secret
You can find more information about configuration here.
To run the project navigate to your project directory and use the command mvn package in your command line interface.
Now navigate to the mini-analytics/Docker directory and start the application with the command docker-compose up $(<services.txt) --build.
To interact with the application just visit localhost:8080.