A distributed and replicated storage service using Java.
- Storage servers (KVServers) are monitored and controlled by an External Configuration Service (ECS).
-
Tasks are assigned to KVServers by consistent hashing.
-
Virtual nodes are introduced to achieve load balancing.
In figure below, we can observe that the 3 servers were not evenly distributed initially, server A handling 2 tasks, server B handling 1 task and server C handling 3 tasks. However, with the introduction of virtual nodes (A*, B*, C*), each server handles 2 tasks, achieving a more balanced distribution of workload.
-
Eventual consistency is achieved by using an optimistic lazy Multi-Primary Replication using gossiping protocol.
-
KVservers can implement any of the 3 caching strategies:
- FIFO (First In First Out)
- LFU (Least Frequently Used)
- LRU (Least Recently Used)
-
Authentication and authorisation is also implemented based on the OAuth authentication framework, using JSON Web Token (JWT) as the access token.
- Build project by running
mvn package- Start ECS e.g.
java -jar target/ecs-server.jar -l ecs.log –ll ALL -a 192.168.1.1 -p 5152- Start KVServers e.g.
java -jar target/kv-server.jar -l kvserver1.log -ll ALL -d data/kvstore1/ -a 192.168.1.2 -p 5153 -b 192.168.1.1:5152- You can add interact with the database using the client
java -jar target/echo-client.jar