Cloud Database

A distributed and replicated storage service using Java.

Approach

• Storage servers (KVServers) are monitored and controlled by an External Configuration Service (ECS).

• Tasks are assigned to KVServers by consistent hashing.

• Virtual nodes are introduced to achieve load balancing.

  In figure below, we can observe that the 3 servers were not evenly distributed initially, server A handling 2 tasks, server B handling 1 task and server C handling 3 tasks. However, with the introduction of virtual nodes (A*, B*, C*), each server handles 2 tasks, achieving a more balanced distribution of workload.

• Eventual consistency is achieved by using an optimistic lazy Multi-Primary Replication using gossiping protocol.

• KVservers can implement any of the 3 caching strategies:

  1. FIFO (First In First Out)
  2. LFU (Least Frequently Used)
  3. LRU (Least Recently Used)

• Authentication and authorisation is also implemented based on the OAuth authentication framework, using JSON Web Token (JWT) as the access token.

How to run

1. Build project by running

mvn package

2. Start ECS e.g.

java -jar target/ecs-server.jar  -l ecs.log –ll ALL -a 192.168.1.1 -p 5152

3. Start KVServers e.g.

java -jar target/kv-server.jar -l kvserver1.log -ll ALL -d data/kvstore1/ -a 192.168.1.2 -p 5153 -b 192.168.1.1:5152

4. You can add interact with the database using the client

java -jar target/echo-client.jar