Thamuz is a distributed file storage system based on Java and gRPC. It was designed to efficiently manage file storage and retrieval in a cluster environment. The system consists of data nodes, which are responsible for storing files, and a coordinator node, which manages data node interactions and handles client requests.
- Distributed Architecture: Data is stored across multiple nodes for reliability and scalability.
- Coordinator and Data Nodes: Clear separation of responsibilities between the coordinator and data nodes.
- Heartbeat Mechanism: Ensures live monitoring of data nodes.
- File Management: Allows file upload and retrieval.
- Extensible Protocol Buffers: Uses gRPC for communication, ensuring high performance and language interoperability.
Ensure maven is using the correct java version:
$ mvn -versionhas to print outJava version: 17...
Makefileto remove tedious chain of commands when building & running the projectDocker(In the future)
$ git clone https://github.com/Chapalax/thamuz.gitthamuz$ make dev-buildYou should do it in separate shells open so that logs of different services don't interleave
thamuz$ make dev-run-coordinatorEach data node should run on a different port. Non-provided GRPC_PORT variable will result to 9091.
thamuz$ make dev-run-datanode GRPC_PORT=<your_value>thamuz$ make dev-run-clientAt present, two functions are available: uploading and downloading. These allow users to save files on the server and to retrieve them, respectively.
-
Send the
uploadcommand. -
Provide the absolute path of an existing file on your computer.
-
Specify the absolute path where the file should be uploaded on the server, including the file extension.
-
Enjoy! 👀
-
Send the
downloadcommand. -
Provide the absolute path of the file located on the server, including the file extension.
-
Receive a message indicating the location of the downloaded file.
-
Enjoy! 👀
This is my first time using gRPC. Don't be too hard on me ^_^
List of available commands:
1) upload;
2) download.
Please enter the command:
upload
Enter the absolute path to the file you want to upload:
pom.xml
Enter the absolute path to the file you would like to create:
/tmp
/tmp
2025-01-24T04:29:56.471+03:00 INFO 40364 --- [grpc-client] [ main] r.u.sonus.service.FileProcessingService : SUCCESS File successfully uploaded.
The data nodes periodically send heartbeat messages to the coordinator to indicate they are active. If a node fails to send a heartbeat within a predefined interval, the coordinator marks the node as inactive.
For questions, suggestions, or issues, please create an issue on GitHub or contact the author directly.