huststore
is a open source high performance distributed database system. It not only provides key-value storage service with extremely high performance, up to 100 thousand QPS, but also supports data structures like hash
, set
, sorted set
, etc. Also, it can store binary data as value from a key-value pair, and thus can be used as an alternative of Redis.
In addtion, huststore
implements a distributed message queue by integrating a special HA
module, features including message Push Stream, and message Publish-SubScribe, these features can be used to as replacements of the corresponding features in rabbitmq and gearman.
huststore
has two core components, hustdb
and HA
. hustdb
uses fastdb
, a database engine developed by our own, in the fundamental architecture. HA
is implemented as a nginx
module. It is well-known that nginx
is a industry-proven high quality code base, thus by inheriting it huststore
gain the below advantages:
-
High Throughput
hustdb
uses libevhtp, a open source network library, as inner network communication system, by incorporating it withfastdb
storage engine,hustdb
achieves a extremely high performance, the benchmark shows thatQPS
hits 100 thousand and even more. -
High Concurrency
Please refer to concurrency report ofnginx
-
High Availability
huststore
architecture supportsReplication
(master-master),load balance
. Therefore, the availability ofHA
is guaranteed bymaster-worker
design. When one ofworker
process is down, themaster
will load anotherworkder
process, multipleworker
s work independently, therefore guarantee theHA
will work stably.
The fundamental design architecture ofhuststore
guarantees the high availability, by usingmaster-master
architecture, when one of the storage node fails,HA
module will re-direct the request to another livemaster
node. Also, when a node failure happens,HA
cluster will automatically re-balancing the data distribution, thus avoid single point of failure.
In addition,HA
cluster uses a distributed architecture design by incorporating LVS as the director, eachHA
node is separated and work independently. When one of theHA
node is down, LVS will re-direct the request to other availableHA
node, thus avoidsHA
's' failure on single point node. -
Language-free Interface
huststroe
usehttp
as the communication protocol, therefore the client side implementation is not limited in any specific programming language. -
Support Binary Key-Value
- Distributed KV storage = HA(hustdb ha) + DB(hustdb)
- Distributed Message Queue = HA(hustmq ha) + DB(hustdb)
Above includes detailed documents of design, deployment documents, API
usage and test samples. You can refer quickly to common problems in FAQ
part.
hustdb
doc
db
ha
sync
hustmq
doc
ha
hustdb/ha
provide service for storage engine, could configured with multi worker
.
hustmq/ha
provide service for message queue, can only configured with one worker
.
Machine Configuration: 24core, 64gb, 1tb sata(7200rpm)
Stress Test: 100 concurrent, 1000 thousand querys
DB CONF: single instance, thread model, 10 workers
Benchmark Result:
(1)PUT
<1>value: 256B; qps: 95 thousand
<2>value: 1KB; qps: 85 thousand
<3>value: 4KB; qps: 25 thousand
<4>value: 16KB; qps: 7 thousand
<5>value: 64KB; qps: 2 thousand
(2)GET
<1>value: 256B; qps: 100 thousand
<2>value: 1KB; qps: 10 thousand
<3>value: 4KB; qps: 25 thousand
<4>value: 16KB; qps: 7 thousand
<5>value: 64KB; qps: 2 thousand
(3)DEL
<1>value: 256B; qps: 100 thousand
<2>value: 1KB; qps: 100 thousand
<3>value: 4KB; qps: 100 thousand
<4>value: 16KB; qps: 100 thousand
<5>value: 64KB; qps: 100 thousand
Machine Configuration: 24core, 64gb, 1tb sata(7200rpm)
Stress Test Parameters: 100 concurrent, 1000 thousand querys, single queue
DB Configuration: single instance, thread model, 10 workers
Benchmark Result:
(1)PUT
<1>item: 256B; qps: 30 thousand
<2>item: 1KB; qps: 25 thousand
<3>item: 4KB; qps: 20 thousand
<4>item: 16KB; qps: 7 thousand
<5>item: 64KB; qps: 2 thousand
(2)GET
<1>item: 256B; qps: 25 thousand
<2>item: 1KB; qps: 20 thousand
<3>item: 4KB; qps: 18 thousand
<4>item: 16KB; qps: 7 thousand
<5>item: 64KB; qps: 2 thousand
huststore
is licensed under New BSD License, a very flexible license to use.
- XuRuibo(hustxrb, [email protected])
- ChengZhuo(jobs, [email protected])