Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation #126

Merged
merged 7 commits into from
Mar 7, 2025
Merged

[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation #126

merged 7 commits into from
Mar 7, 2025

Conversation

alogfans
Copy link
Collaborator

@alogfans alogfans commented Mar 3, 2025 •
edited by stmatengss
Loading

New feature: we implement a distributed object store for vllm xPyD scenarios. It follows the design outlined in this document (https://docs.google.com/document/d/1Ab6TMW1E2CdHJJyCrpJnLhgmE2b_6leH5MVP9k72sjw/edit?tab=t.0#heading=h.611v2r4aqubz). This PR is compatible with our proposal in the VLLM community (vllm-project/vllm#12957).

This is the refine of #114.

CC list: @james0zan @ShangmingCai @alogfans @doujiang24 @stmatengss

alogfans and others added 4 commits February 20, 2025 06:54
@alogfans
Squashed commit of the following:
943c3b6 
commit d67fe5ad48fdd92fb79623dc88def42d428ebedb
Author: Feng Ren <[email protected]>
Date:   Thu Feb 20 06:00:19 2025 +0000

    revise docs

commit 0b6a6545f048d1df70cd50a92869f0fce4785cf6
Author: Feng Ren <[email protected]>
Date:   Thu Feb 20 05:58:20 2025 +0000

    revise docs

commit 16f28ba99ed38da442483cfc57abd0b10a8991ae
Author: hearttosun <[email protected]>
Date:   Tue Feb 18 17:27:59 2025 +0800

    update doc

commit 92acbb38cd7e470dce259ff8e1762d0c59efe8c7
Author: hearttosun <[email protected]>
Date:   Tue Feb 18 16:56:50 2025 +0800

    update doc&proto

commit 00f54b369921964fa78609ed867e2d4923a1dcfe
Author: susun <[email protected]>
Date:   Tue Feb 18 15:32:29 2025 +0800

    Enhance documentation with detailed usage examples for Mooncake Store

commit 40280d1194bc576e6868911aed09fb063f0c0b43
Author: susun <[email protected]>
Date:   Tue Feb 18 12:09:08 2025 +0800

    rebase upstream

commit 9165f844756e0139e3401d28880733c6f2061f8e
Author: Feng Ren <[email protected]>
Date:   Tue Feb 18 02:04:06 2025 +0000

    add provider program to simulate multi machine communication

commit fcaf3be73f8aefb1e5d1dec7c6cd03acee1b1fee
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Feb 17 20:30:36 2025 +0800

    [FIX] change class name to fit python spec

commit 5917a703d0e013a597cb3bcf887338816b613601
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 06:07:01 2025 +0000

    rename

commit eb05c99442337d989eaf19a229be7dd74ef7d231
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 06:06:09 2025 +0000

    make local_buffer_size smaller in py test

commit dd12a0e2b858a49b5d5a57edaf2d4a4bb69869af
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 06:00:36 2025 +0000

    remove dead code

commit 989cb937659d9fd98401b36dc2b90866175f8497
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 02:54:23 2025 +0000

    avoid hardcode

commit 38e831e7bc3129b92b13221c9de6831e0f3c5cc9
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 02:40:53 2025 +0000

    check and update py program

commit 904b698b4f2929089667c890fdb08f5b8d409843
Author: hearttosun <[email protected]>
Date:   Fri Feb 14 17:29:20 2025 +0800

    code optimization

commit 8138bc83cd5a5d148c8103a38e2f986df7d6621f
Author: susun <[email protected]>
Date:   Thu Feb 13 18:38:47 2025 +0800

    fix memory management and error handling in distributed object store

commit 950193fe8fdf2eaf3db0fa234b34b8e721e4c65b
Author: susun <[email protected]>
Date:   Thu Feb 13 15:59:43 2025 +0800

    add E2E fuzz testing for python interface

commit 76f70d771a9abb9ab3af0f19a9ed9ccc4554a343
Author: susun <[email protected]>
Date:   Thu Feb 13 12:46:30 2025 +0800

    improve client code  readability

commit 2e53fc634898b0f3e0ce75d05c5a8c8f242cd44a
Author: susun <[email protected]>
Date:   Thu Feb 13 12:04:09 2025 +0800

    enhance MasterService method and error handling

commit e9dbdb1f79c065e6616f220e0f99fb18e3de2593
Author: susun <[email protected]>
Date:   Wed Feb 12 17:55:44 2025 +0800

    refactor logging and remove verbose flag

commit 3529593c2a8bcfbc49a18943388f6ac0092334ea
Author: susun <[email protected]>
Date:   Wed Feb 12 16:56:53 2025 +0800

    improve error handling in allocation

commit 332224cb0828e848981cee8395a1b9664d585eb1
Author: susun <[email protected]>
Date:   Wed Feb 12 15:48:23 2025 +0800

    add error handling for segment memory allocation

commit cb31395b66a0e8d3aeb1f0daa9bc17318be7e3de
Author: susun <[email protected]>
Date:   Wed Feb 12 14:06:58 2025 +0800

    python concurrent stress test

commit ce83e073b62552ebda6a8f9e5a83d9ac15708712
Author: susun <[email protected]>
Date:   Wed Feb 12 12:46:46 2025 +0800

    add basic python interfrace test

commit 0861c486c2e09802524e28cdd5d802b7bab1ed73
Author: Feng Ren <[email protected]>
Date:   Wed Feb 12 02:16:41 2025 +0000

    fix incorrect memcpy in rdma transport

commit 52856963afaadf33085264a81eb4eb67d3688d17
Author: Feng Ren <[email protected]>
Date:   Tue Feb 11 11:37:09 2025 +0000

    correct transfer write

commit c23651ef9da8c5019490e9275eec2712a5bd429f
Author: susun <[email protected]>
Date:   Tue Feb 11 12:45:43 2025 +0800

    fix buffer allocator test

commit cb1ead55cf325398df672c86119d4645797de4ea
Author: susun <[email protected]>
Date:   Tue Feb 11 12:42:45 2025 +0800

    add multi-slice, multi-replica test for MasterService

commit c62f02acf6c730d91cd2f45563233122031069ef
Author: susun <[email protected]>
Date:   Tue Feb 11 12:31:41 2025 +0800

    client side update slice length correctly

commit 08a74f423636df1b29b57ab14041b052c53b067a
Author: susun <[email protected]>
Date:   Tue Feb 11 12:31:13 2025 +0800

    support put with multiple slices

commit 90acbfe0b1fe88b50a10c79941a81fa89df597a2
Author: Feng Ren <[email protected]>
Date:   Tue Feb 11 03:28:38 2025 +0000

    add unit test for large allocation

commit 02d7bc00d97b77620440440bd13bb746ae9a15d4
Author: susun <[email protected]>
Date:   Mon Feb 10 18:13:45 2025 +0800

    fix the buffer size allocated in client testing.

commit 4c09ce41837a8b7596fdaafd1ba6c72f410958f1
Author: Feng Ren <[email protected]>
Date:   Mon Feb 10 08:57:03 2025 +0000

    move kMinSliceSize and kMaxSliceSize

commit fa9132fd159ac8da4023af6c7e8c6791d30f7128
Author: Feng Ren <[email protected]>
Date:   Mon Feb 10 07:31:34 2025 +0000

    support dynamically allocated buffer in distributed object store

commit 575a886e244c33185ab9047db2fd85782d56b484
Author: Feng Ren <[email protected]>
Date:   Mon Feb 10 06:14:05 2025 +0000

    fix allocator problem

commit a3c9fe89af9772646d66b380b467e3d5d434d121
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Feb 10 11:03:58 2025 +0800

    [MooncakeStore] Add remove interface and fix Get API
    Use 1MB buffer as an workaround to recv KV data (slice)

commit d8d94b7ad97ba88e60aacd9ce6479fc3a0607950
Author: Feng Ren <[email protected]>
Date:   Sat Feb 8 08:08:30 2025 +0000

    make kSize larger

commit e269070c298b82dd72ae20ee09da11938eacb89e
Author: Feng Ren <[email protected]>
Date:   Sat Feb 8 06:14:48 2025 +0000

    support adjustable data length

commit aa367bce86d8d3cd4f8d8e02370ba656fb8e2784
Author: Teng Ma (马腾) <[email protected]>
Date:   Fri Feb 7 17:25:44 2025 +0800

    [MooncakeStore][Fix]
    1. add more build steps
    2. support TCP python

commit 76ab10e3be314acf4a06b40d7c6bfaca4b0b9f69
Author: susun <[email protected]>
Date:   Fri Feb 7 14:55:01 2025 +0800

    optimize moonckae master's performance

commit ad31000290ea7bb6739fbeeb244d0b665a3f1806
Author: Teng Ma (马腾) <[email protected]>
Date:   Thu Feb 6 20:59:19 2025 +0800

    [MooncakeStore] vllm integration
    1. provide put/get interface
    2. class distributed object store
    3. modify CMake

commit e80a85e564af14d81c9f08cd3bb2f4e60c63334c
Author: Feng Ren <[email protected]>
Date:   Thu Feb 6 08:48:03 2025 +0000

    use tcp by default

commit 89a0cbb135cfe5efab3cd81a11685326ae42416f
Author: Feng Ren <[email protected]>
Date:   Thu Feb 6 08:47:32 2025 +0000

    fix stress workload to support multi thread

commit 605e303f6b592fafc39723a408bf611b428a4755
Author: susun <[email protected]>
Date:   Thu Feb 6 15:24:48 2025 +0800

    add get with multiple slices test

commit 64e5d23ba6263ccd687c127fc8572edae5dac516
Merge: 583aa60 d59cc58
Author: Feng Ren <[email protected]>
Date:   Wed Feb 5 11:18:58 2025 +0800

    Merge pull request kvcache-ai#38 from kvcache-ai/fix_slices_size

    Fix client_integration_test.cpp failed. Slice size was not updated.

commit d59cc58f063b931eb9bd2ae0dc1d18eacd372af4
Author: liusy58 <[email protected]>
Date:   Fri Jan 24 16:33:50 2025 +0800

    Fix client_integration_test.cpp failed. Slice size was not updated.

commit 583aa6056b3937a3aba19b0f7e02084a01e94af7
Author: Feng Ren <[email protected]>
Date:   Thu Jan 23 07:58:02 2025 +0000

    hide output in normal exit, and improve stress test result

commit c01d07ed474522f9538c7aad409a9edf4fd4227f
Author: Feng Ren <[email protected]>
Date:   Tue Jan 21 02:27:19 2025 +0000

    fix retry logics in data transfer

commit 12750b43c24274afab5080ee203f8def828fc4f6
Author: SuSun <[email protected]>
Date:   Mon Jan 20 12:37:16 2025 +0000

    fix stress test

commit 80a46241eb66d9326a645e11de7b2b803fbcef34
Author: SuSun <[email protected]>
Date:   Mon Jan 20 20:15:47 2025 +0800

    slices as client interface

commit 49bcb9cc74b255ffb41b7b0fb4145f288c398124
Author: Feng Ren <[email protected]>
Date:   Mon Jan 20 08:48:56 2025 +0000

    fix stress allocation test bug

commit b7383d5446f0107bcd8b9440cb56d53ed7d120a2
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Jan 20 16:02:48 2025 +0800

    [ObjectStore][Test] add stress test

commit e1efe5bb3b63ab2c8658ec3169c0bc84355e48c5
Author: SuSun <[email protected]>
Date:   Mon Jan 20 03:58:07 2025 +0000

    implement random allocation strategy

commit 5b0b8b25ed360cf8eb6527ba1305d2fb9008ef42
Author: SuSun <[email protected]>
Date:   Fri Jan 17 14:14:59 2025 +0000

    client use SimpleAllocator to manager buffer

commit 445054cace3f5b1269014777c7b14ee1b0671f09
Author: Feng Ren <[email protected]>
Date:   Fri Jan 17 02:04:11 2025 +0000

    register buffer when mounting

commit 55ce5643d0b202d6679fa4d4ab480cd63b106d0e
Author: SuSun <[email protected]>
Date:   Thu Jan 16 14:33:30 2025 +0000

    client LogAndCheckRpcStatus

commit e4a8975199f1f9d9867f9cd816e32636aa0f7535
Author: SuSun <[email protected]>
Date:   Thu Jan 16 12:17:06 2025 +0000

    remove useless codes

commit cd243a7537e4fba8be626065b8bc56696d7bc7e5
Author: SuSun <[email protected]>
Date:   Wed Jan 15 17:49:42 2025 +0000

    client suuport protocol and mount segment

commit 51b62af35dc101031ab2b7f0925b8bb972f3b621
Author: SuSun <[email protected]>
Date:   Wed Jan 15 11:55:53 2025 +0000

    execute Makefile before buid

commit 59578581de2bae594f401c55aab34f0f4c39b7c3
Author: Teng Ma (马腾) <[email protected]>
Date:   Wed Jan 15 11:49:20 2025 +0800

    [Build][Feature] Add mooncake store installation & simplify pybind11 detection in the root CMake

commit f0ecfe8620059f3f9802f874f1260f5ce4cb4a47
Author: SuSun <[email protected]>
Date:   Tue Jan 14 17:51:07 2025 +0000

    support AllocateTransferBuffer

commit 075e14559cd531c53614b61b1d3ed687f36fe6d1
Author: SuSun <[email protected]>
Date:   Tue Jan 14 14:46:37 2025 +0000

     Durint put, Get/Remove should fail

commit 16f8bda8eaa330ea7faf3bfae7951defe5c84fa3
Author: SuSun <[email protected]>
Date:   Tue Jan 14 14:34:23 2025 +0000

    refine master service error code

commit e39494096af911a8060c53ad833e932b72b69e61
Author: SuSun <[email protected]>
Date:   Tue Jan 14 13:44:33 2025 +0000

    refine error code

commit 51e60f5cc8bd8cfbe3a0c542876e583e1deb5fb2
Author: SuSun <[email protected]>
Date:   Mon Jan 13 17:08:23 2025 +0000

    merge upstream code

commit 08c6f306ab83f06f26d6d520c9f250d55b0165f8
Author: xiaguan <[email protected]>
Date:   Tue Jan 14 00:56:12 2025 +0800

    upstream

commit 45c8b17044f918f5094a350459eceb70f89dba91
Author: SuSun <[email protected]>
Date:   Mon Jan 13 16:51:45 2025 +0000

    Protobuf generation order in CMake

commit 260acd9b13204f003cce787ccb2d20f385e81c5a
Author: Feng Ren <[email protected]>
Date:   Mon Jan 13 07:44:19 2025 +0000

    make unit test work in rdma network

commit 99a5dc65021f5ac1b2a5c4d8cef508005f4a2a10
Author: SuSun <[email protected]>
Date:   Sun Jan 12 17:19:05 2025 +0000

    add more test

commit 939bc8e2afa6f917a22a4e2a7eda7fc45f8991f0
Author: SuSun <[email protected]>
Date:   Sun Jan 12 15:02:31 2025 +0000

    refine client side logging

commit 300a944d53b1d77f70ef4aa5277a66849075c570
Author: SuSun <[email protected]>
Date:   Sun Jan 12 13:54:47 2025 +0000

    refine master logging

commit c04851fc5f293293584973fa4a34aed550c67342
Author: SuSun <[email protected]>
Date:   Sat Jan 11 12:07:55 2025 +0000

    Hello, World!

commit 37ccf5895c2eba3b2f1a0cbd69953e933435fe98
Author: susun <[email protected]>
Date:   Fri Jan 10 21:35:09 2025 +0800

    integration test

commit 2ba389abf480a9083e5cf0ff2513d76000ac519d
Author: susun <[email protected]>
Date:   Fri Jan 10 16:33:29 2025 +0800

    add integration test, need fix allocate logic

commit 78b053db6222d175af8dea1749d456b8607f7c9f
Author: susun <[email protected]>
Date:   Fri Jan 10 11:34:44 2025 +0800

    gen grpc c++ code makefile

commit 465fc4d038d3bb345b7e64c0d949d9c97dcddaa4
Author: xiaguan <[email protected]>
Date:   Sun Jan 5 02:23:11 2025 +0800

    implment master rpc service

commit 9f168363707fe6e4b1e94d9de4970cd86c64d723
Author: xiaguan <[email protected]>
Date:   Thu Jan 2 06:58:35 2025 +0800

    start to implement client

commit 14382b22f8ec2affd679469eeb457b43a5b0d543
Author: xiaguan <[email protected]>
Date:   Mon Dec 30 05:44:19 2024 +0800

    basic master service impl

commit 1e0ca275562dd0ca77ab31591504c137cc5cad78
Author: Feng Ren <[email protected]>
Date:   Wed Dec 25 09:52:54 2024 +0800

    Update mooncake-store-simple.md

commit 8eafbbccca63e1aa9d4d226ca7a47caf2c5811b8
Author: xiaguan <[email protected]>
Date:   Tue Dec 24 22:09:09 2024 +0800

    refine mooncake store simple design doc

commit 12516717db6aca383797422f3d2d69062df70f8e
Author: xiaguan <[email protected]>
Date:   Fri Dec 20 21:49:46 2024 +0800

    refine doc

commit f5229d9483d8ffd1b82cca451c948210c0a291cb
Author: Feng Ren <[email protected]>
Date:   Fri Dec 20 06:28:12 2024 +0000

    revise doc for adding contents

commit 43f185883d2cb8c746f473ccb68ba58cf1f8e8b0
Author: Feng Ren <[email protected]>
Date:   Fri Dec 20 04:17:27 2024 +0000

    revise doc

commit 2007ef1cac5f4a33fbb18d83ac54bbba501d31d6
Author: xiaguan <[email protected]>
Date:   Thu Dec 19 23:11:37 2024 +0800

    finish basic mooncake store simple design doc

commit af92393
Author: xiaguan <[email protected]>
Date:   Wed Dec 18 22:53:01 2024 +0800

    impl simple controller

commit f9596fe
Author: xiaguan <[email protected]>
Date:   Mon Dec 16 22:23:44 2024 +0800

    cmkae compile done, need fix test

commit 31d05c9
Author: xiaguan <[email protected]>
Date:   Mon Dec 16 21:12:05 2024 +0800

    copy allocator, need fix cmake arch

commit 0ebff6f
Author: xiaguan <[email protected]>
Date:   Sat Dec 14 19:29:48 2024 +0800

    finish

commit aa49ead
Author: xiaguan <[email protected]>
Date:   Sat Dec 14 17:10:06 2024 +0800

    update DistributedObjectStore

commit 7d40935
Author: xiaguan <[email protected]>
Date:   Thu Dec 12 23:09:17 2024 +0800

    update arch picture

commit ec7bcff
Author: xiaguan <[email protected]>
Date:   Thu Dec 12 22:25:51 2024 +0800

    init
@alogfans
add docs for guide
1c12cf6
@alogfans
Squashed commit of the following:
7760ff2 
commit 542f72d9e9fd5e090efc065764adffae2436fc0d
Author: Teng Ma <[email protected]>
Date:   Mon Mar 3 17:37:28 2025 +0800

    [MooncakeStore] refactor: use same cmake_minimum_required

commit 4be5e6312d7e7a7875786d2d5bee6e144f450168
Author: Shangming Cai <[email protected]>
Date:   Mon Mar 3 17:29:49 2025 +0800

    [Doc] update preview note since master bug is fixed.

    Signed-off-by: Shangming Cai <[email protected]>

commit 4fb4302bcf5e5b7096174bac02682d4b6c962b55
Author: susun <[email protected]>
Date:   Mon Mar 3 16:20:31 2025 +0800

    feat: implement ResourceTracker for resource cleanup on termination

commit 1052c2b06046d8ebb645d23ca0996cc3128b2c5a
Author: Shangming Cai <[email protected]>
Date:   Mon Mar 3 15:06:14 2025 +0800

    add mooncake_master installation path

commit 191212510f12f322cada37dc47bff3cd27f9df9b
Author: Shangming Cai <[email protected]>
Date:   Thu Feb 27 15:48:34 2025 +0800

    refactor: change the name

commit d839d3ff24f9efbf61776e7b3175d962203c9039
Author: susun <[email protected]>
Date:   Thu Feb 27 15:27:11 2025 +0800

    Update UnInit method usage in destructor and teardown methods

commit 8df63f294ccba6effe47ece852fef8def8f51749
Author: susun <[email protected]>
Date:   Wed Feb 26 22:13:59 2025 +0800

    add cleanup stale handles support in master service

commit c09a3b33b8e85a00d85625016fcb5fb1f6ff9818
Author: Shangming Cai <[email protected]>
Date:   Wed Feb 26 18:45:23 2025 +0800

    Add preview vllm benchmark with MooncakeStore.

    Signed-off-by: Shangming Cai <[email protected]>

commit fd1fa7e02453496c460a5cd6f4d10914f387c696
Merge: 4229ca7 d66ff94
Author: Feng Ren <[email protected]>
Date:   Wed Feb 26 16:42:01 2025 +0800

    Merge pull request kvcache-ai#39 from kvcache-ai/master_test

    Add tests for master_service_test.cpp

commit d66ff94f616f0e4631a6a1bd1e4f57f260bcbcaa
Author: liusy58 <[email protected]>
Date:   Wed Feb 26 16:28:47 2025 +0800

    Add a note to Mooncake store README

commit 4229ca70178065a43c16aa87a4dcb705425c6174
Author: susun <[email protected]>
Date:   Wed Feb 26 16:16:50 2025 +0800

    Add port availability check and improve server startup error handling

commit a124e2da807753ec09151ca26ae886a244573dea
Author: liusy58 <[email protected]>
Date:   Wed Feb 26 14:28:01 2025 +0800

    Add tests for master_service_test.cpp

commit 37c51233787825b4dd12461233e74d041c94f855
Author: Feng Ren <[email protected]>
Date:   Tue Feb 25 13:29:55 2025 +0000

    update stress benchmark

commit 561bd41afe16e4b7eace395cd3bbbf50c28b6893
Author: Feng Ren <[email protected]>
Date:   Tue Feb 25 11:16:45 2025 +0000

    add Client::UnInit

commit c4f5568f3df782637a15b4cbf168cd86963546fc
Author: Feng Ren <[email protected]>
Date:   Tue Feb 25 08:03:35 2025 +0000

    revise doc

commit 86c66a3514ec40f3ef695727b58d8cf483bc3202
Author: Feng Ren <[email protected]>
Date:   Tue Feb 25 08:02:56 2025 +0000

    revise doc

commit 8acc503125c0660a38a184663f5eda6116828806
Author: susun <[email protected]>
Date:   Tue Feb 25 15:06:21 2025 +0800

    Implement Garbage Collection Mechanism for MasterService

commit 06b6e1e6201acf2df460cae846726d965dcb4e3f
Author: Feng Ren <[email protected]>
Date:   Tue Feb 25 06:12:26 2025 +0000

    add stress benchmark code and reduce output for invalid segments (trying to retrieve data from closed instance)

commit 416c68efaadc122d2169b3cecfe0c17f8fa0ebe2
Author: Feng Ren <[email protected]>
Date:   Tue Feb 25 03:03:42 2025 +0000

    add put revoke rpc to handle data plane error

commit 94961828b777dd3951fce5fb2701b88590a2f7cf
Author: susun <[email protected]>
Date:   Mon Feb 24 21:56:37 2025 +0800

    Update class name in distributed object store tests

commit 445e7180cfde1683ea611fbd95eed25618ac8f27
Author: susun <[email protected]>
Date:   Mon Feb 24 21:52:33 2025 +0800

    enhance error handling and port management

commit 0f13d69474b3b1cf4d2275adc06fafa7f6480842
Author: susun <[email protected]>
Date:   Mon Feb 24 17:56:28 2025 +0800

    Add isExist method to distributed object store

commit e88eb8de79f621916a23df4cd4effec5f94a70d9
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Feb 24 17:34:14 2025 +0800

    [Mooncake Store] fix name and add new teardown op

commit 540e16dcf93dc368a0f2c98ba94d59d6224b5747
Author: susun <[email protected]>
Date:   Mon Feb 24 16:09:59 2025 +0800

    Implement dynamic port selection for distributed object store

commit 2c5907f76033e1f0f5a19d5211410e4eb01f90c5
Author: Feng Ren <[email protected]>
Date:   Mon Feb 24 06:10:19 2025 +0000

    remove invalid arguments in doc

commit ceba66f851db128658137513392bc983c69521cb
Author: Feng Ren <[email protected]>
Date:   Mon Feb 24 02:28:44 2025 +0000

    migrate mooncake store integration test path

commit 92eec24bb8684156fcbe36bf6b803438fcaa4109
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Feb 24 10:03:40 2025 +0800

    [MooncakeStore] Rename class name and pack multiple cpython libs into one

commit 7e071a36891824883cb0ef45913634369ed7fb8f
Author: Teng Ma (马腾) <[email protected]>
Date:   Sun Feb 23 21:57:07 2025 +0800

    [MooncakeStore] integrate mooncake store vllm adapter to mooncake-integration dir

commit 4ac55fd0b2bc81b67386d0c76e2b3bd7d9449d86
Author: Feng Ren <[email protected]>
Date:   Fri Feb 21 11:01:22 2025 +0000

    fix errors in mooncake store docs

commit 17fc2f33f742e5126660192a4d69d4936567f387
Author: Feng Ren <[email protected]>
Date:   Fri Feb 21 06:42:17 2025 +0000

    update readme for mooncake store

commit b7023b8bd4fd96395832686662797edee48759fa
Author: Feng Ren <[email protected]>
Date:   Fri Feb 21 05:08:54 2025 +0000

    update readme for mooncake store

commit 0b32e1135ef6351185b32ec25ed339a995ad0748
Author: Feng Ren <[email protected]>
Date:   Fri Feb 21 03:32:48 2025 +0000

    update readme for mooncake store

commit b10c3c05dac34c5f67ac759e063d20866d477dec
Author: Feng Ren <[email protected]>
Date:   Fri Feb 21 03:17:26 2025 +0000

    update readme

commit d67fe5ad48fdd92fb79623dc88def42d428ebedb
Author: Feng Ren <[email protected]>
Date:   Thu Feb 20 06:00:19 2025 +0000

    revise docs

commit 0b6a6545f048d1df70cd50a92869f0fce4785cf6
Author: Feng Ren <[email protected]>
Date:   Thu Feb 20 05:58:20 2025 +0000

    revise docs

commit 16f28ba99ed38da442483cfc57abd0b10a8991ae
Author: hearttosun <[email protected]>
Date:   Tue Feb 18 17:27:59 2025 +0800

    update doc

commit 92acbb38cd7e470dce259ff8e1762d0c59efe8c7
Author: hearttosun <[email protected]>
Date:   Tue Feb 18 16:56:50 2025 +0800

    update doc&proto

commit 00f54b369921964fa78609ed867e2d4923a1dcfe
Author: susun <[email protected]>
Date:   Tue Feb 18 15:32:29 2025 +0800

    Enhance documentation with detailed usage examples for Mooncake Store

commit 40280d1194bc576e6868911aed09fb063f0c0b43
Author: susun <[email protected]>
Date:   Tue Feb 18 12:09:08 2025 +0800

    rebase upstream

commit 9165f844756e0139e3401d28880733c6f2061f8e
Author: Feng Ren <[email protected]>
Date:   Tue Feb 18 02:04:06 2025 +0000

    add provider program to simulate multi machine communication

commit fcaf3be73f8aefb1e5d1dec7c6cd03acee1b1fee
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Feb 17 20:30:36 2025 +0800

    [FIX] change class name to fit python spec

commit 5917a703d0e013a597cb3bcf887338816b613601
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 06:07:01 2025 +0000

    rename

commit eb05c99442337d989eaf19a229be7dd74ef7d231
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 06:06:09 2025 +0000

    make local_buffer_size smaller in py test

commit dd12a0e2b858a49b5d5a57edaf2d4a4bb69869af
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 06:00:36 2025 +0000

    remove dead code

commit 989cb937659d9fd98401b36dc2b90866175f8497
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 02:54:23 2025 +0000

    avoid hardcode

commit 38e831e7bc3129b92b13221c9de6831e0f3c5cc9
Author: Feng Ren <[email protected]>
Date:   Mon Feb 17 02:40:53 2025 +0000

    check and update py program

commit 904b698b4f2929089667c890fdb08f5b8d409843
Author: hearttosun <[email protected]>
Date:   Fri Feb 14 17:29:20 2025 +0800

    code optimization

commit 8138bc83cd5a5d148c8103a38e2f986df7d6621f
Author: susun <[email protected]>
Date:   Thu Feb 13 18:38:47 2025 +0800

    fix memory management and error handling in distributed object store

commit 950193fe8fdf2eaf3db0fa234b34b8e721e4c65b
Author: susun <[email protected]>
Date:   Thu Feb 13 15:59:43 2025 +0800

    add E2E fuzz testing for python interface

commit 76f70d771a9abb9ab3af0f19a9ed9ccc4554a343
Author: susun <[email protected]>
Date:   Thu Feb 13 12:46:30 2025 +0800

    improve client code  readability

commit 2e53fc634898b0f3e0ce75d05c5a8c8f242cd44a
Author: susun <[email protected]>
Date:   Thu Feb 13 12:04:09 2025 +0800

    enhance MasterService method and error handling

commit e9dbdb1f79c065e6616f220e0f99fb18e3de2593
Author: susun <[email protected]>
Date:   Wed Feb 12 17:55:44 2025 +0800

    refactor logging and remove verbose flag

commit 3529593c2a8bcfbc49a18943388f6ac0092334ea
Author: susun <[email protected]>
Date:   Wed Feb 12 16:56:53 2025 +0800

    improve error handling in allocation

commit 332224cb0828e848981cee8395a1b9664d585eb1
Author: susun <[email protected]>
Date:   Wed Feb 12 15:48:23 2025 +0800

    add error handling for segment memory allocation

commit cb31395b66a0e8d3aeb1f0daa9bc17318be7e3de
Author: susun <[email protected]>
Date:   Wed Feb 12 14:06:58 2025 +0800

    python concurrent stress test

commit ce83e073b62552ebda6a8f9e5a83d9ac15708712
Author: susun <[email protected]>
Date:   Wed Feb 12 12:46:46 2025 +0800

    add basic python interfrace test

commit 0861c486c2e09802524e28cdd5d802b7bab1ed73
Author: Feng Ren <[email protected]>
Date:   Wed Feb 12 02:16:41 2025 +0000

    fix incorrect memcpy in rdma transport

commit 52856963afaadf33085264a81eb4eb67d3688d17
Author: Feng Ren <[email protected]>
Date:   Tue Feb 11 11:37:09 2025 +0000

    correct transfer write

commit c23651ef9da8c5019490e9275eec2712a5bd429f
Author: susun <[email protected]>
Date:   Tue Feb 11 12:45:43 2025 +0800

    fix buffer allocator test

commit cb1ead55cf325398df672c86119d4645797de4ea
Author: susun <[email protected]>
Date:   Tue Feb 11 12:42:45 2025 +0800

    add multi-slice, multi-replica test for MasterService

commit c62f02acf6c730d91cd2f45563233122031069ef
Author: susun <[email protected]>
Date:   Tue Feb 11 12:31:41 2025 +0800

    client side update slice length correctly

commit 08a74f423636df1b29b57ab14041b052c53b067a
Author: susun <[email protected]>
Date:   Tue Feb 11 12:31:13 2025 +0800

    support put with multiple slices

commit 90acbfe0b1fe88b50a10c79941a81fa89df597a2
Author: Feng Ren <[email protected]>
Date:   Tue Feb 11 03:28:38 2025 +0000

    add unit test for large allocation

commit 02d7bc00d97b77620440440bd13bb746ae9a15d4
Author: susun <[email protected]>
Date:   Mon Feb 10 18:13:45 2025 +0800

    fix the buffer size allocated in client testing.

commit 4c09ce41837a8b7596fdaafd1ba6c72f410958f1
Author: Feng Ren <[email protected]>
Date:   Mon Feb 10 08:57:03 2025 +0000

    move kMinSliceSize and kMaxSliceSize

commit fa9132fd159ac8da4023af6c7e8c6791d30f7128
Author: Feng Ren <[email protected]>
Date:   Mon Feb 10 07:31:34 2025 +0000

    support dynamically allocated buffer in distributed object store

commit 575a886e244c33185ab9047db2fd85782d56b484
Author: Feng Ren <[email protected]>
Date:   Mon Feb 10 06:14:05 2025 +0000

    fix allocator problem

commit a3c9fe89af9772646d66b380b467e3d5d434d121
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Feb 10 11:03:58 2025 +0800

    [MooncakeStore] Add remove interface and fix Get API
    Use 1MB buffer as an workaround to recv KV data (slice)

commit d8d94b7ad97ba88e60aacd9ce6479fc3a0607950
Author: Feng Ren <[email protected]>
Date:   Sat Feb 8 08:08:30 2025 +0000

    make kSize larger

commit e269070c298b82dd72ae20ee09da11938eacb89e
Author: Feng Ren <[email protected]>
Date:   Sat Feb 8 06:14:48 2025 +0000

    support adjustable data length

commit aa367bce86d8d3cd4f8d8e02370ba656fb8e2784
Author: Teng Ma (马腾) <[email protected]>
Date:   Fri Feb 7 17:25:44 2025 +0800

    [MooncakeStore][Fix]
    1. add more build steps
    2. support TCP python

commit 76ab10e3be314acf4a06b40d7c6bfaca4b0b9f69
Author: susun <[email protected]>
Date:   Fri Feb 7 14:55:01 2025 +0800

    optimize moonckae master's performance

commit ad31000290ea7bb6739fbeeb244d0b665a3f1806
Author: Teng Ma (马腾) <[email protected]>
Date:   Thu Feb 6 20:59:19 2025 +0800

    [MooncakeStore] vllm integration
    1. provide put/get interface
    2. class distributed object store
    3. modify CMake

commit e80a85e564af14d81c9f08cd3bb2f4e60c63334c
Author: Feng Ren <[email protected]>
Date:   Thu Feb 6 08:48:03 2025 +0000

    use tcp by default

commit 89a0cbb135cfe5efab3cd81a11685326ae42416f
Author: Feng Ren <[email protected]>
Date:   Thu Feb 6 08:47:32 2025 +0000

    fix stress workload to support multi thread

commit 605e303f6b592fafc39723a408bf611b428a4755
Author: susun <[email protected]>
Date:   Thu Feb 6 15:24:48 2025 +0800

    add get with multiple slices test

commit 64e5d23ba6263ccd687c127fc8572edae5dac516
Merge: 583aa60 d59cc58
Author: Feng Ren <[email protected]>
Date:   Wed Feb 5 11:18:58 2025 +0800

    Merge pull request kvcache-ai#38 from kvcache-ai/fix_slices_size

    Fix client_integration_test.cpp failed. Slice size was not updated.

commit d59cc58f063b931eb9bd2ae0dc1d18eacd372af4
Author: liusy58 <[email protected]>
Date:   Fri Jan 24 16:33:50 2025 +0800

    Fix client_integration_test.cpp failed. Slice size was not updated.

commit 583aa6056b3937a3aba19b0f7e02084a01e94af7
Author: Feng Ren <[email protected]>
Date:   Thu Jan 23 07:58:02 2025 +0000

    hide output in normal exit, and improve stress test result

commit c01d07ed474522f9538c7aad409a9edf4fd4227f
Author: Feng Ren <[email protected]>
Date:   Tue Jan 21 02:27:19 2025 +0000

    fix retry logics in data transfer

commit 12750b43c24274afab5080ee203f8def828fc4f6
Author: SuSun <[email protected]>
Date:   Mon Jan 20 12:37:16 2025 +0000

    fix stress test

commit 80a46241eb66d9326a645e11de7b2b803fbcef34
Author: SuSun <[email protected]>
Date:   Mon Jan 20 20:15:47 2025 +0800

    slices as client interface

commit 49bcb9cc74b255ffb41b7b0fb4145f288c398124
Author: Feng Ren <[email protected]>
Date:   Mon Jan 20 08:48:56 2025 +0000

    fix stress allocation test bug

commit b7383d5446f0107bcd8b9440cb56d53ed7d120a2
Author: Teng Ma (马腾) <[email protected]>
Date:   Mon Jan 20 16:02:48 2025 +0800

    [ObjectStore][Test] add stress test

commit e1efe5bb3b63ab2c8658ec3169c0bc84355e48c5
Author: SuSun <[email protected]>
Date:   Mon Jan 20 03:58:07 2025 +0000

    implement random allocation strategy

commit 5b0b8b25ed360cf8eb6527ba1305d2fb9008ef42
Author: SuSun <[email protected]>
Date:   Fri Jan 17 14:14:59 2025 +0000

    client use SimpleAllocator to manager buffer

commit 445054cace3f5b1269014777c7b14ee1b0671f09
Author: Feng Ren <[email protected]>
Date:   Fri Jan 17 02:04:11 2025 +0000

    register buffer when mounting

commit 55ce5643d0b202d6679fa4d4ab480cd63b106d0e
Author: SuSun <[email protected]>
Date:   Thu Jan 16 14:33:30 2025 +0000

    client LogAndCheckRpcStatus

commit e4a8975199f1f9d9867f9cd816e32636aa0f7535
Author: SuSun <[email protected]>
Date:   Thu Jan 16 12:17:06 2025 +0000

    remove useless codes

commit cd243a7537e4fba8be626065b8bc56696d7bc7e5
Author: SuSun <[email protected]>
Date:   Wed Jan 15 17:49:42 2025 +0000

    client suuport protocol and mount segment

commit 51b62af35dc101031ab2b7f0925b8bb972f3b621
Author: SuSun <[email protected]>
Date:   Wed Jan 15 11:55:53 2025 +0000

    execute Makefile before buid

commit 59578581de2bae594f401c55aab34f0f4c39b7c3
Author: Teng Ma (马腾) <[email protected]>
Date:   Wed Jan 15 11:49:20 2025 +0800

    [Build][Feature] Add mooncake store installation & simplify pybind11 detection in the root CMake

commit f0ecfe8620059f3f9802f874f1260f5ce4cb4a47
Author: SuSun <[email protected]>
Date:   Tue Jan 14 17:51:07 2025 +0000

    support AllocateTransferBuffer

commit 075e14559cd531c53614b61b1d3ed687f36fe6d1
Author: SuSun <[email protected]>
Date:   Tue Jan 14 14:46:37 2025 +0000

     Durint put, Get/Remove should fail

commit 16f8bda8eaa330ea7faf3bfae7951defe5c84fa3
Author: SuSun <[email protected]>
Date:   Tue Jan 14 14:34:23 2025 +0000

    refine master service error code

commit e39494096af911a8060c53ad833e932b72b69e61
Author: SuSun <[email protected]>
Date:   Tue Jan 14 13:44:33 2025 +0000

    refine error code

commit 51e60f5cc8bd8cfbe3a0c542876e583e1deb5fb2
Author: SuSun <[email protected]>
Date:   Mon Jan 13 17:08:23 2025 +0000

    merge upstream code

commit 08c6f306ab83f06f26d6d520c9f250d55b0165f8
Author: xiaguan <[email protected]>
Date:   Tue Jan 14 00:56:12 2025 +0800

    upstream

commit 45c8b17044f918f5094a350459eceb70f89dba91
Author: SuSun <[email protected]>
Date:   Mon Jan 13 16:51:45 2025 +0000

    Protobuf generation order in CMake

commit 260acd9b13204f003cce787ccb2d20f385e81c5a
Author: Feng Ren <[email protected]>
Date:   Mon Jan 13 07:44:19 2025 +0000

    make unit test work in rdma network

commit 99a5dc65021f5ac1b2a5c4d8cef508005f4a2a10
Author: SuSun <[email protected]>
Date:   Sun Jan 12 17:19:05 2025 +0000

    add more test

commit 939bc8e2afa6f917a22a4e2a7eda7fc45f8991f0
Author: SuSun <[email protected]>
Date:   Sun Jan 12 15:02:31 2025 +0000

    refine client side logging

commit 300a944d53b1d77f70ef4aa5277a66849075c570
Author: SuSun <[email protected]>
Date:   Sun Jan 12 13:54:47 2025 +0000

    refine master logging

commit c04851fc5f293293584973fa4a34aed550c67342
Author: SuSun <[email protected]>
Date:   Sat Jan 11 12:07:55 2025 +0000

    Hello, World!

commit 37ccf5895c2eba3b2f1a0cbd69953e933435fe98
Author: susun <[email protected]>
Date:   Fri Jan 10 21:35:09 2025 +0800

    integration test

commit 2ba389abf480a9083e5cf0ff2513d76000ac519d
Author: susun <[email protected]>
Date:   Fri Jan 10 16:33:29 2025 +0800

    add integration test, need fix allocate logic

commit 78b053db6222d175af8dea1749d456b8607f7c9f
Author: susun <[email protected]>
Date:   Fri Jan 10 11:34:44 2025 +0800

    gen grpc c++ code makefile

commit 465fc4d038d3bb345b7e64c0d949d9c97dcddaa4
Author: xiaguan <[email protected]>
Date:   Sun Jan 5 02:23:11 2025 +0800

    implment master rpc service

commit 9f168363707fe6e4b1e94d9de4970cd86c64d723
Author: xiaguan <[email protected]>
Date:   Thu Jan 2 06:58:35 2025 +0800

    start to implement client

commit 14382b22f8ec2affd679469eeb457b43a5b0d543
Author: xiaguan <[email protected]>
Date:   Mon Dec 30 05:44:19 2024 +0800

    basic master service impl

commit 1e0ca275562dd0ca77ab31591504c137cc5cad78
Author: Feng Ren <[email protected]>
Date:   Wed Dec 25 09:52:54 2024 +0800

    Update mooncake-store-simple.md

commit 8eafbbccca63e1aa9d4d226ca7a47caf2c5811b8
Author: xiaguan <[email protected]>
Date:   Tue Dec 24 22:09:09 2024 +0800

    refine mooncake store simple design doc

commit 12516717db6aca383797422f3d2d69062df70f8e
Author: xiaguan <[email protected]>
Date:   Fri Dec 20 21:49:46 2024 +0800

    refine doc

commit f5229d9483d8ffd1b82cca451c948210c0a291cb
Author: Feng Ren <[email protected]>
Date:   Fri Dec 20 06:28:12 2024 +0000

    revise doc for adding contents

commit 43f185883d2cb8c746f473ccb68ba58cf1f8e8b0
Author: Feng Ren <[email protected]>
Date:   Fri Dec 20 04:17:27 2024 +0000

    revise doc

commit 2007ef1cac5f4a33fbb18d83ac54bbba501d31d6
Author: xiaguan <[email protected]>
Date:   Thu Dec 19 23:11:37 2024 +0800

    finish basic mooncake store simple design doc

commit af92393
Author: xiaguan <[email protected]>
Date:   Wed Dec 18 22:53:01 2024 +0800

    impl simple controller

commit f9596fe
Author: xiaguan <[email protected]>
Date:   Mon Dec 16 22:23:44 2024 +0800

    cmkae compile done, need fix test

commit 31d05c9
Author: xiaguan <[email protected]>
Date:   Mon Dec 16 21:12:05 2024 +0800

    copy allocator, need fix cmake arch

commit 0ebff6f
Author: xiaguan <[email protected]>
Date:   Sat Dec 14 19:29:48 2024 +0800

    finish

commit aa49ead
Author: xiaguan <[email protected]>
Date:   Sat Dec 14 17:10:06 2024 +0800

    update DistributedObjectStore

commit 7d40935
Author: xiaguan <[email protected]>
Date:   Thu Dec 12 23:09:17 2024 +0800

    update arch picture

commit ec7bcff
Author: xiaguan <[email protected]>
Date:   Thu Dec 12 22:25:51 2024 +0800

    init
@stmatengss
Merge branch 'kvcache-ai:main' into add-mooncake-store
ab45c7b
@ShangmingCai
Copy link
Collaborator

Will finish the review tomorrow.

@stmatengss
Copy link
Collaborator

@alogfans @ShangmingCai, if this PR is approved, I will squash and merge it, rewrite the commit log, and add a co-author list.

@ShangmingCai
Copy link
Collaborator

@alogfans @ShangmingCai, if this PR is approved, I will squash and merge it, rewrite the commit log, and add a co-author list.

Application level tests have passed, and I am now in the final round of overall review. I will ping you when it is completed.

ShangmingCai
ShangmingCai approved these changes Mar 4, 2025
View reviewed changes
Copy link
Collaborator

@ShangmingCai ShangmingCai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@alogfans
Copy link
Collaborator Author

alogfans commented Mar 5, 2025

LGTM

@stmatengss stmatengss merged commit ac98ea9 into kvcache-ai:main Mar 7, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stmatengss stmatengss stmatengss approved these changes

@ShangmingCai ShangmingCai ShangmingCai approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alogfans @ShangmingCai @stmatengss