Skip to content

Commit

Permalink
gRPC spec update: Add support for fetching temporal range of recordin…
Browse files Browse the repository at this point in the history
…gs (#9010)

### What

Adding support for fetching chunks that belong to specific range of the
recording. This consists of the 2 APIs calls:
* fetching chunk ids for a given range
* fetching chunks for the given chunk ids (which we might want to
leverage separately in the future)

As there is a limitation for the client side gRPC streaming on the web
(see
[here](#8877 (comment))),
we've decided to send a batch of chunk ids to the server. To make the
APIs somewhat symmetrical, the 1st API call will stream batches of chunk
ids.

Closes
* #8877
  • Loading branch information
zehiko authored and jprochazk committed Feb 13, 2025
1 parent 299a24c commit 8a9550f
Show file tree
Hide file tree
Showing 2 changed files with 237 additions and 0 deletions.
28 changes: 28 additions & 0 deletions crates/store/re_protos/proto/rerun/v0/remote_store.proto
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ service StorageNode {
rpc CreateIndex(CreateIndexRequest) returns (CreateIndexResponse) {}
rpc ReIndex(ReIndexRequest) returns (ReIndexResponse) {}

rpc GetChunkIds(GetChunkIdsRequest) returns (stream GetChunkIdsResponse) {}
rpc GetChunks(GetChunksRequest) returns (stream rerun.common.v0.RerunChunk) {}

// The response to `SearchIndex` a RecordBatch with 3 columns:
// - 'resource_id' column with the id of the resource
// - timepoint column with the values representing the points in time
Expand Down Expand Up @@ -46,6 +49,31 @@ message DataframePart {
bytes payload = 1000;
}

// ---------------- GetChunkIds ------------------

message GetChunkIdsRequest {
// recording id from which we're want to fetch the chunk ids
rerun.common.v0.RecordingId recording_id = 1;
// timeline for which we specify the time range
rerun.common.v0.IndexColumnSelector time_index = 2;
// time range for which we want to fetch the chunk ids
rerun.common.v0.TimeRange time_range = 3;
}

message GetChunkIdsResponse {
// a batch of chunk ids for chunks that are within the specified time range
repeated rerun.common.v0.Tuid chunk_ids = 1;
}

// ---------------- GetChunk ---------------------

message GetChunksRequest {
// recording id from which we're want to fetch the chunk ids
rerun.common.v0.RecordingId recording_id = 1;
// batch of chunk ids for which we want to stream back chunks
repeated rerun.common.v0.Tuid chunk_ids = 2;
}

// ---------------- CreateIndex ------------------

// used to define which column we want to index
Expand Down
209 changes: 209 additions & 0 deletions crates/store/re_protos/src/v0/rerun.remote_store.v0.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 8a9550f

Please sign in to comment.