gRPC: download only specific time point or range

[emilk]

When looking at a recording, I want to be able to send a link to a colleague pointing them at a specific time point or time range (a point of interest). This should generate a rerun:// link on a format similar to this: rerun://ip:port/store/ID?log_time=2025-01-31T02:55:20Z...2025-01-31T02:58:40Z

• See Define and implement gRPC URI structure #8876 for discussions around specific URL format

gRPC protocol

I suggest we implement this by doing a latest-at or range-query on the server to find all chunk IDs that the viewer would need to show the given time range, and return their IDs. A separate rpc call would be used to then fetch the chunks (rpc get_chunks(StoreId, stream ChunkId) -> stream Chunk).

There are two big advantages to this approach

• The server doesn't have to do expensive chunk slicing (saving CPU and memory)
• We can reuse the same rpc calls to implement larger-than-RAM (on-demand streaming)

The disadvantage is a potential waste of bandwidth (streaming parts of chunks that are outside the time range). This is mitigated by limiting the sizes of chunks during compaction.

Viewer

When opening such a URI, we also set the timeline and time cursor according to the query parameter. If it is a single time point, just use it.
If it is a range, use the start of the range, and set the loop-range to the query range, and enable looping. Also zoom the time panel so that the time range fills the width of the screen.

Related issues

• Define and implement gRPC URI structure #8876

[jprochazk]

Note that RPCs in the form of

rpc F(stream Request) (Response) {}
rpc F(stream Request) (stream Response) {}

are not supported by grpc-web at the moment, so are a non-starter. The blocker on their side is general availability of the WebTransport API... which will take a while.

[emilk]

@jprochazk Can you elaborate please? What is not supported? Streaming parameters? Is there some issue or discussion you can link to?

I guess your comment is about my proposed rpc get_chunks(StoreId, stream ChunkId) -> stream Chunk, but this could just as well be a rpc get_chunk(StoreId, ChunkId) -> Chunk (which I assume grpc-web supports), so afaict this is not really blocking this issue?

[jprochazk]

Issue in grpc-web: grpc/grpc-web#24

We talked about this briefly in Slack, and we can likely do something like

rpc GetChunks(StoreId, ChunkIdSet) returns (stream RerunChunk) {}

instead, to amortize protocol overhead.