From ba3261f38ebc7341bddef3aa6ae2f2f2a0c0f849 Mon Sep 17 00:00:00 2001 From: Chris Hager Date: Thu, 13 Jun 2024 17:56:57 +0200 Subject: [PATCH] bidcollect docs update --- docs/2024-06_bidcollect.md | 57 ++++++++++++++----- .../bidcollect/website/templates/base.html | 3 +- 2 files changed, 44 insertions(+), 16 deletions(-) diff --git a/docs/2024-06_bidcollect.md b/docs/2024-06_bidcollect.md index b499f24..81cfc37 100644 --- a/docs/2024-06_bidcollect.md +++ b/docs/2024-06_bidcollect.md @@ -1,20 +1,49 @@ -# Bid Collection +# Relayscan Bid Archive -Relayscan collects bids across [relays](../vars/relays.go) with these methods: +Relayscan can run in a special bid-collection mode to build an archive of bids across [relays](../vars/relays.go). -1. [getHeader polling](https://ethereum.github.io/builder-specs/#/Builder/getHeader) -2. [Data API polling](https://flashbots.github.io/relay-specs/#/Data/getReceivedBids) -3. [Ultrasound top-bid websocket stream](https://github.com/ultrasoundmoney/docs/blob/main/top-bid-websocket.md) +Public bid archive: https://bidarchive.relayscan.io -Output: +--- -1. CSV file archive +### Bids source types -See also: +0. [getHeader polling](https://ethereum.github.io/builder-specs/#/Builder/getHeader) +1. [Data API polling](https://flashbots.github.io/relay-specs/#/Data/getReceivedBids) +2. [Ultrasound top-bid websocket stream](https://github.com/ultrasoundmoney/docs/blob/main/top-bid-websocket.md) + +### Output + +The output is (daily) CSV files: +1. All bids +2. Top bids only + + +### Collected fields + +| Field | Description | Source Types | +| ---------------------- | ------------------------------------------------------ | ------------ | +| source_type | 0: getHeader, 1: Data API, 2: Ultrasound stream | all | +| received_at_ms | Timestamp when the bid was first seen at the collector | all | +| timestamp_ms | Timestamp when the bid was received by the relay | 1 + 2 | +| slot | Slot the bid was submitted for | all | +| slot_t_ms | Slot timestamp | all | +| value | Bid value in wei | all | +| block_hash | Block hash | all | +| parent_hash | Parent hash | all | +| builder_pubkey | Builder pubkey | 1 + 2 | +| block_number | Block number | all | +| block_fee_recipient | Block fee recipient | all | +| relay | Relay name | all | +| proposer_pubkey | Proposer pubkey | 1 + 2 | +| proposer_fee_recipient | Proposer fee recipient | 1 + 2 | +| optimistic_submission | Optimistic submission flag | 0 + 1 | + +### See also -- [Example output](https://gist.github.com/metachris/061c0443afb8b8d07eed477a848fa395) -- [Pull request #37](https://github.com/flashbots/relayscan/pull/37) - Live data: https://bidarchive.relayscan.io +- [Pull request #37](https://github.com/flashbots/relayscan/pull/37) +- [Example output](https://gist.github.com/metachris/061c0443afb8b8d07eed477a848fa395) --- @@ -27,18 +56,18 @@ Source types: Different data sources have different limitations: -- `getHeader` polling: +- `getHeader` polling ([code](/services/bidcollect/getheader-poller.go)): - The received header only has limited information, with these implications: - Optimistic is always `false` - No `builder_pubkey` - No bid timestamp (need to use receive timestamp) - getHeader bid timestamps are always when the response from polling at t=1s comes back (but not when the bid was received at a relay) - Some relays only allow a single `getHeader` request per slot, so we time it at `t=1s` -- Data API polling: +- Data API polling ([code](/services/bidcollect/data-api-poller.go): - Has all the necessary information - Due to rate limits, we only poll at specific times - - Polling at t-4, t-2, t-0.5, t+0.5, t+2 (see also [`services/bidcollect/data-api-poller.go`](services/bidcollect/data-api-poller.go#64-69)) -- Ultrasound websocket stream + - Polling at t-4, t-2, t-0.5, t+0.5, t+2 (see also [`/services/bidcollect/data-api-poller.go`](/services/bidcollect/data-api-poller.go#64-69)) +- Ultrasound websocket stream ([code](/services/bidcollect/ultrasound-stream.go): - doesn't expose optimistic, thus that field is always `false` ## Other notes diff --git a/services/bidcollect/website/templates/base.html b/services/bidcollect/website/templates/base.html index 72d8554..3d509af 100644 --- a/services/bidcollect/website/templates/base.html +++ b/services/bidcollect/website/templates/base.html @@ -113,8 +113,7 @@

{{ $title }}

- - docs · https://github.com/flashbots/relayscan + docs · https://github.com/flashbots/relayscan

Illuminate, Democratize, Distribute