Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add blockmeta block number provider #37

Merged
merged 8 commits into from
Sep 13, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions packages/blocknumber/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@
"author": "",
"license": "ISC",
"dependencies": {
"viem": "2.17.10",
"@ebo-agent/shared": "workspace:*"
"@ebo-agent/shared": "workspace:*",
"axios": "1.7.7",
"viem": "2.17.10"
},
"devDependencies": {
"axios-mock-adapter": "2.0.0"
}
}
7 changes: 7 additions & 0 deletions packages/blocknumber/src/exceptions/undefinedBlockNumber.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export class UndefinedBlockNumber extends Error {
constructor(isoTimestamp: string) {
super(`Undefined block number at ${isoTimestamp}.`);

this.name = "UndefinedBlockNumber";
}
}
11 changes: 8 additions & 3 deletions packages/blocknumber/src/providers/blockNumberProviderFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { FallbackTransport, HttpTransport, PublicClient } from "viem";
import { UnsupportedChain } from "../exceptions/unsupportedChain.js";
import { Caip2ChainId } from "../types.js";
import { Caip2Utils } from "../utils/index.js";
import { BlockmetaJsonBlockNumberProvider } from "./blockmetaJsonBlockNumberProvider.js";
import { EvmBlockNumberProvider } from "./evmBlockNumberProvider.js";

const DEFAULT_PROVIDER_CONFIG = {
Expand All @@ -16,20 +17,24 @@ export class BlockNumberProviderFactory {
* Build a `BlockNumberProvider` to handle communication with the specified chain.
*
* @param chainId CAIP-2 chain id
* @param client a viem public client
* @param evmClient a viem public client
* @param logger a ILogger instance
* @returns
*/
public static buildProvider(
chainId: Caip2ChainId,
client: PublicClient<FallbackTransport<HttpTransport[]>>,
evmClient: PublicClient<FallbackTransport<HttpTransport[]>>,
blockmetaConfig: { baseUrl: URL; servicePath: string; bearerToken: string },
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im thinking if this should be a config object like:

{
evmClient?: PublicClient<FallbackTransport<HttpTransport[]>>,
blockmetaConfig?: { baseUrl: URL; servicePath: string; bearerToken: string }
}: ProviderConfig

and here validate that the proper client/config is defined depending on the switch case. passing an evmClient when instantiating a Solana's one is not necessary actually

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's kinda too "functional-y" as it is right now, this factory could probably be a instance initialized with the config you mention and then by defining the new BlockNumberProviderFactory().build() instance method, there's no need to pass the config as it's already inside the factory instance.

logger: ILogger,
) {
const chainNamespace = Caip2Utils.getNamespace(chainId);

switch (chainNamespace) {
case EBO_SUPPORTED_CHAINS_CONFIG.evm.namespace:
return new EvmBlockNumberProvider(client, DEFAULT_PROVIDER_CONFIG, logger);
return new EvmBlockNumberProvider(evmClient, DEFAULT_PROVIDER_CONFIG, logger);

case EBO_SUPPORTED_CHAINS_CONFIG.solana.namespace:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solana 👀 going shopping for those 200GB ram for running validators xd

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xd

return new BlockmetaJsonBlockNumberProvider(blockmetaConfig, logger);

default:
throw new UnsupportedChain(chainId);
Expand Down
126 changes: 126 additions & 0 deletions packages/blocknumber/src/providers/blockmetaJsonBlockNumberProvider.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
import { ILogger, Timestamp } from "@ebo-agent/shared";
import axios, { AxiosInstance, AxiosResponse, isAxiosError } from "axios";

import { UndefinedBlockNumber } from "../exceptions/undefinedBlockNumber.js";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we should import these from index.js but we can clean up later

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, got a task already for that! 👌

import { BlockNumberProvider } from "./blockNumberProvider.js";

type BlockByTimeResponse = {
num: string;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this should be blockNum so it's a bit more descriptive--I had to see where it's being used to understand the type

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is based on the blockmeta BlockByTime service response, sadly we cannot change it as the keys are defined by the third-party service.

id: string;
time: string;
};

export type BlockmetaClientConfig = {
baseUrl: URL;
servicePath: string;
bearerToken: string;
};

/**
* Consumes the blockmeta.BlockByTime substreams' service via HTTP POST JSON requests to provide
* block numbers based on timestamps
*
* Refer to these web pages for more information:
* * https://thegraph.market/
* * https://substreams.streamingfast.io/documentation/consume/authentication
*/
export class BlockmetaJsonBlockNumberProvider implements BlockNumberProvider {
private readonly axios: AxiosInstance;

constructor(
private readonly options: BlockmetaClientConfig,
private readonly logger: ILogger,
) {
const { baseUrl, bearerToken } = options;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a long lived Bearer Token like an API Key?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, people have to get this token through the The Graph webapp which is kinda weird as they have no documented endpoint to automate it.

But now that you mention, we should definitely notify when this token is going to expire, even if it's an extremely long-lived token (1 year).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Config or options ? 🤣


this.axios = axios.create({
baseURL: baseUrl.toString(),
headers: {
common: {
"Content-Type": "application/json",
Authorization: `Bearer ${bearerToken}`,
},
},
});
}

/** @inheritdoc */
async getEpochBlockNumber(timestamp: Timestamp): Promise<bigint> {
if (timestamp > Number.MAX_SAFE_INTEGER || timestamp < Number.MIN_SAFE_INTEGER)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (timestamp > Number.MAX_SAFE_INTEGER || timestamp < Number.MIN_SAFE_INTEGER)
if (timestamp > Number.MAX_SAFE_INTEGER || timestamp < 0)

Number.MIN_SAFE_INTEGER = -9007199254740991

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for fun I've requested the BlockByTime/After after '{"time": "1950-01-01T00:00:00.000Z"}' and got:

"num":"20737620", "time":"1915-04-21T01:10:00.999Z"

There's probably a weird overflow bug in substreams lol should we report it? Doesn't seem too big of an issue as no sane person would try to fetch blocks before computers even existed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, maybe we should just tell the graph team on slack

throw new RangeError(`Timestamp ${timestamp.toString()} cannot be casted to a Number.`);

const timestampNumber = Number(timestamp);
const isoTimestamp = new Date(timestampNumber).toISOString();

try {
// Try to get the block number at a specific timestamp
const blockNumberAt = await this.getBlockNumberAt(isoTimestamp);

return blockNumberAt;
} catch (err) {
const isAxios404 = isAxiosError(err) && err.response?.status === 404;
const isUndefinedBlockNumber = err instanceof UndefinedBlockNumber;

if (!isAxios404 && !isUndefinedBlockNumber) throw err;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would we want to also handle Unauthorized errors?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking on this, we could use some ep on constructor (like an echo get or smth simple) to check that credentials are working or else throw on constructor, wdyt?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a nice idea. I'm generally reluctant about async-ing constructors but I think that by using a static method we could do something like:

// BlockmetaJsonBlockNumberProvider.ts
async static initialize(config, ...) {
  const provider = new BlockmetaJsonBlockNumberProvider();
  const successfulConnection = await provider.testConnection();

  if (successfulConnection) return provider;
  else throw new Error("BANG");
}

Wdyt?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol we are aligned without even trying


// If no block has its timestamp exactly equal to the specified timestamp,
// try to get the most recent block before the specified timestamp.
const blockNumberBefore = await this.getBlockNumberBefore(isoTimestamp);

return blockNumberBefore;
}
}

/**
* Gets the block number at a specific timestamp.
*
* @param isoTimestamp ISO UTC timestamp
* @throws { UndefinedBlockNumber } if request was successful but block number is invalid/not present
* @throws { AxiosError } if request fails
* @returns a promise with the block number at the timestamp
*/
private async getBlockNumberAt(isoTimestamp: string): Promise<bigint> {
const { servicePath } = this.options;

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we validate isoTimestamp here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ended up passing around Date instances and each method is in charge of calling the toISOString() method on it. No need for validating now 🪄

const response = await this.axios.post(`${servicePath}/At`, { time: isoTimestamp });

return this.parseBlockByTimeResponse(response, isoTimestamp);
}

/**
* Gets the most recent block number before the specified timestamp.
*
* @param isoTimestamp ISO UTC timestamp
* @throws { UndefinedBlockNumber } if request was successful but block number is invalid/not present
* @throws { AxiosError } if request fails
* @returns a promise with the most recent block number before the specified timestamp
*/
private async getBlockNumberBefore(isoTimestamp: string): Promise<bigint> {
const { servicePath } = this.options;

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

const response = await this.axios.post(`${servicePath}/Before`, { time: isoTimestamp });

return this.parseBlockByTimeResponse(response, isoTimestamp);
}

/**
* Parse the BlockByTime response and extracts the block number.
*
* @param response an AxiosResponse of a request to BlockByTime endpoint
* @param isoTimestamp the timestamp that was sent in the request
* @returns the block number inside a BlockByTime service response
*/
private parseBlockByTimeResponse(response: AxiosResponse, isoTimestamp: string): bigint {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private parseBlockByTimeResponse(response: AxiosResponse, isoTimestamp: string): bigint {
private parseBlockByTimeResponse(data: unknown, isoTimestamp: string): bigint {

Opinions ? @0xnigir1

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wouldn't use unknown because it's an AxiosResponse object, what we should validate is response.data, probablyAxiosRespose<unknown> might be better (if im not wrong it's generic)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AxiosResponse<unknown> works for me!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what i wanted to point out here was that the signature shouldn't rely on Axios types, just receives the data

const { data } = response;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should specify what we're returning in the function def right?

// TODO: validate with zod instead
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hehehe

const blockNumber = (data as BlockByTimeResponse)["num"];

if (blockNumber === undefined) {
this.logger.error(`Couldn't find a block number for timestamp ${isoTimestamp}`);

throw new UndefinedBlockNumber(isoTimestamp);
}

return BigInt(blockNumber);
}
}
4 changes: 4 additions & 0 deletions packages/blocknumber/src/providers/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export * from "./blockNumberProvider.js";
export * from "./blockNumberProviderFactory.js";
export * from "./blockmetaJsonBlockNumberProvider.js";
export * from "./evmBlockNumberProvider.js";
9 changes: 8 additions & 1 deletion packages/blocknumber/src/services/blockNumberService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { EBO_SUPPORTED_CHAIN_IDS, ILogger, Timestamp } from "@ebo-agent/shared";
import { createPublicClient, fallback, http } from "viem";

import { ChainWithoutProvider, EmptyRpcUrls, UnsupportedChain } from "../exceptions/index.js";
import { BlockmetaClientConfig } from "../providers/blockmetaJsonBlockNumberProvider.js";
import { BlockNumberProvider } from "../providers/blockNumberProvider.js";
import { BlockNumberProviderFactory } from "../providers/blockNumberProviderFactory.js";
import { Caip2ChainId } from "../types.js";
Expand All @@ -20,6 +21,7 @@ export class BlockNumberService {
*/
constructor(
chainRpcUrls: Map<Caip2ChainId, RpcUrl[]>,
private readonly blockmetaConfig: BlockmetaClientConfig,
private readonly logger: ILogger,
) {
this.blockNumberProviders = this.buildBlockNumberProviders(chainRpcUrls);
Expand Down Expand Up @@ -81,7 +83,12 @@ export class BlockNumberService {
transport: fallback(urls.map((url) => http(url))),
});

const provider = BlockNumberProviderFactory.buildProvider(chainId, client, this.logger);
const provider = BlockNumberProviderFactory.buildProvider(
chainId,
client,
this.blockmetaConfig,
this.logger,
);

if (!provider) throw new ChainWithoutProvider(chainId);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,29 +1,69 @@
import { Logger } from "@ebo-agent/shared";
import { createPublicClient, fallback, http } from "viem";
import {
createPublicClient,
fallback,
FallbackTransport,
http,
HttpTransport,
PublicClient,
} from "viem";
import { describe, expect, it } from "vitest";

import { UnsupportedChain } from "../../src/exceptions";
import { BlockNumberProviderFactory } from "../../src/providers/blockNumberProviderFactory";
import { EvmBlockNumberProvider } from "../../src/providers/evmBlockNumberProvider";
import { Caip2ChainId } from "../../src/types";
import { UnsupportedChain } from "../../src/exceptions/index.js";
import {
BlockmetaClientConfig,
BlockmetaJsonBlockNumberProvider,
BlockNumberProviderFactory,
EvmBlockNumberProvider,
} from "../../src/providers/index.js";
import { Caip2ChainId } from "../../src/types.js";

describe("BlockNumberProviderFactory", () => {
const logger = Logger.getInstance();

describe("buildProvider", () => {
const client = createPublicClient({ transport: fallback([http("http://localhost:8545")]) });
const client: PublicClient<FallbackTransport<HttpTransport[]>> = createPublicClient({
transport: fallback([http("http://localhost:8545")]),
});

it("builds a provider", () => {
const provider = BlockNumberProviderFactory.buildProvider("eip155:1", client, logger);
const blockmetaConfig: BlockmetaClientConfig = {
baseUrl: new URL("localhost:443"),
servicePath: "/sf.blockmeta.v2.BlockByTime",
bearerToken: "bearer-token",
};

describe("buildProvider", () => {
it("builds an EVM provider", () => {
const provider = BlockNumberProviderFactory.buildProvider(
"eip155:1",
client,
blockmetaConfig,
logger,
);

expect(provider).toBeInstanceOf(EvmBlockNumberProvider);
});

it("builds a Solana Blockmeta provider", () => {
const provider = BlockNumberProviderFactory.buildProvider(
"solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
client,
blockmetaConfig,
logger,
);

expect(provider).toBeInstanceOf(BlockmetaJsonBlockNumberProvider);
});

it("fails if chain is not supported", () => {
const unsupportedChainId = "solana:80085" as Caip2ChainId;
const unsupportedChainId = "antelope:f16b1833c747c43682f4386fca9cbb32" as Caip2ChainId;

expect(() => {
BlockNumberProviderFactory.buildProvider(unsupportedChainId, client, logger);
BlockNumberProviderFactory.buildProvider(
unsupportedChainId,
client,
blockmetaConfig,
logger,
);
}).toThrow(UnsupportedChain);
});
});
Expand Down
Loading