Subgraph-compatible Autogenerated API (#26)

* wip: custom graphql schema loaded

* wip: basic collection queries working

* wip: literally encode ids instead of eq conditions

* fix: store labelhash in domains

* fix: resolvedAddress is a relationship not an address

* fix: lowercase resolver address in resolverId

* fix: (in)correctly mark root as migrated and include exception in NewResolver to match subgraph behavior

* fix: default null array fields for backwards compat

* docs: add documentation to graphql.ts

* fix: remove unnecesary context injection

* fix: coinType, texts array default, makeResolverId arg order

* fix: handle dangling makeResolverId refactor

* fix: index all resolver-looking events not just NewResolver

* docs: add v2 notes, add comment about root node isMigrated

* fix: use enum for OrderDirection to match subgraph

* feat: implement scalar Entity_orderBy enums

* docs: add rest of graphql autogen api spec

* docs: update backfill time estimation

* feat: host subgraph-compatible api at /subgraph instead of root

* fix: handle subgraph case where version is changed but domain doesn't exist

* chore: better comment on the other domain expected error

* docs: document which api features are explicitly not supported

* chore: remove subgraph-api todo from readme

* chore: update package name to ensnode

* docs: tidy v2 notes, move _nocase to potential todo

* fix: use upsert in Resolver#handleVersionChanged

* docs: update backfill time note in README

* fix: 13 hours for mainnet backfill

* deps: ponder to latest

* docs: note about CLUSTER

* fix: correctly nullify texts and cointypes on version changed

* fix: simplify expiryDate logic o

* docs: update comments post feedback

---------

Co-authored-by: Tomasz Kopacki <[email protected]>

.gitignore

@@ -16,5 +16,3 @@ yarn-error.log*
 # Ponder
 /generated/
 /.ponder/
-
-checkpoints/

README.md

@@ -2,9 +2,9 @@
 
 > a multichain ENS indexer, powered by Ponder
 
-estimated backfill time @ 50rps = 24-36 hours on M1 Macbook (~10x speedup)
+estimated mainnet-only backfill time @ <=500rps = **~13 hours** on M1 Macbook (>10x speedup vs subgraph)
 
-### goals
+## goals
 
 > an optimized, multichain ens indexer that the community loves and integrates
 
@@ -23,26 +23,24 @@ estimated backfill time @ 50rps = 24-36 hours on M1 Macbook (~10x speedup)
   - (possible) continued backwards compatibility with subgraph
   - support indexing subset of data, i.e. only domains under parent node
 
-#### next up
+## next up
 
+- [ ] `_nocase` case-insensitive where filters
+  - not used interally but ensjs does technically expose this as an available filter to users
 - [ ] confirm all the schema relations are configured correctly
 - [ ] integrate rainbow tables for label healing
   - load the table dump into pglite (or just postgres) & query synchronously to match existing behavior
   - https://github.com/graphprotocol/ens-rainbow
-- [ ] subgraph graphql implementation within ponder
-  - [ ] implement subgraph-style pagination api
-  - [ ] support the well-known queries in `GRAPHQL.md`
-  - [ ] support collection queries as well, to power `snapshot-eq`
 - [ ] CI/CD with indexing?
-  - more recent endlbock for gut checks
+  - integrate index to recent block (10m?) and validate with `snapshot-eq` b4 passing
 - [ ] better understand reverse resolution & how that pertains to L2 primary names and impacts the future schema, etc
 
-### notes
+## notes
 
 - eth registry is ERC721, has many controllers (), no knowledge of pricing — delegated to registrar controllers
 - eth old registry & new registry migration due to security issue, new then fallback to old, therefore ignore all old evens on domains that have been seen by new registry
 
-### ENSIP Ideas
+## ENSIP Ideas
 
 - unable to automatically identify subname registries via onchain event, CCIP standard dosn't include any info about data source, so we'll need to encode manually for now
 - ENSIP - shared interface for subdomain registrars

docs/GRAPHQL.md

@@ -1,5 +1,20 @@
 # graphql info/spec
 
+## backwards-compatibility notes
+
+the following features of the subgraph graphql api are explicitly unsupported, because they are not used by ensjs or ens-app-v3
+
+- [fulltext search queries](https://thegraph.com/docs/en/subgraphs/querying/graphql-api/#fulltext-search-queries)
+- [1-level-nested Entity `_orderBy` param](https://thegraph.com/docs/en/subgraphs/querying/graphql-api/#example-for-nested-entity-sorting)
+- [subgraph `_Meta_` object](https://thegraph.com/docs/en/subgraphs/querying/graphql-api/#subgraph-metadata) (ponder's `_meta` is available)
+- [time travel queries](https://thegraph.com/docs/en/subgraphs/querying/graphql-api/#time-travel-queries)
+- [_change_block filtering](https://thegraph.com/docs/en/subgraphs/querying/graphql-api/#example-for-block-filtering)
+
+## notes
+
+- offset queries slow? `CLUSTER` your table so scans are faster
+  - `CLUSTER public.domains USING domains_pkey;`
+
 ## goals
 
 1. ponder indexer 1:1 equivalency of results as compared to subgraph

docs/V2.md

@@ -41,6 +41,8 @@ the 'empty' domains should be handled more accurately, depending on how importan
   - removes need to increment during domain creation
   - new impl likely needs to exclude 'empty' domains (see registry notes for context)
 
+domain createdAt should not update on re-registration, should be original createdAt
+
 various resources use both null and zeroAddress to indicate emptiness, this is horrible and creates numerous checks like [this](https://github.com/ensdomains/ensjs/blob/main/packages/ensjs/src/functions/subgraph/getNamesForAddress.ts#L255) where they check for `!== NULL && !== zeroAddress`
 
 wrappedOwnerId should not be materialized onto domain, should just be resolved through wrappedDomain.owner
@@ -75,6 +77,8 @@ any resolver that implements the CCIP Read standard will have to have its record
 
 in the subgraph implementation, resolver handlers must upsert resolvers because people can set records etc for a node that has not (yet) specified this resolver as active, meaning the create in `Registry:NewResolver` has yet to fire. in the ideal scenario, this lookup is keyed only by `(chainId, address)` and we can use pure updates instead of an upsert
 
+v1: resolvers are discovered by tracking all emissions of events that look like resolver spec events. very inefficient, as a resolver is only relevant to the protocol once it's been set as a resolver. for v2, we could use factory like normal, starting to track events on `Registry#NewResolver` and then backfilling the events (using the same handlers) with ponder's cached viem client. not sure if this is more or less intensive than just parsing every event ever
+
 ### registrar
 
 the subgraph implements all of the BaseRegistrar, EthRegistrarController, and EthRegistrarControllerOld logic together

package.json

@@ -1,5 +1,5 @@
 {
-  "name": "ens-multichain",
+  "name": "ensnode",
   "version": "0.0.1",
   "private": true,
   "type": "module",
@@ -13,6 +13,15 @@
   },
   "dependencies": {
     "@ensdomains/ensjs": "^4.0.2",
+    "@escape.tech/graphql-armor-max-aliases": "^2.6.0",
+    "@escape.tech/graphql-armor-max-depth": "^2.4.0",
+    "@escape.tech/graphql-armor-max-tokens": "^2.5.0",
+    "change-case": "^5.4.4",
+    "dataloader": "^2.2.3",
+    "drizzle-orm": "^0.38.3",
+    "graphql": "^16.10.0",
+    "graphql-scalars": "^1.24.0",
+    "graphql-yoga": "^5.10.9",
     "hono": "^4.6.14",
     "ponder": "^0.8.26",
     "ts-deepmerge": "^7.0.2",

ponder.schema.ts

@@ -17,7 +17,7 @@ export const domain = onchainTable("domains", (t) => ({
   subdomainCount: t.integer("subdomain_count").notNull().default(0),
 
   // Address logged from current resolver, if any
-  resolvedAddress: t.hex("resolved_address"),
+  resolvedAddressId: t.hex("resolved_address_id"),
 
   // The resolver that controls the domain's settings
   resolverId: t.text(),
@@ -45,7 +45,10 @@ export const domain = onchainTable("domains", (t) => ({
 }));
 
 export const domainRelations = relations(domain, ({ one, many }) => ({
-  // has one owner
+  resolvedAddress: one(account, {
+    fields: [domain.resolvedAddressId],
+    references: [account.id],
+  }),
   owner: one(account, {
     fields: [domain.ownerId],
     references: [account.id],
@@ -105,9 +108,11 @@ export const resolver = onchainTable("resolvers", (t) => ({
   // The content hash for this resolver, in binary format
   contentHash: t.text("content_hash"),
   // The set of observed text record keys for this resolver
-  texts: t.text().array().notNull().default([]),
+  // NOTE: we avoid .notNull.default([]) to match subgraph behavior
+  texts: t.text().array(),
   // The set of observed SLIP-44 coin types for this resolver
-  coinTypes: t.bigint("coin_types").array().notNull().default([]),
+  // NOTE: we avoid .notNull.default([]) to match subgraph behavior
+  coinTypes: t.bigint("coin_types").array(),
 
   // TODO: has many events
 }));