Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: indexer service #1

Merged
merged 4 commits into from
Oct 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .npmrc
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
save-exact=true
save-exact=true
# Needed for ts build folder to have
# access to rescript node_modules
shamefully-hoist=true
# Needed so users can run `pnpm install` in the root of the repo without requiring the `-w` flag.
ignore-workspace-root-check=true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i like the necessity of adding the -w flag so you don't mistakenly install a dep globally when it should specific to a package/app

90 changes: 19 additions & 71 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,94 +1,42 @@
# ts-turborepo-boilerplate
# Grants Stack Indexer v2

## Features
Grants Stack Indexer v2 is a tool designed to index blockchain events generated by Allo contracts, making the data accessible over HTTP in JSON format. The data is organized in a structure that enables efficient querying and easy access to key components of the protocol.

### Boilerplate monorepo setup
## 📖 Overview

Quickly start developing your offchain monorepo project with
minimal configuration overhead using Turborepo
This repository is a monorepo that contains 1 package and 2 applications:

### Sample library with Viem
- @ts-turborepo-boilerplate/sample-lib: A sample library for querying account balances.
- @ts-turborepo-boilerplate/sample-app: A demo application demonstrating the use of sample-lib.
- @grants-stack-indexer/indexer: An [`envio`](https://docs.envio.dev/) indexer, that collects all the relevant events from Allo contracts.

Simple provider that uses Viem client to query account balances
## 🚀 Local Deployment (Dockerized)

### Sample contracts with Foundry
### Prerequisites

Basic Greeter contract with an external interface
Ensure you have the following installed on your machine:

Foundry configuration out-of-the-box
- [Docker](https://www.docker.com/get-started)
- [Docker Compose](https://docs.docker.com/compose/install/)

### Sample app that consumes the library
### Run

How much ETH do Vitalik and the Zero address hold together?
To deploy locally with Docker, run the following command:

### Testing

Unit test setup with Vitest framework

### Lint and format

Use ESLint and Prettier to easily find issues as you code

### Github workflows CI

Lint code and check commit messages format on every push.

Run all tests and see the coverage before merging changes.

## Overview

This repository is a monorepo consisting of 2 packages and 1 app:

- [`@ts-turborepo-boilerplate/contracts`](./packages/contracts): A library for writing all required smart contracts
- [`@ts-turborepo-boilerplate/sample-lib`](./packages/sample-lib): A sample library for querying account balances
- [`@ts-turborepo-boilerplate/sample-app`](./apps/sample-app): A demo sample app that uses the sample-lib

## 📋 Prerequisites

- Ensure you have `node 20` and `pnpm 9.7.1` installed.

## Tech stack

- [pnpm](https://pnpm.io/): package and workspace manager
- [turborepo](https://turbo.build/repo/docs): for managing the monorepo and the build system
- [foundry](https://book.getfoundry.sh/forge/): for writing Solidity smart contracts
- [husky](https://typicode.github.io/husky/): tool for managing git hooks
- tsc: for transpiling TS and building source code
- [prettier](https://prettier.io/): code formatter
- [eslint](https://typescript-eslint.io/): code linter
- [vitest](https://vitest.dev/): modern testing framework
- [Viem](https://viem.sh/): lightweight library to interface with EVM based blockchains

### Configuring Prettier sort import plugin

You can further add sorting rules for your monorepo, for example in `.prettierrc` you can add:

```json
...
"importOrder": [
"<TYPES>",
...
"",
"<TYPES>^@myproject", //added
"^@myproject/(.*)$", //added
"",
...
],
...
```

We use [IanVs prettier-plugin-sort-imports](https://github.com/IanVS/prettier-plugin-sort-imports)
docker-compose up -d --build
```

## Contributing

Wonderland is a team of top Web3 researchers, developers, and operators who believe that the future needs to be open-source, permissionless, and decentralized.

[DeFi sucks](https://defi.sucks), but Wonderland is here to make it better.

### 💻 Conventional Commits
## 📝 Conventional Commits

We follow the Conventional Commits [specification](https://www.conventionalcommits.org/en/v1.0.0/#specification).

## License
## 📜 License

The primary license for the boilerplate is MIT. See the [`LICENSE`](./LICENSE) file for details.
This project is licensed under the MIT License. See the [`LICENSE`](./LICENSE) file for more details.
5 changes: 5 additions & 0 deletions apps/indexer/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules
.git
.gitignore
*.md
generated
36 changes: 36 additions & 0 deletions apps/indexer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
*.exe
*.obj
*.out
*.compile
*.native
*.byte
*.cmo
*.annot
*.cmi
*.cmx
*.cmt
*.cmti
*.cma
*.a
*.cmxa
*.obj
*~
*.annot
*.cmj
*.bak
lib/*
*.mlast
*.mliast
.vscode
.merlin
.bsb.lock
/node_modules/
benchmarks/
artifacts
cache
generated
logs
*.bs.js
*.bs.mjs
*.gen.ts
build
5 changes: 5 additions & 0 deletions apps/indexer/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Needed for ts build folder to have
# access to rescript node_modules
shamefully-hoist=true
# Needed so users can run `pnpm install` in the root of the repo without requiring the `-w` flag.
ignore-workspace-root-check=true
11 changes: 11 additions & 0 deletions apps/indexer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM node:20-slim AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
COPY . /app
WORKDIR /app

RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the target= point to $PNPM_HOME/store?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

RUN pnpm dlx envio codegen

CMD pnpm dlx envio local db-migrate setup && pnpm dlx envio start
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to ask, do we want to db-migrate setup the DB whenever a new container boots up? The docs say that it drops the schema before running all migrations.

If this is going to be used during dev only, we could rename this Dockerfile to Dockerfile.dev

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, i think we can replace it like this

Suggested change
CMD pnpm dlx envio local db-migrate setup && pnpm dlx envio start
CMD pnpm dlx envio local db-migrate up && pnpm dlx envio start

and won't drop older tables i think.

Lets keep Dockerfile name as it is, if we need to improve it or create a new one in the future we can rename it :)

56 changes: 56 additions & 0 deletions apps/indexer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Indexer Service

This repository contains the Envio indexer service, which is ready to run on the Envio hosted platform. It includes a Dockerfile for the indexer process.

## 🚀 Deployment

### Envio Hosted Solution

To deploy the indexer on the Envio hosted service, refer to the official documentation:
[Envio Hosted Service Deployment](https://docs.envio.dev/docs/HyperIndex/hosted-service-deployment)

## 🛠 Getting Started

### Install Dependencies

Run the following command to install all necessary dependencies:

```bash
pnpm i
```

### Local Development

For local development, start the service with:

```bash
pnpm dev
```

## ➕ Adding a New Event

To add a new event to the indexer, follow these steps:

### Step 1: Modify Configuration

Update the `config.yaml` file to include the new event you wish to handle. Ensure the event is correctly configured.

### Step 2: Add an Event Handler

Create a handler for the new event. This handler is essential for ensuring the event is indexed. Without a handler, the event (and its corresponding entry in the `raw_events` table) will not be saved or processed. Each event must have a handler to be included in `raw_events`.

### Step 3: Redeploy the Environment

After making the necessary changes, redeploy your environment to apply the updates.

## 🔍 Notes

Ensure each event is properly handled and thoroughly tested before redeploying the environment to prevent issues in the indexing process.

## 🧪 Testing

To run tests, use the following command:

```bash
pnpm test
```
Loading