Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: indexer service #1

Merged
merged 4 commits into from
Oct 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
############################################
############### POSTGRES ###################
############################################
POSTGRES_EXPOSED_PORT=5433
POSTGRES_PASSWORD=testing
POSTGRES_USER=postgres
POSTGRES_DB=envio-dev

############################################
############### HASURA #####################
############################################
HASURA_EXPOSED_PORT=8080
HASURA_GRAPHQL_DATABASE_URL=postgres://postgres:testing@envio-postgres:5432/envio-dev
HASURA_GRAPHQL_ENABLE_CONSOLE=true
HASURA_GRAPHQL_ENABLED_LOG_TYPES=startup, http-log, webhook-log, websocket-log, query-log
HASURA_GRAPHQL_NO_OF_RETRIES=10
HASURA_GRAPHQL_ADMIN_SECRET=testing
HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES="true"
PORT=8080
HASURA_GRAPHQL_UNAUTHORIZED_ROLE=public

############################################
############### INDEXER ####################
############################################
ENVIO_PG_HOST=envio-postgres
ENVIO_PG_PORT=5432
ENVIO_POSTGRES_PASSWORD=testing
ENVIO_PG_USER=postgres
ENVIO_PG_DATABASE=envio-dev
HASURA_GRAPHQL_ENDPOINT=http://graphql-engine:8080/v1/metadata
TUI_OFF=true
92 changes: 24 additions & 68 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,94 +1,50 @@
# ts-turborepo-boilerplate
# Grants Stack Indexer v2

## Features
Grants Stack Indexer v2 is a tool designed to index blockchain events generated by Allo contracts, making the data accessible over HTTP in JSON format. The data is organized in a structure that enables efficient querying and easy access to key components of the protocol.

### Boilerplate monorepo setup
## 📖 Overview

Quickly start developing your offchain monorepo project with
minimal configuration overhead using Turborepo
This repository is a monorepo that contains 1 package and 2 applications:

### Sample library with Viem
- @ts-turborepo-boilerplate/sample-lib: A sample library for querying account balances.
- @ts-turborepo-boilerplate/sample-app: A demo application demonstrating the use of sample-lib.
- @grants-stack-indexer/indexer: An [`envio`](https://docs.envio.dev/) indexer, that collects all the relevant events from Allo contracts.

Simple provider that uses Viem client to query account balances
## 🚀 Local Deployment (Dockerized)

### Sample contracts with Foundry
### Prerequisites

Basic Greeter contract with an external interface
Ensure you have the following installed on your machine:

Foundry configuration out-of-the-box
- [Docker](https://www.docker.com/get-started)
- [Docker Compose](https://docs.docker.com/compose/install/)

### Sample app that consumes the library
### Run

How much ETH do Vitalik and the Zero address hold together?
To deploy locally with Docker, run the following command:

### Testing

Unit test setup with Vitest framework

### Lint and format

Use ESLint and Prettier to easily find issues as you code

### Github workflows CI

Lint code and check commit messages format on every push.

Run all tests and see the coverage before merging changes.

## Overview

This repository is a monorepo consisting of 2 packages and 1 app:

- [`@ts-turborepo-boilerplate/contracts`](./packages/contracts): A library for writing all required smart contracts
- [`@ts-turborepo-boilerplate/sample-lib`](./packages/sample-lib): A sample library for querying account balances
- [`@ts-turborepo-boilerplate/sample-app`](./apps/sample-app): A demo sample app that uses the sample-lib

## 📋 Prerequisites

- Ensure you have `node 20` and `pnpm 9.7.1` installed.

## Tech stack

- [pnpm](https://pnpm.io/): package and workspace manager
- [turborepo](https://turbo.build/repo/docs): for managing the monorepo and the build system
- [foundry](https://book.getfoundry.sh/forge/): for writing Solidity smart contracts
- [husky](https://typicode.github.io/husky/): tool for managing git hooks
- tsc: for transpiling TS and building source code
- [prettier](https://prettier.io/): code formatter
- [eslint](https://typescript-eslint.io/): code linter
- [vitest](https://vitest.dev/): modern testing framework
- [Viem](https://viem.sh/): lightweight library to interface with EVM based blockchains

### Configuring Prettier sort import plugin
```
docker-compose up -d --build
```

You can further add sorting rules for your monorepo, for example in `.prettierrc` you can add:
Once the deployment is complete, you can access Hasura by navigating to:

```json
...
"importOrder": [
"<TYPES>",
...
"",
"<TYPES>^@myproject", //added
"^@myproject/(.*)$", //added
"",
...
],
...
```
http://localhost:8080/
```

We use [IanVs prettier-plugin-sort-imports](https://github.com/IanVS/prettier-plugin-sort-imports)
Use the default password: `testing`.

## Contributing

Wonderland is a team of top Web3 researchers, developers, and operators who believe that the future needs to be open-source, permissionless, and decentralized.

[DeFi sucks](https://defi.sucks), but Wonderland is here to make it better.

### 💻 Conventional Commits
## 📝 Conventional Commits

We follow the Conventional Commits [specification](https://www.conventionalcommits.org/en/v1.0.0/#specification).

## License
## 📜 License

The primary license for the boilerplate is MIT. See the [`LICENSE`](./LICENSE) file for details.
This project is licensed under the MIT License. See the [`LICENSE`](./LICENSE) file for more details.
5 changes: 5 additions & 0 deletions apps/indexer/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules
.git
.gitignore
*.md
generated
36 changes: 36 additions & 0 deletions apps/indexer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
*.exe
*.obj
*.out
*.compile
*.native
*.byte
*.cmo
*.annot
*.cmi
*.cmx
*.cmt
*.cmti
*.cma
*.a
*.cmxa
*.obj
*~
*.annot
*.cmj
*.bak
lib/*
*.mlast
*.mliast
.vscode
.merlin
.bsb.lock
/node_modules/
benchmarks/
artifacts
cache
generated
logs
*.bs.js
*.bs.mjs
*.gen.ts
build
5 changes: 5 additions & 0 deletions apps/indexer/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Needed for ts build folder to have
# access to rescript node_modules
shamefully-hoist=true
# Needed so users can run `pnpm install` in the root of the repo without requiring the `-w` flag.
ignore-workspace-root-check=true
11 changes: 11 additions & 0 deletions apps/indexer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM node:20-slim AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
COPY . /app
WORKDIR /app

RUN --mount=type=cache,id=pnpm,target=$PNPM_HOME/store pnpm install --frozen-lockfile
RUN pnpm dlx envio codegen

CMD pnpm dlx envio local db-migrate up && pnpm dlx envio start
56 changes: 56 additions & 0 deletions apps/indexer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Indexer Service

This repository contains the Envio indexer service, which is ready to run on the Envio hosted platform. It includes a Dockerfile for the indexer process.

## 🚀 Deployment

### Envio Hosted Solution

To deploy the indexer on the Envio hosted service, refer to the official documentation:
[Envio Hosted Service Deployment](https://docs.envio.dev/docs/HyperIndex/hosted-service-deployment)

## 🛠 Getting Started

### Install Dependencies

Run the following command to install all necessary dependencies:

```bash
pnpm i
```

### Local Development

For local development, start the service with:

```bash
pnpm dev
```

## ➕ Adding a New Event

To add a new event to the indexer, follow these steps:

### Step 1: Modify Configuration

Update the `config.yaml` file to include the new event you wish to handle. Ensure the event is correctly configured.

### Step 2: Add an Event Handler

Create a handler for the new event. This handler is essential for ensuring the event is indexed. Without a handler, the event (and its corresponding entry in the `raw_events` table) will not be saved or processed. Each event must have a handler to be included in `raw_events`.

### Step 3: Redeploy the Environment

After making the necessary changes, redeploy your environment to apply the updates.

## 🔍 Notes

Ensure each event is properly handled and thoroughly tested before redeploying the environment to prevent issues in the indexing process.

## 🧪 Testing

To run tests, use the following command:

```bash
pnpm test
```
Loading