Skip to content

Commit

Permalink
feat: indexer service (#1)
Browse files Browse the repository at this point in the history
# πŸ€– Linear

Closes GIT-30 GIT-31 GIT-50 GIT-51

## Description
Adds indexer service, docker file for the indexer process from envio,
base docker-compose for the full app , docs and last but not least an
important test that ensures that all the events from envio config.yaml
file are handled.

## Checklist before requesting a review

- [x] I have conducted a self-review of my code.
- [x] I have conducted a QA.
- [x] If it is a core feature, I have included comprehensive tests.
  • Loading branch information
0xkenj1 authored Oct 4, 2024
2 parents e4548cc + c9c20f5 commit 8500783
Show file tree
Hide file tree
Showing 31 changed files with 5,598 additions and 626 deletions.
31 changes: 31 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
############################################
############### POSTGRES ###################
############################################
POSTGRES_EXPOSED_PORT=5433
POSTGRES_PASSWORD=testing
POSTGRES_USER=postgres
POSTGRES_DB=envio-dev

############################################
############### HASURA #####################
############################################
HASURA_EXPOSED_PORT=8080
HASURA_GRAPHQL_DATABASE_URL=postgres://postgres:testing@envio-postgres:5432/envio-dev
HASURA_GRAPHQL_ENABLE_CONSOLE=true
HASURA_GRAPHQL_ENABLED_LOG_TYPES=startup, http-log, webhook-log, websocket-log, query-log
HASURA_GRAPHQL_NO_OF_RETRIES=10
HASURA_GRAPHQL_ADMIN_SECRET=testing
HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES="true"
PORT=8080
HASURA_GRAPHQL_UNAUTHORIZED_ROLE=public

############################################
############### INDEXER ####################
############################################
ENVIO_PG_HOST=envio-postgres
ENVIO_PG_PORT=5432
ENVIO_POSTGRES_PASSWORD=testing
ENVIO_PG_USER=postgres
ENVIO_PG_DATABASE=envio-dev
HASURA_GRAPHQL_ENDPOINT=http://graphql-engine:8080/v1/metadata
TUI_OFF=true
92 changes: 24 additions & 68 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,94 +1,50 @@
# ts-turborepo-boilerplate
# Grants Stack Indexer v2

## Features
Grants Stack Indexer v2 is a tool designed to index blockchain events generated by Allo contracts, making the data accessible over HTTP in JSON format. The data is organized in a structure that enables efficient querying and easy access to key components of the protocol.

### Boilerplate monorepo setup
## πŸ“– Overview

Quickly start developing your offchain monorepo project with
minimal configuration overhead using Turborepo
This repository is a monorepo that contains 1 package and 2 applications:

### Sample library with Viem
- @ts-turborepo-boilerplate/sample-lib: A sample library for querying account balances.
- @ts-turborepo-boilerplate/sample-app: A demo application demonstrating the use of sample-lib.
- @grants-stack-indexer/indexer: An [`envio`](https://docs.envio.dev/) indexer, that collects all the relevant events from Allo contracts.

Simple provider that uses Viem client to query account balances
## πŸš€ Local Deployment (Dockerized)

### Sample contracts with Foundry
### Prerequisites

Basic Greeter contract with an external interface
Ensure you have the following installed on your machine:

Foundry configuration out-of-the-box
- [Docker](https://www.docker.com/get-started)
- [Docker Compose](https://docs.docker.com/compose/install/)

### Sample app that consumes the library
### Run

How much ETH do Vitalik and the Zero address hold together?
To deploy locally with Docker, run the following command:

### Testing

Unit test setup with Vitest framework

### Lint and format

Use ESLint and Prettier to easily find issues as you code

### Github workflows CI

Lint code and check commit messages format on every push.

Run all tests and see the coverage before merging changes.

## Overview

This repository is a monorepo consisting of 2 packages and 1 app:

- [`@ts-turborepo-boilerplate/contracts`](./packages/contracts): A library for writing all required smart contracts
- [`@ts-turborepo-boilerplate/sample-lib`](./packages/sample-lib): A sample library for querying account balances
- [`@ts-turborepo-boilerplate/sample-app`](./apps/sample-app): A demo sample app that uses the sample-lib

## πŸ“‹ Prerequisites

- Ensure you have `node 20` and `pnpm 9.7.1` installed.

## Tech stack

- [pnpm](https://pnpm.io/): package and workspace manager
- [turborepo](https://turbo.build/repo/docs): for managing the monorepo and the build system
- [foundry](https://book.getfoundry.sh/forge/): for writing Solidity smart contracts
- [husky](https://typicode.github.io/husky/): tool for managing git hooks
- tsc: for transpiling TS and building source code
- [prettier](https://prettier.io/): code formatter
- [eslint](https://typescript-eslint.io/): code linter
- [vitest](https://vitest.dev/): modern testing framework
- [Viem](https://viem.sh/): lightweight library to interface with EVM based blockchains

### Configuring Prettier sort import plugin
```
docker-compose up -d --build
```

You can further add sorting rules for your monorepo, for example in `.prettierrc` you can add:
Once the deployment is complete, you can access Hasura by navigating to:

```json
...
"importOrder": [
"<TYPES>",
...
"",
"<TYPES>^@myproject", //added
"^@myproject/(.*)$", //added
"",
...
],
...
```
http://localhost:8080/
```

We use [IanVs prettier-plugin-sort-imports](https://github.com/IanVS/prettier-plugin-sort-imports)
Use the default password: `testing`.

## Contributing

Wonderland is a team of top Web3 researchers, developers, and operators who believe that the future needs to be open-source, permissionless, and decentralized.

[DeFi sucks](https://defi.sucks), but Wonderland is here to make it better.

### πŸ’» Conventional Commits
## πŸ“ Conventional Commits

We follow the Conventional Commits [specification](https://www.conventionalcommits.org/en/v1.0.0/#specification).

## License
## πŸ“œ License

The primary license for the boilerplate is MIT. See the [`LICENSE`](./LICENSE) file for details.
This project is licensed under the MIT License. See the [`LICENSE`](./LICENSE) file for more details.
5 changes: 5 additions & 0 deletions apps/indexer/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules
.git
.gitignore
*.md
generated
36 changes: 36 additions & 0 deletions apps/indexer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
*.exe
*.obj
*.out
*.compile
*.native
*.byte
*.cmo
*.annot
*.cmi
*.cmx
*.cmt
*.cmti
*.cma
*.a
*.cmxa
*.obj
*~
*.annot
*.cmj
*.bak
lib/*
*.mlast
*.mliast
.vscode
.merlin
.bsb.lock
/node_modules/
benchmarks/
artifacts
cache
generated
logs
*.bs.js
*.bs.mjs
*.gen.ts
build
5 changes: 5 additions & 0 deletions apps/indexer/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Needed for ts build folder to have
# access to rescript node_modules
shamefully-hoist=true
# Needed so users can run `pnpm install` in the root of the repo without requiring the `-w` flag.
ignore-workspace-root-check=true
11 changes: 11 additions & 0 deletions apps/indexer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM node:20-slim AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
COPY . /app
WORKDIR /app

RUN --mount=type=cache,id=pnpm,target=$PNPM_HOME/store pnpm install --frozen-lockfile
RUN pnpm dlx envio codegen

CMD pnpm dlx envio local db-migrate up && pnpm dlx envio start
56 changes: 56 additions & 0 deletions apps/indexer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Indexer Service

This repository contains the Envio indexer service, which is ready to run on the Envio hosted platform. It includes a Dockerfile for the indexer process.

## πŸš€ Deployment

### Envio Hosted Solution

To deploy the indexer on the Envio hosted service, refer to the official documentation:
[Envio Hosted Service Deployment](https://docs.envio.dev/docs/HyperIndex/hosted-service-deployment)

## πŸ›  Getting Started

### Install Dependencies

Run the following command to install all necessary dependencies:

```bash
pnpm i
```

### Local Development

For local development, start the service with:

```bash
pnpm dev
```

## βž• Adding a New Event

To add a new event to the indexer, follow these steps:

### Step 1: Modify Configuration

Update the `config.yaml` file to include the new event you wish to handle. Ensure the event is correctly configured.

### Step 2: Add an Event Handler

Create a handler for the new event. This handler is essential for ensuring the event is indexed. Without a handler, the event (and its corresponding entry in the `raw_events` table) will not be saved or processed. Each event must have a handler to be included in `raw_events`.

### Step 3: Redeploy the Environment

After making the necessary changes, redeploy your environment to apply the updates.

## πŸ” Notes

Ensure each event is properly handled and thoroughly tested before redeploying the environment to prevent issues in the indexing process.

## πŸ§ͺ Testing

To run tests, use the following command:

```bash
pnpm test
```
Loading

0 comments on commit 8500783

Please sign in to comment.