Restructure + Support poetry format (#73)

* Restracture + support poetry format

* .

* Install poetry in CI

* Run pytest

* fix test

* .

* Fix test

* .

* Fix lint

* Add linter to the CI

* Fix lint

* Update poetry lock

* Fix linter in CI

* Add direnv

* Add dev readme

* Fix itay comment

* Fix publish

* update lock

* add .env to ignore

* Refacot pypi

* Fix publish

* Fix release version

* Fix publish

* Fix toml

* Add tag for version

* Fix publish

* Fix publish

* Fix publish

* Fix publish

* Fix publish

* Fix publish

* Fix publish

* Try diff source for poetry

* Fix publish

* Fix publish

* Fix comments

.envrc

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# .envrc
+#
+# A complete example that:
+#   1) Loads environment variables from a local .env (optional)
+#   2) Discovers/creates the Poetry virtual environment
+#   3) Activates that environment automatically via direnv
+#
+# Usage:
+#   - Create/modify .envrc in your project root
+#   - Run `direnv allow`
+#   - On next `cd` into this repo, your Poetry environment is auto-activated
+
+###############################################################################
+# 1) OPTIONAL: Load environment variables from a `.env` file
+#    - If you have a file named `.env` in this directory for secrets or
+#      general environment variables, the following line will load it.
+#    - (Requires the direnv stdlib function `dotenv` or do `source .env`)
+###############################################################################
+dotenv 2>/dev/null
+# If you prefer a non-direnv approach, you can do:  source .env
+
+###############################################################################
+# 2) Poetry environment discovery or creation
+#    - The snippet below tries to locate an existing Poetry venv via
+#      `poetry env info --path`.
+#    - If no venv is found, it runs `poetry install`.
+#    - Finally, if a venv is located, we run `layout python /path/to/venv/bin/python`.
+###############################################################################
+use_poetry() {
+  POETRY_ENV_PATH="$(poetry env info --path 2>/dev/null)"
+
+  if [[ -z "$POETRY_ENV_PATH" ]]; then
+    echo "No existing Poetry venv found. Installing dependencies..."
+    poetry install
+    POETRY_ENV_PATH="$(poetry env info --path 2>/dev/null)"
+  fi
+
+  if [[ -z "$POETRY_ENV_PATH" || ! -d "$POETRY_ENV_PATH" ]]; then
+    echo "Error: Could not locate or create a Poetry environment."
+    exit 1
+  fi
+
+  # Now actually activate the Python environment:
+  layout python "$POETRY_ENV_PATH/bin/python"
+  echo "Poetry env activated at: $POETRY_ENV_PATH"
+}
+
+use_poetry
+
+###############################################################################
+# 3) OPTIONAL: Additional environment variables or logic
+#    - Here you can set custom environment variables or run commands.
+###############################################################################
+# export ETHSCAN_API_KEY="ABCD1234"
+# export MY_CUSTOM_FLAG="true"
+
+echo "Quorum environment is now set up! Happy coding."

.github/workflows/CI.yml

@@ -33,38 +33,46 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v4
 
+      - name: Install Poetry
+        run: pip3 install poetry==1.8.5
+
       - name: Install Quorum Repo
-        run: pip3.11 install .
+        run: poetry install
+
+      - name: Lint with Ruff
+        run: |
+          poetry run ruff check src
+          poetry run ruff format --check src
 
       - name: Run unit tests
-        run: pytest Quorum/tests --maxfail=1 --disable-warnings --tb=short
+        run: poetry run pytest src/quorum/tests --maxfail=1 --disable-warnings --tb=short
 
       - name: Run Quorum setup
-        run: quorum setup --working-dir workdir
+        run: poetry run quorum setup --working-dir workdir
 
       - name: Init working directory
         run: |
-          mv Quorum/tests/regression.json workdir/regression.json
-          mv Quorum/tests/ground_truth.json workdir/ground_truth.json
+          mv src/quorum/tests/regression.json workdir/regression.json
+          mv src/quorum/tests/ground_truth.json workdir/ground_truth.json
           echo "ETHSCAN_API_KEY=${ETHSCAN_API_KEY}" >> workdir/.env
           echo "ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}" >> workdir/.env
-          
+
       - name: Single Address Test
         working-directory: workdir
-        run: quorum validate-address --protocol-name Aave --chain Ethereum --payload-address 0xAD6c03BF78A3Ee799b86De5aCE32Bb116eD24637
+        run: poetry run quorum validate-address --protocol-name Aave --chain Ethereum --payload-address 0xAD6c03BF78A3Ee799b86De5aCE32Bb116eD24637
 
       - name: Batch Test
         working-directory: workdir
-        run: quorum validate-batch --config regression.json
+        run: poetry run quorum validate-batch --config regression.json
 
       - name: Proposal ID Test
         working-directory: workdir
-        run: quorum validate-by-id --proposal-id 137 --protocol-name Aave
+        run: poetry run quorum validate-by-id --proposal-id 137 --protocol-name Aave
 
       - name: IPFS Test
         working-directory: workdir
-        run: quorum validate-ipfs --proposal-id 20 --chain Scroll --payload-address 0x2B25cb729D90630395Cd3140f3460a73A41Fe5f0
+        run: poetry run quorum validate-ipfs --proposal-id 20 --chain Scroll --payload-address 0x2B25cb729D90630395Cd3140f3460a73A41Fe5f0
 
       - name: Generate Report
         working-directory: workdir
-        run: quorum generate-report --proposal_id 137
+        run: poetry run quorum generate-report --proposal_id 137

.github/workflows/publish-pypi.yml

@@ -3,30 +3,35 @@ name: publish-pypi
 on:
   release:
     types: [published]
-  # workflow_dispatch:  # We may want to add manual publishing as well.
+  # Uncomment the following line to enable manual publishing
+  # workflow_dispatch:
 
 jobs:
   publish:
     runs-on: ubuntu-latest
     permissions:
-      id-token: write
-      contents: read
+      contents: write
     env:
-      TWINE_USERNAME: ${{ secrets.PYPI_USER }}
-      TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
-      TAG: ${{ github.ref_name }}
+      POETRY_DYNAMIC_VERSIONING_BYPASS: ${{ github.ref_name }}
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
-
-      - name: Update Quorum version
-        run: echo "__version__ = '$TAG'" > Quorum/__init__.py
 
-      - name: Install requirements for publishing
-        run: pip3 install build twine
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'  # Specify the Python version compatible with your project
 
-      - name: Build distros
-        run: python3 -m build
-
-      - name: Upload to PyPi
-        run: python3 -m twine upload dist/* --verbose
+      - name: Install Poetry Locally
+        run: |
+          curl -sSL https://install.python-poetry.org | python3 -
+          echo "${HOME}/.local/bin" >> $GITHUB_PATH
+
+      - name: Install Automatic Versioning Plugin
+        run: poetry self add "poetry-dynamic-versioning[plugin]"
+
+      - name: Configure PyPI Repository
+        run: poetry config pypi-token.pypi ${{ secrets.PYPI_PASSWORD }}
+
+      - name: "Publish to PyPI with version: ${{ env.POETRY_DYNAMIC_VERSIONING_BYPASS }}"
+        run: poetry publish --build

.gitignore

@@ -10,4 +10,6 @@ CustomerClones/
 **.DS_Store
 !**cached_llm.py
 quorum_project/
-dist/
+dist/
+.direnv/
+.env

.pre-commit-config.yaml

@@ -1,9 +1,10 @@
 repos:
-  - repo: local
-    hooks:
-      - id: update-version
-        name: update-version
-        entry: make
-        args: ["update-version"]
-        language: system
-        pass_filenames: false
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  # Ruff version.
+  rev: v0.8.6
+  hooks:
+    # Run the linter.
+    - id: ruff
+      args: [ --fix ]
+    # Run the formatter.
+    - id: ruff-format

DEVREADME.md

@@ -0,0 +1,182 @@
+# Quorum Developer Guide
+
+Welcome to the **Quorum** project! This guide helps developers get set up with the local environment, including:
+
+1. Installing and configuring **Poetry** for dependency management  
+2. Using **direnv** for automatic environment activation  
+3. Running **Quorum** commands and verifying everything is correct  
+
+## 1. Requirements
+
+1. **Python** version `>=3.11,<4.0`
+2. **Poetry** version `1.8+` (recommended)
+3. **direnv** (optional but highly recommended for convenience)
+4. (Optional) A local `.env` file for secret variables, personal overrides, etc.
+
+## 2. Project Structure
+
+A quick look at our relevant files and directories:
+
+```
+.
+├── LICENSE
+├── MANIFEST.in
+├── Makefile
+├── README.md
+├── poetry.lock
+├── pyproject.toml
+├── .envrc                 # direnv config file
+├── .env.example           # Example environment variables
+└── src
+    └── quorum            # Project code
+       ├── __init__.py
+       ├── entry_points
+       ├── ...
+```
+
+## 3. Setting up the Project Environment
+
+### 3.1 Install or Update Poetry
+
+If you haven’t installed Poetry (or have an older version), do so with:
+
+```bash
+curl -sSL https://install.python-poetry.org | python3 -
+```
+
+Verify the version:
+
+```bash
+poetry --version
+```
+
+### 3.2 Configure direnv (Optional but Recommended)
+
+**direnv** is a tool that automatically loads/unloads environment variables when entering/leaving directories. We use it to:
+
+- **Auto-activate** the correct Poetry virtual environment
+- Optionally load environment variables from a `.env` file
+
+#### 3.2.1 Install direnv
+
+On **macOS** (Homebrew):
+
+```bash
+brew install direnv
+```
+
+On **Linux** (e.g., Ubuntu/Debian):
+
+```bash
+sudo apt-get update
+sudo apt-get install direnv
+```
+
+#### 3.2.2 Integrate with Shell
+
+For **bash** or **zsh**, add this to your `~/.bashrc`, `~/.zshrc`, etc.:
+
+```bash
+eval "$(direnv hook bash)"   # or zsh, fish, etc.
+```
+
+Then run:
+
+```bash
+direnv allow
+```
+
+So every time you `cd` into the project folder, your environment is **automatically loaded** 
+And fully configured according to your .env and all packages already installed.
+
+### 3.3 Install Dependencies
+
+#### 3.3.1 Using Poetry
+
+Once inside your project directory (and optionally, after `direnv allow`), run:
+
+```bash
+poetry install
+```
+
+This installs all dependencies (including dev dependencies) listed in `pyproject.toml`.
+
+## 4. Running Quorum Locally
+
+We provide a CLI **`quorum`**. Once your environment is active, you can run:
+
+```bash
+poetry run quorum --help
+```
+
+Or, if you’re using **direnv** or in the Poetry shell:
+
+```bash
+quorum --help
+```
+
+### 4.1 Example Commands
+
+- **Single Payload Validation**:
+  ```bash
+  quorum validate-address --protocol-name Aave --chain Ethereum --payload-address 0xAD6...
+  ```
+- **Batch Validation** (config-based):
+  ```bash
+  quorum validate-batch --config path/to/config.json
+  ```
+- **Proposal ID Check**:
+  ```bash
+  quorum validate-by-id --proposal-id 137 --protocol-name Aave
+  ```
+- **IPFS Validation**:
+  ```bash
+  quorum validate-ipfs --proposal-id 20 --chain Scroll --payload-address 0x2B25cb...
+  ```
+- **Generate Report**:
+  ```bash
+  quorum generate-report --proposal_id 137
+  ```
+
+## 5. Developing with Ruff & Pre-Commit
+
+**Ruff** is our linter and auto-fixer. **pre-commit** runs Ruff automatically before each commit if you have it configured. Typical usage:
+
+```bash
+# Run ruff checks manually:
+poetry run ruff check src
+
+# Auto-fix issues:
+poetry run ruff check src --fix
+```
+
+If you have a `.pre-commit-config.yaml` with a ruff hook, you can do:
+
+```bash
+pre-commit install
+pre-commit run --all-files
+```
+
+## 6. Additional Environment Variables (Optional)
+
+If you want to store secrets like `ETHSCAN_API_KEY`, `ANTHROPIC_API_KEY`, etc., do one of the following:
+
+- Put them in a local `.env` file in the project root (which is `.gitignore`d). Example:
+  ```bash
+  ETHSCAN_API_KEY="abc123"
+  ANTHROPIC_API_KEY="xxx"
+  ```
+  Then add `dotenv 2>/dev/null` in your `.envrc`.
+- Or set them in your shell profile (`~/.bashrc`, `~/.zshrc`, etc.).
+- Or pass them directly in your CI environment.
+
+---
+
+## That’s It!
+
+You are now set up with:
+
+- **Poetry** for dependency and environment management
+- **direnv** for auto-activating your environment
+- **Ruff** linting with optional pre-commit hooks
+- **Quorum** CLI commands to run local validations and generate reports

MANIFEST.in

@@ -1,5 +1,4 @@
 include LICENSE
 include version
-include requirements.txt
-include Quorum/templates/*
-recursive-include Quorum *.j2
+include src/quorum/templates/*
+recursive-include src/quorum *.j2