Merge branch 'main' into drand-voting

.github/.all-contributorsrc

@@ -99,6 +99,24 @@
       "contributions": [
         "code"
       ]
+    },
+    {
+      "login": "petscheit",
+      "name": "Paul Etscheit",
+      "avatar_url": "https://avatars.githubusercontent.com/u/10947187?v=4",
+      "profile": "https://github.com/petscheit",
+      "contributions": [
+        "code"
+      ]
+    },
+    {
+      "login": "stefanMadzharov",
+      "name": "stefanMadzharov",
+      "avatar_url": "https://avatars.githubusercontent.com/u/83451593?v=4",
+      "profile": "https://github.com/stefanMadzharov",
+      "contributions": [
+        "code"
+      ]
     }
   ],
   "contributorsPerLine": 7,

.github/workflows/hydra.yml

@@ -30,6 +30,7 @@ jobs:
           toolchain: stable
           override: true
           profile: minimal
+          components: rustfmt
 
       - uses: actions/checkout@v3
       - name: Set up Python 3.10.14
@@ -50,3 +51,19 @@ jobs:
         run: |
           source venv/bin/activate
           pytest -n auto
+      - name: Set up Scarb
+        uses: software-mansion/setup-scarb@v1
+        with:
+          scarb-version: "2.8.2"
+      - name: Run make rewrite and check for unstaged changes
+        run: |
+          source venv/bin/activate
+          if ! make rewrite; then
+            echo "Error: 'make rewrite' command failed"
+            exit 1
+          fi
+          if [[ -n $(git status --porcelain) ]]; then
+            echo "Error: Unstaged changes detected after running 'make rewrite'"
+            git status
+            exit 1
+          fi

README.md

@@ -29,7 +29,7 @@
 Garaga enables efficient elliptic curve operations on Starknet.
 It achieves state of the art performance by
 -  using a dedicated builtin made by Starkware for emulated modular arithmetic
-- using a suite of non-deterministic techniques for extension field multiplication, pairings, and multi scalar multiplication to reduce the number of steps to verifiy results.
+- using a suite of non-deterministic techniques for extension field multiplication, pairings, and multi scalar multiplication to reduce the number of steps to verify results.
 
 Garaga currently supports:
 - Scalar & Multi-scalar multiplication for any Weirstrass curve, including BN254, BLS12_381, SECP256/R1, and ED25519. You can add the one you need by specifying the curve parameters.
@@ -211,6 +211,8 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="14.28%"><a href="http://shramee.me/"><img src="https://avatars.githubusercontent.com/u/11048263?v=4?s=100" width="100px;" alt="Shramee Srivastav"/><br /><sub><b>Shramee Srivastav</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/garaga/commits?author=shramee" title="Code">💻</a></td>
       <td align="center" valign="top" width="14.28%"><a href="https://github.com/akinovak"><img src="https://avatars.githubusercontent.com/u/28649205?v=4?s=100" width="100px;" alt="Andrija Novakovic"/><br /><sub><b>Andrija Novakovic</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/garaga/commits?author=akinovak" title="Code">💻</a></td>
       <td align="center" valign="top" width="14.28%"><a href="https://github.com/Liam-Eagen"><img src="https://avatars.githubusercontent.com/u/5618692?v=4?s=100" width="100px;" alt="Liam Eagen"/><br /><sub><b>Liam Eagen</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/garaga/commits?author=Liam-Eagen" title="Code">💻</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/petscheit"><img src="https://avatars.githubusercontent.com/u/10947187?v=4?s=100" width="100px;" alt="Paul Etscheit"/><br /><sub><b>Paul Etscheit</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/garaga/commits?author=petscheit" title="Code">💻</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/stefanMadzharov"><img src="https://avatars.githubusercontent.com/u/83451593?v=4?s=100" width="100px;" alt="stefanMadzharov"/><br /><sub><b>stefanMadzharov</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/garaga/commits?author=stefanMadzharov" title="Code">💻</a></td>
     </tr>
   </tbody>
   <tfoot>

docs/gitbook/installation/npm-package.md

@@ -1,3 +1,124 @@
 # Npm package
 
-soon
+## Installation via NPM Registry (recommended)
+
+The easiest way to install Garaga is via your prefered Node.js package manager, such as `npm` or `yarn`.
+
+1. Open your terminal or command prompt.
+2. Run the following command:
+
+    ```bash
+    npm i -S garaga
+    ```
+
+    or
+
+    ```bash
+    yarn add garaga
+    ```
+
+## Building the package from source code
+
+The package can be build directly from source code by cloning the garaga repository. Make sure you have both [Rust](https://www.rust-lang.org/tools/install) and [Node.js](https://nodejs.org/en/learn/getting-started/how-to-install-nodejs) installed in you machine.
+
+1. Open your terminal or command prompt.
+2. Install `wasm-pack` by running:
+
+    ```bash
+    cargo install wasm-pack
+    ```
+
+3. Run the following commands:
+
+    ```bash
+    git clone https://github.com/keep-starknet-strange/garaga.git
+    cd tools/npm/garaga_ts
+    npm ci
+    npm run build
+    npm pack
+    ```
+
+4. The .tgz file with the package contents will be available in the current folder.
+5. Install the .tgz file in your project
+
+    ```bash
+    npm i -S <path-to-tgz-package-file>
+    ```
+
+For reproducible builds, one can use instead docker compose. Make sure [docker](https://docs.docker.com/engine/install/) is installed in you machine.
+
+1. Open your terminal or command prompt.
+2. Run the following commands:
+
+    ```bash
+    git clone https://github.com/keep-starknet-strange/garaga.git
+    cd tools/npm/garaga_ts
+    docker compose up --build
+    ```
+
+3. The .tgz file with the package contents will be available in the current folder.
+4. Install the .tgz file in your project
+
+    ```bash
+    npm i -S <path-to-tgz-package-file>
+    ```
+
+## Development notes
+
+The Garaga NPM package is a mixed package. It is implemented in TypeScript but also reuses Rust code targeted to WebAssembly (WASM) with the help of [`wasm-pack`](https://rustwasm.github.io/wasm-pack/).
+
+The `src` folder is organized into two subfolders: `node` which contains the implementation in TypeScript; and `wasm` which has the interoperabilty code produced by `wasm-pack`.
+
+Changes to the TypeScript library should only be made to files under the `node` subfolder. Changes to the Rust implementation requires regenerating files under the `wasm` subfolder.
+
+Onces changes are in place they can be made permanent into the repository by committing the contents of both folders. Here is the bulk of the process:
+
+1. Open your terminal or command prompt.
+2. Use `git` to clone the repository:
+
+    ```bash
+    git clone https://github.com/keep-starknet-strange/garaga.git
+    cd tools/npm/garaga_ts
+    npm ci
+    ```
+
+3. If you make TypeScript only changes, you can quickly rebuild the package using the `build:node` NPM script:
+
+    ```bash
+    npm run build:node
+    npm pack
+    ```
+
+4. If instead you make Rust changes, it is necessary to generate the WASM interoperability code using the `build` NPM script:
+
+    ```bash
+    npm run build
+    npm pack
+    ```
+
+5. However, before commiting changes, it is necessary to generate the WASM interoperability code in a reproducible manner using docker:
+
+    ```bash
+    docker compose up --build
+    git commit .
+    ```
+### How `wasm-pack` is used to achieve interoperability
+
+Internaly the `build` NPM script uses `wasm-pack` to produce the WASM interoperability code. This is achieved by running
+
+    ```bash
+    cd tools/garaga_rs && wasm-pack build --target web --out-dir ../npm/garaga_ts/src/wasm/pkg --release --no-default-features
+    cd tools/npm/garaga_ts && node patch.wasm.cjs
+    ```
+Let's unpack it.
+
+In the Rust source folder we run `wasm-pack` in `--target web` mode. This generates TypeScript code targeting web pages.
+The `--release` option is required to minimize the size of the WASM module.
+And the `--no-default-features` disables the need to build non WASM features of garaga_rs.
+
+Once the `wasm-pack` is done, the code is generated under the folder `src/wasm/pkg` of garaga_ts that houses the TypeScript source code.
+
+We then run a custom script `patch.wasm.cjs` which makes minimal changes to the code generated by wasm-pack to facilitate seamless support of the WASM module in both the browser and Node.js.
+Basically it converts the WASM module to a [Base64](https://en.wikipedia.org/wiki/Base64) string that can be loaded in a portable way in both environments, amongst other minor tweaks.
+
+(It is important to note that the use of a custom script is only required so long `wasm-pack` itself does not provide a more portable/universal target mode.)

hydra/garaga/extension_field_modulo_circuit.py

@@ -176,6 +176,8 @@ def create_powers_of_Z(
             max_degree = self.extension_degree
         if isinstance(Z, PyFelt):
             Z = self.write_cairo_native_felt(Z)
+        elif isinstance(Z, int):
+            Z = self.write_cairo_native_felt(self.field(Z))
         elif isinstance(Z, ModuloCircuitElement):
             pass
         else:
@@ -566,100 +568,64 @@ def update_RHS_state(
         )
         return
 
-    def get_Z_and_nondeterministic_Q(
-        self, extension_degree: int, mock: bool = False
-    ) -> tuple[PyFelt, tuple[list[PyFelt], list[PyFelt]]]:
-        nondeterministic_Qs = [Polynomial([self.field.zero()]) for _ in range(2)]
-        # Start by hashing circuit input
-        if self.hash_input:
-            self.transcript.hash_limbs_multi(self.circuit_input)
-
+    def finalize_circuit(
+        self,
+        extension_degree: int = None,
+        mock=False,
+    ):
+        ######### Flags #########
+        extension_degree = extension_degree or self.extension_degree
         double_extension = self.accumulate_poly_instructions[1].n > 0
-
-        # Compute Random Linear Combination coefficients
         acc_indexes = [0, 1] if double_extension else [0]
+        #########################
 
-        for acc_index in acc_indexes:
-            for i, instruction_type in enumerate(
-                self.accumulate_poly_instructions[acc_index].types
-            ):
-                # print(f"{i=}, Hashing {instruction_type}")
-                match instruction_type:
-                    case AccPolyInstructionType.MUL:
-                        self.transcript.hash_limbs_multi(
-                            self.accumulate_poly_instructions[acc_index].Ris[i],
-                            self.accumulate_poly_instructions[acc_index].r_sparsities[
-                                i
-                            ],
-                        )
-                    case AccPolyInstructionType.SQUARE_TORUS:
-                        self.transcript.hash_limbs_multi(
-                            self.accumulate_poly_instructions[acc_index].Ris[i]
-                        )
-
-                    case AccPolyInstructionType.DIV:
-                        self.transcript.hash_limbs_multi(
-                            self.accumulate_poly_instructions[acc_index].Pis[i][0],
-                        )
+        ################ Get base rlc coefficient c0 ################
+        if self.hash_input:
+            self.transcript.hash_limbs_multi(self.circuit_input)
+        self.transcript.hash_limbs_multi(self.commitments)
 
-                    case _:
-                        raise ValueError(
-                            f"Unknown instruction type: {instruction_type}"
-                        )
+        c0 = self.write_cairo_native_felt(self.field(self.transcript.s1))
+        ##################################################################
+        ################ Compute Qs ################
+        Qs = [Polynomial([self.field.zero()]) for _ in range(2)]
 
+        for acc_index in acc_indexes:
+            self.accumulate_poly_instructions[acc_index].rlc_coeffs.append(c0)
+            # Computes Q = Σ(ci * Qi)
+            Qs[acc_index] = self.accumulate_poly_instructions[acc_index].Qis[0] * c0
+            for i in range(1, self.accumulate_poly_instructions[acc_index].n):
                 self.accumulate_poly_instructions[acc_index].rlc_coeffs.append(
-                    self.write_cairo_native_felt(self.field(self.transcript.RLC_coeff))
+                    self.mul(
+                        self.accumulate_poly_instructions[acc_index].rlc_coeffs[i - 1],
+                        c0,
+                    )
                 )
-            # Computes Q = Σ(ci * Qi)
-            for i, coeff in enumerate(
-                self.accumulate_poly_instructions[acc_index].rlc_coeffs
-            ):
-                nondeterministic_Qs[acc_index] += (
-                    self.accumulate_poly_instructions[acc_index].Qis[i] * coeff
+                Qs[acc_index] += (
+                    self.accumulate_poly_instructions[acc_index].Qis[i]
+                    * self.accumulate_poly_instructions[acc_index].rlc_coeffs[i]
                 )
-
-            # Extend Q with zeros if needed to match the expected degree.
-            nondeterministic_Qs[acc_index] = nondeterministic_Qs[acc_index].get_coeffs()
-            nondeterministic_Qs[acc_index] = nondeterministic_Qs[acc_index] + [
-                self.field.zero()
-            ] * (
-                (acc_index + 1) * extension_degree
-                - 1
-                - len(nondeterministic_Qs[acc_index])
+            Qs[acc_index] = Qs[acc_index].get_coeffs()
+            # Extend Q with zeros if needed to match the minimal expected degree.
+            Qs[acc_index] = Qs[acc_index] + [self.field.zero()] * (
+                (acc_index + 1) * extension_degree - 1 - len(Qs[acc_index])
             )
-        # HASH(COMMIT0, COMMIT1, Q0, Q1)
-        # Add Q to transcript to get Z.
-        if not mock:
-            self.transcript.hash_limbs_multi(nondeterministic_Qs[0])
-            if double_extension:
-                self.transcript.hash_limbs_multi(nondeterministic_Qs[1])
+        ##################################################################
 
-        Z = self.field(self.transcript.continuable_hash)
-
-        return (Z, nondeterministic_Qs)
-
-    def finalize_circuit(
-        self,
-        extension_degree: int = None,
-        mock=False,
-    ):
-        # print("\n Finalize Circuit")
-        extension_degree = extension_degree or self.extension_degree
+        Q = [self.write_elements(Qs[0], WriteOps.COMMIT)]
 
-        z, Qs = self.get_Z_and_nondeterministic_Q(extension_degree, mock)
         compute_z_up_to = max(max(len(Qs[0]), len(Qs[1])) - 1, extension_degree)
-        # print(f"{self.name} compute_z_up_to: {compute_z_up_to}")
-
-        Q = [self.write_elements(Qs[0], WriteOps.COMMIT)]
-        double_extension = self.accumulate_poly_instructions[1].n > 0
+        self.big_q_len = len(Q[0])
 
+        self.transcript.hash_limbs_multi(Q[0])
         if double_extension:
             Q.append(self.write_elements(Qs[1], WriteOps.COMMIT))
+            self.transcript.hash_limbs_multi(Q[1])
             compute_z_up_to = max(compute_z_up_to, extension_degree * 2)
+            self.big_q_len = self.big_q_len + len(Q[1])
 
-        self.create_powers_of_Z(z, mock=mock, max_degree=compute_z_up_to)
+        z = self.transcript.continuable_hash
 
-        acc_indexes = [0, 1] if double_extension else [0]
+        self.create_powers_of_Z(z, mock=mock, max_degree=compute_z_up_to)
 
         for acc_index in acc_indexes:
             for i in range(self.accumulate_poly_instructions[acc_index].n):
@@ -710,7 +676,6 @@ def finalize_circuit(
                 else:
                     eq_check = self.sub(rhs, lhs)
                     self.extend_output([eq_check])
-
         return True
 
     def summarize(self):
@@ -740,12 +705,12 @@ def compile_circuit_cairo_zero(
                 "add_offsets_ptr",
                 "mul_offsets_ptr",
                 "output_offsets_ptr",
-                "poseidon_indexes_ptr",
             ],
             "felt": [
                 "constants_ptr_len",
                 "input_len",
                 "commitments_len",
+                "big_Q_len",
                 "witnesses_len",
                 "output_len",
                 "continuous_output",
@@ -759,7 +724,6 @@ def compile_circuit_cairo_zero(
         },
     ) -> str:
         dw_arrays = self.values_segment.get_dw_lookups()
-        dw_arrays["poseidon_indexes_ptr"] = self.transcript.poseidon_ptr_indexes
         name = function_name or self.values_segment.name
         function_name = f"get_{name}_circuit"
         code = f"func {function_name}()->(circuit:{self.class_name}*)" + "{" + "\n"
@@ -774,6 +738,7 @@ def compile_circuit_cairo_zero(
         code += f"let input_len = {len(self.values_segment.segment_stacks[WriteOps.INPUT])*N_LIMBS};\n"
         code += f"let commitments_len = {len(self.commitments)*N_LIMBS};\n"
         code += f"let witnesses_len = {len(self.values_segment.segment_stacks[WriteOps.WITNESS])*N_LIMBS};\n"
+        code += f"let big_Q_len = {self.big_q_len*N_LIMBS};\n"
         code += f"let output_len = {len(self.output)*N_LIMBS};\n"
         continuous_output = self.continuous_output
         code += f"let continuous_output = {1 if continuous_output else 0};\n"
@@ -827,14 +792,6 @@ def compile_circuit_cairo_zero(
                     for val in dw_values:
                         code += f"\t dw {val};\n"
 
-            elif dw_array_name in [
-                "poseidon_indexes_ptr",
-            ]:
-                for val in dw_values:
-                    code += (
-                        f"\t dw {POSEIDON_BUILTIN_SIZE*val+POSEIDON_OUTPUT_S1_INDEX};\n"
-                    )
-
         code += "\n"
         code += "}\n"
         return code