Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check for coinbase maturity #128

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
merge main
  • Loading branch information
maciejka committed Sep 3, 2024
commit 2c111803e7f6ed42e4c8f6db455c474e766beba2
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,5 @@ Cargo.lock

.env
.venv
.python-version
.python-version
__pycache__
43 changes: 30 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,52 +49,68 @@ Although this is a highly experimental project without immediate plans for deplo

## Roadmap

### Milestone 1 - Block header verification
### Milestone 1 - Block header validation

Implement a reduced light client that can verify a range of blocks starting at genesis.
It does not have to validate execution, just check that the block header fields follow the protocol.

Tasks:

* [ ] block hash computation
* [x] block hash computation
* [x] proof-of-work validation/computation
* [x] block time validation/computation
* [ ] block difficulty adjustment
* [ ] transaction ID calculation
* [x] transaction root computation
* [x] block difficulty adjustment
* [x] script for fetching arbitrary block data
* [ ] script for running the program e2e for a span of blocks
* [ ] script for preparing program arguments
* [ ] script for running the program e2e for multiple block

### Milestone 2 - Partial transaction verification
### Milestone 2 - Partial transaction validation

Extend light client with partial transaction validation, but without UTXO checks.

Tasks:

* [ ] reassess validation check list (analyze Bitcoin core codebase)
* [ ] generate & run integration tests e2e instead of Cairo codegen
* [x] transaction ID calculation
* [x] transaction root computation
* [x] validate transaction fee
* [x] validate coinbase transaction
* [ ] validate segwit specific data
* [ ] validate that transaction can be mined (locktime, sequence, coinbase maturity)
* [ ] validate segwit specific data (wtxid commitment)
* [ ] validate block weight
* [x] script that fetches blocks extended with references UTXOs
* [ ] script that runs the program e2e for a span of blocks

### Milestone 3 - UTXO set verification
### Milestone 3 - Bitcoin script validation

Try to run script validation with external Cairo crate.

Tasks:

* [ ] Integrate Shinigami

### Milestone 4 - UTXO set verification

Add inclusion proofs for the UTXOs included in the block.

Tasks:

* [ ] implement cache for UTXOs spent in the same block they are created
* [ ] isolate unspendable outputs (OP_RETURN, etc)
* [ ] implement cache for UTXOs spent in the same block they are created (*)
* [ ] implement transaction outpoint hashing
* [ ] implement Utreexo accumulator (addition)
* [ ] Utreexo backend that maintains utxo set and Utreexo roots
* [ ] implement Utreexo single inclusion proof verification
* [ ] implement Utreexo single output removal
* [ ] implement Utreexo bridge node that generates individual inclusion proofs
* [ ] implement script that runs the program e2e for a span of blocks

### Milestone 4 - Full consensus verification
### Milestone 5 - Full consensus validation

Verify full execution, including the Bitcoin scripts check.
Validate full block execution, including the Bitcoin scripts checks and Utreexo.

### Milestone 5 - Incrementally verifiable computation
### Milestone 6 - Proving the validation

Recursively verify STARK proofs of chain state updates.

Expand Down Expand Up @@ -180,6 +196,7 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/manlikeHB"><img src="https://avatars.githubusercontent.com/u/109147010?v=4?s=100" width="100px;" alt="Yusuf Habib"/><br /><sub><b>Yusuf Habib</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/raito/commits?author=manlikeHB" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Gerson2102"><img src="https://avatars.githubusercontent.com/u/71728860?v=4?s=100" width="100px;" alt="Gerson"/><br /><sub><b>Gerson</b></sub></a><br /><a href="https://github.com/keep-starknet-strange/raito/commits?author=Gerson2102" title="Code">💻</a></td>
</tr>
</tbody>
</table>
Expand Down
1 change: 1 addition & 0 deletions Scarb.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ edition = "2024_07"
[scripts]
get_block= "python3 ./scripts/data/get_block.py"
get_blocks= "./scripts/data/get_blocks.sh"
light_client= "./scripts/data/light_client.sh"

[dependencies]

Expand Down
75 changes: 75 additions & 0 deletions scripts/data/felt_writer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# This module provides helpers for serializing Python objects into Cairo runner arguments (felts and arrays).
#
# Supported types:
# int (unsigned) -> felt252
# [0-9] str -> u256 which is (felt252, felt252)
# 0x str (reversed) -> Hash which is [u32; 8] which is a tuple of felt252
# bytes -> ByteArray which is (Array<felt252>, felt252, felt252)
# list<int> -> Array<felt252>
# dict<str, V> -> (V, V, ...)
import sys
import os
import json
from pathlib import Path

def serialize(obj):
if isinstance(obj, int):
# This covers u8, u16, u32, u64, u128, felt252
assert(obj >= 0 and obj < 2 ** 252)
return obj
elif isinstance(obj, str):
if obj.startswith('0x') and len(obj) == 66:
# Hex string into 4-byte words then into BE u32
rev = list(reversed(bytes.fromhex(obj[2:])))
return tuple(int.from_bytes(rev[i:i+4], 'big') for i in range(0, 32, 4))
else:
# Try to cast to int and then to a pair of felts
num = int(obj)
assert(num >= 0 and num < 2 ** 256)
lo = num % 2 ** 128
hi = num // 2 ** 128
return (lo, hi)
elif isinstance(obj, bytes):
# Split into 31-byte chunks and save the remainder
num_chunks = len(obj) // 31
main_len = num_chunks * 31
rem_len = len(obj) - main_len
main = [int.from_bytes(obj[i:i+31], 'big') for i in range(0, main_len, 31)]
# TODO: check if this is correct
rem = int.from_bytes(obj[main_len:].rjust(31, b'\x00'), 'big')
return (main, rem, rem_len)
elif isinstance(obj, list):
return list(map(serialize, obj))
elif isinstance(obj, dict):
return tuple(map(serialize, obj.values()))
elif isinstance(obj, tuple):
return obj
else:
raise NotImplementedError(obj)

def flatten_tuples(src):
res = []
def iter_obj(obj):
if isinstance(obj, int):
res.append(obj)
elif isinstance(obj, list):
res.append(obj)
elif isinstance(obj, tuple):
for item in obj:
iter_obj(item)
else:
raise NotImplementedError(obj)
iter_obj(src)
return res

def main():
try:
if (len(sys.argv) != 2):
raise TypeError("Error: bad arguments")
args = json.loads(Path(sys.argv[1]).read_text())
res = flatten_tuples(serialize(args))
print(res)
except Exception as error:
print(error)

main()
8 changes: 4 additions & 4 deletions scripts/data/get_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,8 @@ def tx_input(tx):


def check_segwit(tx):
if not int(tx["version"]) % 2:
hex = tx["hex"][8:12] # Skip the first 4 bytes (version) and take the next 4 bytes (marker + flag)
if hex == "0001":
return "true"
return "false"

Expand Down Expand Up @@ -153,15 +154,14 @@ def main():
file_payload = f'''// THIS CODE IS GENERATED BY SCRIPT, DO NOT EDIT IT MANUALLY

use raito::types::transaction::{{Transaction, TxIn, TxOut, OutPoint}};
use raito::types::block::{{Block, Header}};
use raito::types::block::{{Block, Header, TransactionData}};
use raito::utils::hex::from_hex;
use raito::utils::hash::Hash;

// block_hash: {block_hash}
pub fn block_{block["height"]}() -> Block {{
Block {{
header: {header(block)},
txs: array![{txs_process(txsraw, block["nTx"])}].span(),
data: TransactionData::Transactions(array![{txs_process(txsraw, block["nTx"])}].span()),
}}
}}'''

Expand Down
1 change: 0 additions & 1 deletion scripts/data/get_blocks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ get_single_block() {

main() {
local block_hashes=(
"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f" # Genesis block (0)
"00000000d1145790a8694403d4063f323d499e655c83426834d4ce2f8dd4a2ee" # Block containing first P2P tx to Hal Finney (170)
"00000000132fbe8314fc571c0be60b31ccd461c9ee85f42bde8c6d160a9dacc0" # Block containing first off ramp tx from Martti Malmi (24835)
"00000000152340ca42227603908689183edc47355204e7aca59383b0aaac1fd8" # Block containing pizza tx (57043)
Expand Down
8 changes: 8 additions & 0 deletions scripts/data/light_client.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -e;
set -o pipefail;

INPUT_FILE="./scripts/data/light_client_args_170.json"
ARGUMENTS=$(python ./scripts/data/felt_writer.py "$INPUT_FILE")

scarb cairo-run "$ARGUMENTS"
34 changes: 34 additions & 0 deletions scripts/data/light_client_args_170.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"chain_state": {
"block_height": 169,
"total_work": "730155581610",
"best_block_hash": "0x000000002a22cfee1f2c846adbd12b3e183d4f97683f85dad08a79780a84bd55",
"current_target": "26959535291011309493156476344723991336010898738574164086137773096960",
"epoch_start_time": 1231006505,
"prev_timestamps": [
1231702618,
1231703466,
1231704197,
1231704464,
1231714917,
1231715347,
1231716245,
1231716329,
1231716711,
1231717181,
1231730523
]
},
"block": {
"header": {
"version": 1,
"time": 1231731025,
"bits": 486604799,
"nonce": 1889418792
},
"data": {
"variant_id": 0,
"merkle_root": "0x7dac2c5666815c17a3b36427de37bb9d2e2c5ccec3f8633eb91a4205cb4c10ff"
}
}
}
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.