Stacks Hyperchains (fka subnets)

[kantai]

Overview

The Stacks network was designed with several decentralization axioms. When it comes to overall network throughput, the most important of these axioms are:

Open Membership. The Stacks network allows anyone to become a miner, and therefore the consensus algorithm must have either a Proof-of-Work limit on each produced block or some other globally available ordering (Proof-of-Transfer uses the Bitcoin chain for this).

User verifiable. The Stacks network assumes that any user is able to validate and verify the working state of the Stacks blockchain. This requires that each block be processable on commodity hardware well within the time limit imposed by the open membership scheme. This also requires that the total network bandwidth required for downloading blocks be far less than commonly available user network speeds.

By loosening these decentralization axioms, the overall throughput of the network could be increased. However, rather than force the network overall to accept a "lower level" of decentralization, the goal of Stacks subnets is to allow users to accept less decentralization and increased risks for themselves in order to increase their own network throughput.

A Stacks subnet is an independent blockchain, with its own consensus rules and its own network limits that is ultimately able to materialize changes on the Stacks chain. This is a broad definition, but the critical elements are that the subnet is able to make progress independently of the Stacks blockchain and that users are able to move Stacks assets into the subnet and out of the subnet. This discussion outlines three implementations of subnets with different trust assumptions and ultimately proposes a plan for implementing subnets that Hiro is planning to pursue. The primary concern of these implementations is the mechanics of moving Stacks assets into and out of the subnet.

Presentation of Content

Most of the content of this proposal and topic were outlined in the most recent Blockchain architecture meeting. The slides are attached to this topic here

Subnet Goals

Layer 2. Subnets are blockchains which can confirm transactions and produce blocks independent of the Stacks blockchain.

Asset Interaction. Subnets can interact with Stacks-defined assets by depositing those assets into the subnet and then withdrawing out of the subnet.

Multiple Subnets. The Stacks chain can support many subnets with different consensus rules, trust assumptions, and intended use cases (from general to application specific).

Lossy Blocks. Subnet blocks should be lossy with respect to the Stacks chain: subnet blocks should be capable of much more data storage, mutations, and general operations than Stacks blocks. These operations should not have a 1-1 correspondence with operations on the Stacks chains when processing withdrawals (i.e., operations in the subnet should "roll-up" into operations on the Stacks chain during withdrawals).

Stacks Consensus Neutral. Any implementation of subnets in the Stacks ecosystem should not require any consensus rules changes on the Stacks chain. In this sense, subnets projects are neutral with respect to the Stacks main chain's consensus rules.

Interactions with Subnets

Stacks subnets run Clarity VMs, allowing them to process and validate transactions exactly like the Stacks mainnet chain:

• The transaction wire format remains the same
• Stacks node event emitters remains the same
• Subnet nodes expose a strict subset of the stacks-node RPC interface

Addresses in subnets use the same version bytes as the main chain, but transactions will use different ChainId bytes (so that transactions in subnets cannot be replayed in the main chain).

Wallets and other kinds of blockchain clients will require updates to interact with subnets. Namely, the clients will need to be able to alter transaction ChainId bytes and be able to select different API backends for specific subnets.

Trust in Subnets

While participation in subnets is optional, once a user begins participating in a subnet, the amount of trust placed in the subnet miners will vary depending on the consensus rules of the subnet and the subnet's deposit and withdrawal scheme.

In a fully-trusted scheme, subnets miners are responsible for issuing subnet blocks and users can validate those blocks. However, withdrawals from the subnet are issued arbitrarily by a subnet miner. This trust can be federated with a BFT protocol with the miners for block issuance. This federation of trust would require that the majority of miners approve any withdrawals.

In a fully-trustless scheme, subnets miners are responsible for issuing subnet blocks, users can validate those blocks, and withdrawals are issued only if they correspond to a correct state in the most recent valid subnet block. Such a scheme would create an ideal subnet, but it requires either that the subnet blocks be validated on the Stacks chain (violating the goal of "lossiness"), or it requires novel cryptographic techniques (PCPs) which remain an open research topic.

Finally, in an incentivized trust scheme, miners may issue arbitrary withdrawals, but they may be punished for doing so. These schemes introduce reward and punishment knobs that can be adjusted to incentivize miners to behave correctly.

Hiro Subnet Proposal

Hiro is proposing and planning to implement an iterative approach to subnets. This plan emphasizes the lossiness goal, as opposed to approaches discussed elsewhere such as peer swaps and vector-clocking
merkle trees.

The plan is to begin with a fully-trusted approach, federating miners using a BFT protocol. This approach will use a Stacks smart contract to handle deposits and withdrawals, and will require some new P2P messages for miners to communicate on block construction.

The first iteration on this is to introduce an incentive scheme for processing user to user asset transfers on the subnet. This scheme would protect asset transfers between users, but not between smart contracts and users. Such a scheme can be implemented through a combination of merkle proofs and miner responsiveness challenges.

Further iterations on this scheme will involve a fair amount of research and could be categorized as far-future work. The next step after the first iteration would be to update the incentivization scheme to protect smart contract to user asset transfers. This requires validation of contract execution and is similar to the solution posed by Arbitrum. This would likely require new functionality in the Clarity VM. The final approach would be to develop a full trustless scheme using PCPs as the theoretical framework for a Stacks smart contract to act as a "verifier" to a layer 2 "prover". This is similar to the promise of ZK rollups, and would require similar amount of refinement and research to discover if the approach was workable.

Fully Trusted Approach with BFT

The first implementation of subnets will use a full trusted approach, federating that trust among miners using a BFT protocol. In this approach, a Stacks smart contract would govern the subnet, specifying who the miners are (and if there are "view changes", materializing them in the contract), accepting deposits, and processing withdrawals.

The subnet miners (BFT participants) would run a stacks-subnet-node, which is a new binary that is either a fork of the stacks-blockchain repository, or preferably a client of the library outputs of the stacks-blockchain repository. This node accepts transactions, exposes the normal RPC interface, emits events for API clients, monitors the Stacks chain for withdrawal requests, and implements the BFT protocol between miners for building and issuing blocks.

Subnet users may, but do not need to run nodes. They will need to use an RPC interface or API endpoint to query the subnet state and submit transactions. Subnet APIs and explorers would need to run follower
nodes.

Stacks contract interface

The entry point for withdrawals and deposits into the subnet would be a subnet smart contract. This smart contract would have the following interface:

(define-read-only (get-miners))
 ;; returns (list principal) of the subnet’s miners

(define-public (deposit-ft (fungible-token <ft-trait>) (amount uint))
 ;; deposits a fungible token in the subnet

(define-public (deposit-nft (non-fungible-token <nft-trait>) (nft-id uint))
 ;; deposits a non-fungible token in the subnet

(define-public (request-ft-withdrawal (ft-contract principal) (amount uint))
 ;; initiates a ft-withdrawal request on behalf of the tx-sender, returns a withdrawal ID

(define-public (approve-ft-withdrawal (withdrawal-id uint))
 ;; invoked by a miner to approve a pending withdrawal

(define-public (execute-ft-withdrawal (fungible-token <ft-trait>) (withdrawal-id uint))
;; once a withdrawal has been approved, execute the withdrawal by invoking the ft contract

Each of these methods would produce transaction events via (print) statements. These transaction receipts will be consumed by miners' stacks-subnet-node instances in order to monitor for deposits and withdrawals. Note that in order to protect the subnet contract from malicious token implementations, the smart contract will need an "approved list" of tokens.

Semi-Trusted Approach with BFT

The first iteration of the trusted BFT approach will introduce miner staking. These staked assets will be slashed in the event that miners misbehave in the processing of withdrawals. Miners' stakes are deposited into the subnet's smart contract.

Subnet users that deposit into the contract will receive a deposit identifier. Subnet transactions will operate on whole deposits, and deposits can be divided by a Stacks chain transaction on the subnet contract.

Periodically, subnet blocks are written as a commitment to the Stacks smart contract. This commitment is the merkle root hash reflecting the current state of deposit holders and the signed-by-sender transforms since the last commitment. This merkle tree is keyed by deposit identifier and has a value of the current owner and the list of signed transformations that resulted in this ownership. When a withdrawal is made, it must include the path in the current block.

The subnet's smart contract contains an interface for issuing a responsiveness challenge to the miners-- this a contract call that creates a pending challenge for the miners: asking for the merkle path and value of a given deposit ID in a committed to block. Once accepted into the Stacks chain, miners have a window of time within which they must respond. Otherwise, the miners forfeit their stakes.

[gregorycoppola]

It would be good if whatever we implement can naturally grow from "v1" to eventual versions we will make later.

I had an idea to use the MARF to allow trusted withdrawal from a sub-net.

1. User asks sub-chain to authorize a withdrawal, from sub-chain to parent chain.
2. Sub-chain sends a SNARK of a MARF proof to the parent chain, to show that the user really has that balance (on that fork).
3. Parent chain verifies the SNARK'ed MARF proof, and allows the withdrawal. Because SNARK's are small (~300 bytes), the entire proof can be written to the parent block chain.

Then, we don't have to build SNARK's for arbitrary computation, which is a hard problem, but just for the MARF proof. That means we only really need one SNARK circuit.

[gregorycoppola]

On the topic of "Proof of Collateralization", I'm not saying it's in the existing proposal.

I'm just interested in the space of possible/feasible "sub-net" architectures, which we can keep in mind as we implement the first concrete sub-net. So, possibly we can identify avenues for generalization as we build the first version.

One could imagine a structure in which miner membership is open, and then driven by proof of burn/collateralization.

[kantai]

Right -- this approach would be pretty much equivalent to the "Fully Trustless" approach discussed briefly in this proposal (and slightly less briefly in the linked presentation). Ultimately, a SNARK based approach, in order to provide any trust advantage over the higher trust requirement schemes, needs to use its PCPs to prove the following:

1. Each subnet block is valid.
2. A given subnet block is the most recent.
3. A given withdrawal request is valid with respect to the most recent subnet block.

A MARF proof only proves the third statement. SNARKs are theoretically capable of proving statements 1 and 2, but it involves proofs that the computation of each subnet block is correct, and the consensus algorithm for mining is being applied correctly.

As for making the implementation generic enough to admit this, that is definitely a goal for the first implemenation of subnets. In most cases, the differences between the subnet variations comes down to differences in their Stacks smart contract. However, as with any attempts to make something generic, the real test of the abstraction boundaries we make will come when the variations are actually implemented.

[gregorycoppola]

1. We don't actually have to build out the generalizations in the API right away. But, even can just conceptually understand what can be generalized.

2. I think it's not just about being the most recent block, but about being on the ultimately winning fork.

[jcnelson]

Do subnets fork?

[kantai]

Do subnets fork?

Some variants could fork independently of the Stacks chain, but in the first iteration, no, a subnet would only "fork" in the event of a Stacks chain fork (something close to the relationship of Stacks 1.0 to Bitcoin).