Create new share repo

[rootulp]

Summary

There exist two copies of share related code.

1. https://github.com/celestiaorg/celestia-core/blob/v0.34.x-celestia/types/share_splitting.go
2. https://github.com/celestiaorg/celestia-app/blob/main/pkg/shares/share_splitting.go

Problem Definition

It will be difficult to maintain two copies of this code across separate repos because changes applied to one will not immediately apply to the other. celestia-core needs some share logic (ex. SplitTx) but ideally this application specific logic would live in our application specific repo (celestia-app). celestia-core can't import this code from celestia-app because that would introduce a circular dependency.

Proposal

Create a new repo and public Go module that can be depended on by both celestia-core and celestia-app.

Note: the celestia-app version has been refactored (see celestiaorg/celestia-app#637) so it should likely be the candidate for migration.

Pros

• Moves share logic out of celestia-core and therefore minimizes the diff between celestia-core and tendermint/tendermint.

Cons

• Additional version friction when developing across repos. In other words, a change that modifies share logic entails making the change, cutting a new release, bumping to that version in both consuming apps, and then using that change where applicable.

Alternatives

Keep share logic in celestia-core. Delete share logic from celestia-app. Import share code from celestia-core into celestia-app. This works and does involve adding any new dependencies.

Ref: https://github.com/celestiaorg/celestia-app/blob/13076a62c567f624b2508752b7e129467cd71bf9/go.mod#L164

Blocked by

• Tracking: Non-interactive defaults celestia-app#382

Blocking

• Delete share related code #842
• Move share constants to celestia-app #841

[evan-forbes]

Another thing to note is that the new encoding code relies entirely on the use of malleated transactions, a concept only known to the app. In fact, properly testing the share encoding and decoding logic is impossible with out the use of similar if not identical application logic encoded in PreprocessProposal. This was the entire reason why the splitting and merging logic was moved to the app in the first place.

The best solution is to remove the need for logic in core, and then only keep this logic in the application where it can be properly tested and easily maintained. Unfortunately, this is not possible due to us needing to provide inclusion proofs to txs, and tendermint's keeping the rpc server and block store within its scope instead of the application's (IOTWAL).

[evan-forbes]

I know I keep changing my preference for what we should do, but given that each option is terrible in its own way, we might want to continue chewing on this and eventually find a solution that isn't.