Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/zero-copy-collections #12

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

ebrightfield
Copy link

@ebrightfield ebrightfield commented Oct 15, 2024

Context

Note: This is an alternative implementation of the previous Header* traits I drafted. I'm not sure whether this is better, main "objective improvement" in this PR is just working it into AsAccount.

One of the pain points of Pod account data is storing collections of items, especially collections whose lengths may differ across different accounts.

More generally, lots of times it's nice to "parse a header first, then parse the rest," as demonstrated by programs in the wild like Bonfida's agnostic-orderbook, SPL Account Compression, or this random dude's account history program.

In Steel's case, the header type can be Discriminator + Pod. Then, using the header, the remaining bytes can be deserialized. For example, collections could resolve to either generically sized arrays, or structs with slices whose lengths are stored in the header. This PR provides a way to define these fancy "two-step" types with the AsAccount interface.

Implementation

  • Bolted on a couple new trait methods to the AsAccount interface.
  • Added traits FromHeader and FromHeaderMut. Removed the old header-related traits.
  • Added a bunch of tests at the bottom to demonstrate usage and behavior. Can / should maybe slim them down and make some examples.
  • Any type that implements FromHeader<'_, T> or FromHeaderMut<'_, T> for some T: AccountDeserialize can be used in the following ways:
        //  SliceAccount: FromHeader<_, T>
        //  T: AccountDeserialize
        let foo = SliceAccount::from_header(&data).unwrap();
        
        let account: AccountInfo<'_>;
        let foo = account
            .as_account_mut_with_header::<_, SliceAccountMut>(&owner)
            .unwrap();

A Couple Call-Outs

  • Definitely was a challenge to nail down something flexible, small surface area, and ergonomic on the user side. I think I struck a decent balance, but very open to changes. I wonder if there are more lifetime generics I can remove.
  • I had to split up the header traits into immutable and mutable halves, because each requires different structs to represent either shared or mutable access to collections stored as immutable vs mutable slices respectively.
struct Foo<'a> {
    bar: &'a [u64],
}
// impl<'a, T> FromHeader<_, T> for Foo<'a>

struct FooMut<'a> {
    bar: &'a mut [u64],
}
// impl<'a, T> FromHeaderMut<_, T> for FooMut<'a>

@ebrightfield ebrightfield force-pushed the feature/zero-copy-collections branch from eb144a5 to f763cf1 Compare October 15, 2024 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant