[renderers-rust] Generated rust arrays longer than 32 do not serde Serialize

[wjthieme]

Arrays longer than 32 cannot be Serialized with serde directly (catch-32). I suspect that is why the #[cfg_attr(feature = "serde", serde(with = "serde_with::As::<serde_with::Bytes>"))] proc macro is added to array fields longer than 32.

This however still fails with the following error:

the trait bound `serde_with::Bytes: SerializeAs<[T; N]>` is not satisfied

Example: The TickArray account in https://github.com/orca-so/whirlpools/tree/main/rust-sdk/client when compiled with serde feature enabled. This account has a field pub ticks: [Tick; 88] which it fails on.

[lorisleiva]

Hey Will, thanks for raising this!

May I ask, do you need the serde feature/traits in your case?

@febo is the mastermind behind the Rust client and I know he's got some plans to make some of these traits configurables so you don't have to worry about them if you don't need them. He's OOO at the moment but might be able to help with this when he's back.

Otherwise, do you have an idea of what I could add to the generated Rust code to make this work?

[wjthieme]

Don't think we'd immediately need the 'serde' feature on the Codama generated code but would be nice to have for any consumer of our sdk (in case anyone needs it).

Since this is just a one-off case we currently have a custom (de)serializer for the field.

/// Serialize the fixed size tick array as a vector.
/// This is needed because Serde only supports fixed size arrays with <= 32 elements.
pub fn serialize_ticks<S>(value: &[Tick], serializer: S) -> Result<S::Ok, S::Error>
where
    S: Serializer,
{
    let mut seq = serializer.serialize_seq(Some(TICK_ARRAY_SIZE_USIZE))?;

    for element in value {
        seq.serialize_element(element)?;
    }

    seq.end()
}

/// Deserialize the fixed size tick array from a vector.
/// This is needed because Serde only supports fixed size arrays with <= 32 elements.
pub fn deserialize_ticks<'de, D>(deserializer: D) -> Result<[Tick; TICK_ARRAY_SIZE_USIZE], D::Error>
where
    D: Deserializer<'de>,
{
    let vec = <Vec<Tick>>::deserialize(deserializer)?;

    vec.try_into().map_err(|_| de::Error::custom("wrong size"))
}

[wjthieme]

Using #[cfg_attr(feature = "serde", serde(with = "BigArray"))] instead seems to work but it requires the serde-big-array crate.

Supposedly serde_with also has something for arrays larger than 32 but I couldn't get that to work: https://docs.rs/serde_with/latest/serde_with/#large-and-const-generic-arrays

[lorisleiva]

Hey @wjthieme, I think this PR maybe help with this issue.

Let me know what you think. 🙏

[wjthieme]

@lorisleiva Awesome! Does this only work for Derive traits or can I also use it to add the serde(with = "BigArray") attribute?

Either way. the attribute that gets added to large arrays now (#[cfg_attr(feature = "serde", serde(with = "serde_with::As::<serde_with::Bytes>"))]) does not compile so that should probably be changed

[lorisleiva]

It only works for Derive traits right now but we could extend the options to provide custom attributes if necessary. We tried to focus on the options that would have the most impact for people right now but let me know if there's anything else we can work on to unblock you.

[wjthieme]

Probably the easiest fix would be to replace #[cfg_attr(feature = "serde", serde(with = "serde_with::As::<serde_with::Bytes>"))] with something that works. I kinda like serde(with = "BigArray") from serde-big-array crate but that would mean having to add an extra dependeny