Replace Envelope with Elem.SuccessElem

[dantb]

Fixes #3745

[dantb]

Doesn't feel great that this is optional - most of the concrete types for which we're querying elems have a project and it's a mandatory field in the scoped tables. I guess it's because Envelope was used for both global and scoped?

[shinyhappydan]

It's confusing how the elem wouldn't already have the project set since it's coming from within it..?

[imsdu]

You should be able to populate this field from the original SQL query.
Yes, it was because Envelope and Elem were designed to work for both global and scoped entities.
Not sure if there is still a use case for global entities though now that we got rid of the sse events for them.

[shinyhappydan]

What was a stream of Envelopes, and what is a stream of SuccessElems?

[shinyhappydan]

It's confusing how the elem wouldn't already have the project set since it's coming from within it..?

[dantb]

Looking into why the project was optional led me to try to understand the usage of elem / envelope before... My feeling is it's used in places it shouldn't be which leads to incidental complexity like streams of Elem where nothing on the Elem is ever used. The usages I've seen:

1. Streaming from global tables. Now we use GlobalStateValue purely for the Read instance when streaming from global_states. Only value is needed.
2. Streaming from scoped tables. This seems fine. However for the projections, the coordinators were using a stream of Elem rather than SuccessElem for views. This meant the type suggested that there could be failing / dropped elements, but there would never be. I've changed that now.
3. Streaming from projection_restarts. No project here, I haven't modified this.
4. Streaming from composite_restarts. There is a project here.
5. Streaming graph resources - haven't yet gone down this rabbit hole.

3 and 5 mean the project remains optional. I don't understand enough to comment on what should be used and where, but it's still pretty confusing in its current state. So I still can't really answer your question @shinyhappydan 😂 maybe @imsdu can!

Also deleted a load of unused streaming methods from the event stores and logs. Probably enough for now!

[imsdu]

Maybe you could use a tuple (S, Offset) here ?
Or make GlobalStateValueprivate ?

[dantb]

Thanks, less code!