services/soroban-rpc: getEvents method

[paulbellamy]

epic - stellar/stellar-rpc#43

depends on api design and data acquisition

See design output from #4676

For Nov v1, get ledgers from Horizon (until long term backend pattern is established)

• need to get websocket setup
• wiring JSON rpc
• push notifications

[paulbellamy]

Depends on #4558

[sreuland]

@paulbellamy , scope-wise, what is the expectation on interface of the subscription, any request params, like filter on specific event types, etc, or just gets all. is the ws subscription endpoint defined in soroban api docs for ref? or is this effort expected to update the api docs? What event models are expected to be published, any links to source code or docs on those?

[paulbellamy]

Default is gets all. Should allow filtering via contract for v1. That's probably enough for now.

endpoint is defined in the soroban-rpc api docs, yes. But there are some undefined parts there (like event structure). So design those and update the docs on confluence.

[paulbellamy]

Note, the events here are coming from the txmetas emitted by the soroban-rpc captive core. No persistence, just "ingest" the txmetas as they arrive, and see if any of the events match the active subscriptions.

[sreuland]

@paulbellamy @tamirms , just in short term for the rpc server, could it potentially proxy from horizon api, i.e. http client to streaming(sse) endpoint for transactions to obtain same txmeta? Just wondering if that option is worth considering as stop-gap in short term if it's less effort/complexity?

[tamirms]

@sreuland I think that's a good idea. I think we still need captive core for the following endpoints as this info is not available in Horizon's API:

getContract
getContractData
simulateTransaction (Meridian)

But perhaps we could proxy those requests to horizon's captive core instead of spawning our own instance.

[sreuland]

@tamirms , ok, for those endpoints that need core, the goal is to package up dev stack onto quickstart docker image which will provide a singular core instance that should be configurable to be in standalone/futurenet and which both horizon and sorobon-rpc services hosted in same docker container should be able to simultaneously connect to that single core instance txmeta pipe/rpc ports, will that work?

quickstart spike/discussion on this packaging - #4590 (comment)

[tamirms]

both horizon and sorobon-rpc services hosted in same docker container should be able to simultaneously connect to that single core instance txmeta pipe/rpc ports

@sreuland both horizon and soroban can access a shared core instance via the http ports. However, the txmeta pipe cannot be broadcast to more than one listener unless we write some code to allow that. But, if we obtain txmeta in soroban-rpc via horizon then we will not need to read from the captive core txmeta pipe in the soroban-rpc service

[sreuland]

@tamirms , thanks for catching that, if obtain txmeta in soroban-rpc via horizon means doing sql to horizon's ingested rdbms, then I think this proposed image layout for quickstart container should support both those scenarios with single core, please confirm:

[tamirms]

@sreuland by obtain txmeta in soroban-rpc via horizon I meant what you suggested earlier:

potentially proxy from horizon api, i.e. http client to streaming(sse) endpoint for transactions to obtain same txmeta

I think that would be easier than querying the horizon postgres db.

[sreuland]

@tamirms , ok, sounds like the design reduces to:

thanks, this also helps in scoping down the requirements for spike on that quickstart container #4590

[2opremio]

Making soroban-rpc depend on Horizon seems reasonable but worries me.

This requires (almost) full Soroban support in Horizon which is a big short-term task to take up.

[tsachiherman]

I think that from scheduling perspective, we should avoid design + implement in the same sprint by the same person as it might not have the proper "separation". That being said, if we can get enough buy-ins, I don't really care.

One way to achieve that is to define the dependent issue very clearly, including the required degree of buy-ins.

[sreuland]

@paulbellamy , is there a corresponding dev ticket for soroban-js-sdk to have client wrapper around this api in tandem during same dev cycle, or is that fine in next iteration, etc?

[edit] found the sorobanj-js-sdk ticket, was out there - stellar/js-soroban-client#2 , guess we just need to decide which/when sprint to pull this into.

[Shaptic]

@sreuland there's also stellar/stellar-cli#61 which is the Rust client side of things. I think JS is lower priority?

[sreuland]

there's also stellar/stellar-cli#61 which is the Rust client side of things. I think JS is lower priority?

it appears that way, stellar/js-soroban-client#2 is in Backlog, maybe during pre-pre-grooming, can discuss if this should be queued up for Next Sprint Proposal. @jcx120 , any insights on expected priority?

[jcx120]

JS will enable us to test the event sub against our reference fundraising DApp in its ultimate deployment environment. My preference is for this to be done first.
@paulbellamy - thoughts on whether its more beneficial (and manageable) to do the CLI first? (perhaps not given we are reconsidering supporting a CLI env?)

[paulbellamy]

I'd say they're about equal priority. Maybe JS should be a bit higher. We can use the CLI to test stuff all works, but like @jcx120 says the example dapp would be the more immediate use case.

There are two use cases here:

• a web client subscribing for updates to do "instant" ui updates when stuff happens (e.g. balance update when transfer completes)
  • This is a nice ux and scalability improvement, but not critical.
• a backend component subscribing to events to ingest them into their own database for indexing.
  • This is the way we want to push devs to building their infra, so long-term is more important. But this also doesn't super depend on the CLI, as I'd expect the dev's custom backend component to either subscribe to soroban-rpc directly, or push the events into a message-queue for processing.

[paulbellamy]

Per #4676, no websockets or push notifications for this. Just the request/response getEvents method, as per: https://docs.google.com/document/d/1TZUDgo_3zPz7TiPMMHVW_mtogjLyPL0plvzGMsxSz6A/edit#heading=h.t03hji406t76