Integration with external/remote Services

[sleipnir]

Discussed in #92

^{Originally posted by sleipnir October 26, 2022}
One of the mainstays of what we do with Eigr, inherited from the Cloudstate days and discussed later on Massa (a discussion was opened but the ideas were never properly recorded here, and which Marcel and I discussed a few times), is the idea of that we should abstract the infrastructure, and by infrastructure I don't just mean the state, from the developer. So that he can think only in terms of Business Domain and not so much in infrastructure code. We know, of course, that this is an almost utopian goal because, no matter what we do, the developer will always want to do more or another way. However, I think it is healthy to give him the tools possible so that he can at least greatly reduce his need to touch the infrastructure.

This is how I present the idea of ​​Actors acting as a proxy for remote services or APIs. So that it continues to use the current way of invoking actors to also access remote services.

We could call these special actors ProxyActors or BindingActors. They in turn could be registered in the traditional way that actors are registered but we would create a new category of Commandos (today we have Commands and TimerCommands) that we would use to map remote services to Spawn actions. It would probably also be necessary to create additional attributes in the ActorSettings of the Actor so that the proxy knows that it is an actor that only acts as a proxy for a service. So that it can perform the necessary request based on the configurations of the given commands.

I think each SDK could have good ways of declaring this new Actor type. In Java they would probably just be an annotated Interface, in Elixir they could be behaviors and etc...

One thing we must take into account is the types of formats we will support. At first I find it easy to support HTTP+Json since protobuf types can be easily transformed into Json types. gRPC would be more complicated as we would have to dynamically generate a grpc client and I think this is quite complicated to do at first. Once the user defines the types in the usual Protobuf-based way we could transcode this to Json types easily I think.

It would not be possible to have actors of this type persistently, but we can certainly investigate that option as well. Maybe it's not smart to mix these concepts, maybe it is, the fact is that I don't know the best approach for this case.

Wdyt?

[sleipnir]

Any thoughts here guys?

[sleipnir]

ping @marcellanz @eliasdarruda

[sleipnir]

Guys, I'm going to follow with a PR. So we can discuss with something concrete

[sleipnir]

@eliasdarruda @marcellanz Perhaps we can start this off by making use of the Worflows API by including this in Forward and SideEffects. Instead of trying to start with a complete entity like the Actor, we can just use the Forward feature and implement remote invocation through it.

[marcellanz]

Great idea, like that a lot. From spawns point of view a forward fits well with an external remote service invocation.

[sleipnir]

Great idea, like that a lot. From spawns point of view a forward fits well with an external remote service invocation.

I think the big difficulty to implement this is the types. Spawn knows very little about users' data types, types for Spawn are nothing more than a byte array and some information coming from Any of type protobuf. Serializing and deserializing without the proxy having the actual compiled types can be a problem.
This is actually something of a boon and a curse for the proxy. The fact that user types are opaque to the proxy is good for security, but it prevents the proxy from doing some things that Massa was possible to do.