404
+ +Page not found
+ + +'+ escapeHtml(title) + '
' + escapeHtml(summary) +'
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Application Configuration
+The following parameters are used by the system. You can define them in either the application.properties or +application.yml file.
+When you use both application.properties and application.yml, the parameters in application.properties will take +precedence.
+| Key | +Value (example) | +Required | +
|---|---|---|
| application.name | +Application name | +Yes | +
| spring.application.name | +Alias for application name | +Yes*1 | +
| info.app.version | +major.minor.build (e.g. 1.0.0) | +Yes | +
| info.app.description | +Something about your application | +Yes | +
| web.component.scan | +your own package path or parent path | +Yes | +
| server.port | +e.g. 8083 | +Yes*1 | +
| rest.server.port | +e.g. 8085 | +Optional | +
| websocket.server.port | +Alias for rest.server.port | +Optional | +
| rest.automation | +true if you want to enable automation | +Optional | +
| yaml.rest.automation | +Config location e.g. classpath:/rest.yaml | +Optional | +
| yaml.event.over.http | +Config location classpath:/event-over-http.yaml | +Optional | +
| yaml.multicast | +Config location classpath:/multicast.yaml | +Optional | +
| yaml.journal | +Config location classpath:/journal.yaml | +Optional | +
| yaml.route.substitution | +Config location | +Optional | +
| yaml.topic.substitution | +Config location | +Optional | +
| yaml.cron | +Config location | +Optional | +
| yaml.flow.automation | +Config location. e.g. classpath:/flows.yaml | +EventScript | +
| static.html.folder | +classpath:/public/ | +Yes | +
| spring.web.resources.static-locations | +(alias for static.html.folder) | +Yes*1 | +
| mime.types | +Map of file extensions to MIME types (application.yml only) |
+Optional | +
| spring.mvc.static-path-pattern | +/** | +Yes*1 | +
| show.env.variables | +comma separated list of variable names | +Optional | +
| show.application.properties | +comma separated list of property names | +Optional | +
| cloud.connector | +kafka, none, etc. | +Optional | +
| cloud.services | +e.g. some.interesting.service | +Optional | +
| snake.case.serialization | +true (recommended) | +Optional | +
| protect.info.endpoints | +true to disable actuators. Default: true | +Optional | +
| trace.http.header | +comma separated list. Default "X-Trace-Id" | +Optional | +
| hsts.feature | +default is true | +Optional* | +
| application.feature.route.substitution | +default is false | +Optional | +
| application.feature.topic.substitution | +default is false | +Optional | +
| kafka.replication.factor | +3 | +Kafka | +
| cloud.client.properties | +e.g. classpath:/kafka.properties | +Connector | +
| user.cloud.client.properties | +e.g. classpath:/second-kafka.properties | +Connector | +
| default.app.group.id | +groupId for the app instance. Default: appGroup |
+Connector | +
| default.monitor.group.id | +groupId for the presence-monitor. Default: monitorGroup |
+Connector | +
| monitor.topic | +topic for the presence-monitor. Default: service.monitor |
+Connector | +
| app.topic.prefix | +Default: multiplex (DO NOT change) | +Connector | +
| app.partitions.per.topic | +Max Kafka partitions per topic. Default: 32 |
+Connector | +
| max.virtual.topics | +Max virtual topics = partitions * topics. Default: 288 |
+Connector | +
| max.closed.user.groups | +Number of closed user groups. Default: 10, range: 3 - 30 |
+Connector | +
| closed.user.group | +Closed user group. Default: 1 | +Connector | +
| transient.data.store | +Default is "/tmp/reactive" | +Optional | +
| running.in.cloud | +Default is false (set to true if containerized) | +Optional | +
| deferred.commit.log | +Default is false (for unit tests only) | +Optional | +
| kernel.thread.pool | +Default 100. Not more than 200. | +Optional | +
* - when using the "rest-spring" library
Base configuration files
+By default, the system assumes the following application configuration files:
+-
+
- application.properties +
- application.yml +
You can change this behavior by adding the app-config-reader.yml in your project's resources folder.
resources:
+ - application.properties
+ - application.yml
+You can tell the system to load application configuration from different set of files. +You can use either PROPERTIES or YAML files. YAML files can use "yml" or "yaml" extension.
+For example, you may use only "application.yml" file without scanning application.properties.
+Partial support of Spring Active Profiles
+When the parameter "spring.profiles.active" is available in application.properties or application.yml, +the AppConfigReader will try to load the additional configuration files.
+For example, if "spring.profiles.active=dev", the system will load "application-dev.properties" +and "application-dev.yml" accordingly.
+When more than one active profile is needed, you can use a comma separated list of profiles in +"spring.profiles.active".
+For Spring Boot compatibility, the filename prefix "application-" is fixed.
+Special handling for PROPERTIES file
+Since application.properties and application.yml can be used together, +the system must enforce keyspace uniqueness because YAML keyspaces are hierarchical.
+For example, if you have x.y and x.y.z, x.y is the parent of x.y.z.
+Therefore, you cannot set a value for the parent key since the parent is a key-value container.
+This hierarchical rule is enforced for PROPERTIES files. +If you have x.y=3 and x.y.z=2 in the same PROPERTIES file, x.y will become a parent of x.y.z and its intended +value of 3 will be lost.
+Optional Service
+The OptionalService annotation may be used with the following class annotations:
-
+
- BeforeApplication +
- MainApplication +
- PreLoad +
- WebSocketService +
When the OptionalService annotation is available, the system will evaluate the annotation value as a +conditional statement where it supports one or more simple condition using a key-value in the application +configuration.
+For examples:
+OptionalService("rest.automation") - the class will be loaded when rest.automation=true
+OptionalService("!rest.automation") - the class will be loaded when rest.automation is false or non-exist
+OptionalService("interesting.key=100") - the system will load the class when "interesting.key" is set to 100 +in application configuration.
+To specify more than one condition, use a comma separated list as the value like this: +OptionalService("web.socket.enabled, rest.automation") - this tells the system to load the class when +either web.socket.enabled or rest.automation is true.
+Static HTML contents
+You can place static HTML files (e.g. the HTML bundle for a UI program) in the "resources/public" folder or +in the local file system using the "static.html.folder" parameter.
+The system supports a bare minimal list of file extensions to MIME types. If your use case requires additional
+MIME type mapping, you may define them in the application.yml configuration file under the mime.types
+section like this:
mime.types:
+ pdf: 'application/pdf'
+ doc: 'application/msword'
+Note that application.properties file cannot be used for the "mime.types" section because it only supports text +key-values.
+HTTP and websocket port assignment
+If rest.automation=true and rest.server.port or server.port are configured, the system will start
+a lightweight non-blocking HTTP server. If rest.server.port is not available, it will fall back to server.port.
If rest.automation=false and you have a websocket server endpoint annotated as WebsocketService, the system
+will start a non-blocking Websocket server with a minimalist HTTP server that provides actuator services.
+If websocket.server.port is not available, it will fall back to rest.server.port or server.port.
If you add Spring Boot dependency, Spring Boot will use server.port to start Tomcat or similar HTTP server.
The built-in lightweight non-blocking HTTP server and Spring Boot can co-exist when you configure
+rest.server.port and server.port to use different ports.
Note that the websocket.server.port parameter is an alias of rest.server.port.
Transient data store
+The system handles back-pressure automatically by overflowing events from memory to a transient data store. +As a cloud native best practice, the folder must be under "/tmp". The default is "/tmp/reactive". +The "running.in.cloud" parameter must be set to false when your apps are running in IDE or in your laptop. +When running in kubernetes, it can be set to true.
+Snake or Camel case serializers
+Serialization and de-serialization of events are performed automatically.
+If there is a genuine need to programmatically perform serialization, you may use the pre-configured serializer +so that the serialization behavior is consistent.
+You can get an instance of the serializer with SimpleMapper.getInstance().getMapper().
The serializer may perform snake case or camel serialization depending on the parameter snake.case.serialization.
If you want to ensure snake case or camel, you can select the serializer like this:
+SimpleObjectMapper snakeCaseMapper = SimpleMapper.getInstance().getSnakeCaseMapper();
+SimpleObjectMapper camelCaseMapper = SimpleMapper.getInstance().getCamelCaseMapper();
+The trace.http.header parameter
+The trace.http.header parameter sets the HTTP header for trace ID. When configured with more than one label,
+the system will retrieve trace ID from the corresponding HTTP header and propagate it through the transaction
+that may be served by multiple services.
If trace ID is presented in an HTTP request, the system will use the same label to set HTTP response traceId header.
+X-Trace-Id: a9a4e1ec-1663-4c52-b4c3-7b34b3e33697
+or
+X-Correlation-Id: a9a4e1ec-1663-4c52-b4c3-7b34b3e33697
+Kafka specific configuration
+If you use the kafka-connector (cloud connector) and kafka-presence (presence monitor), you may want to +externalize kafka.properties like this:
+cloud.client.properties=file:/tmp/config/kafka.properties
+Note that "classpath" refers to embedded config file in the "resources" folder in your source code and "file" +refers to an external config file.
+You want also use the embedded config file as a backup like this:
+cloud.client.properties=file:/tmp/config/kafka.properties, classpath:/kafka.properties
+Distributed trace
+To enable distributed trace logging, please set this in log4j2.xml:
+<logger name="org.platformlambda.core.services.DistributedTrace" level="INFO" />
+Built-in XML serializer
+The platform-core includes built-in serializers for JSON and XML in the AsyncHttpClient and +Spring RestController. The XML serializer is designed for simple use cases. If you need to handle more +complex XML data structure, you can disable the built-in XML serializer by adding the following HTTP +request header.
+X-Raw-Xml=true
+Custom content types
+If you use custom content types in your application, you may add the following section in the application.yml +configuration file:
+custom.content.types:
+ - 'application/vnd.my.org-v2.0+json -> application/json'
+In the "custom.content.types" section, you can configure a list of content-type mappings. +The left-hand-side is the custom content-type and the right-hand-side is a standard content-type.
+The content-type mapping tells the system to treat the custom content type as if it is the standard content +type.
+In the above example, the HTTP payload with the custom content type is treated as a regular JSON content.
+If you want to put the custom content types in a separate configuration file, please put them in a file named +"custom-content-type.yml" under the "resources" folder.
+| Chapter-9 | +Home | +Appendix-II | +
|---|---|---|
| API Overview | +Table of Contents | +Reserved names and headers | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/APPENDIX-II/index.html b/docs/guides/APPENDIX-II/index.html
new file mode 100644
index 00000000..55f33ffc
--- /dev/null
+++ b/docs/guides/APPENDIX-II/index.html
@@ -0,0 +1,477 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Reserved names
+The system reserves some route names and headers for routing purpose.
+System route names
+The Mercury foundation code is written using the same core API and each function has a route name.
+The following route names are reserved. Please DO NOT overload them in your application functions +to avoid breaking the system unintentionally.
+| Route | +Purpose | +Modules | +
|---|---|---|
| actuator.services | +Actuator endpoint services | +platform-core | +
| elastic.queue.cleanup | +Elastic event buffer clean up task | +platform-core | +
| distributed.tracing | +Distributed tracing logger | +platform-core | +
| system.ws.server.cleanup | +Websocket server cleanup service | +platform-core | +
| http.auth.handler | +REST automation authentication router | +platform-core | +
| event.api.service | +Event API service | +platform-core | +
| event.script.manager | +Instantiate new event flow instance | +event-script | +
| task.executor | +Perform event choreography | +event-script | +
| http.flow.adapter | +Built-in flow adapter | +event-script | +
| no.op | +no-operation placeholder function | +event-script | +
| system.service.registry | +Distributed routing registry | +Connector | +
| system.service.query | +Distributed routing query | +Connector | +
| cloud.connector.health | +Cloud connector health service | +Connector | +
| cloud.manager | +Cloud manager service | +Connector | +
| presence.service | +Presence signal service | +Connector | +
| presence.housekeeper | +Presence keep-alive service | +Connector | +
| cloud.connector | +Cloud event emitter | +Connector | +
| init.multiplex.* | +reserved for event stream startup | +Connector | +
| completion.multiplex.* | +reserved for event stream clean up | +Connector | +
| async.http.request | +HTTP request event handler | +REST automation | +
| async.http.response | +HTTP response event handler | +REST automation | +
| cron.scheduler | +Cron job scheduler | +Simple Scheduler | +
| init.service.monitor.* | +reserved for event stream startup | +Service monitor | +
| completion.service.monitor.* | +reserved for event stream clean up | +Service monitor | +
Optional user defined functions
+The following optional route names will be detected by the system for additional user defined features.
+| Route | +Purpose | +
|---|---|
| additional.info | +User application function to return information about your application status |
+
| distributed.trace.forwarder | +Custom function to forward performance metrics to a telemetry system |
+
| transaction.journal.recorder | +Custom function to record transaction request-response payloads into an audit DB |
+
The additional.info function, if implemented, will be invoked from the "/info" endpoint and its response
+will be merged into the "/info" response.
For distributed.trace.forwarder and transaction.journal.recorder, please refer to Chapter-5
+for details.
No-op function
+The "no.op" function is used as a placeholder for building skeleton or simple decision function for +an event flow use case.
+Reserved event header names
+The following event headers are injected by the system as READ only metadata. They are available from the +input "headers". However, they are not part of the EventEnvelope.
+| Header | +Purpose | +
|---|---|
| my_route | +route name of your function | +
| my_trace_id | +trace ID, if any, for the incoming event | +
| my_trace_path | +trace path, if any, for the incoming event | +
You can create a trackable PostOffice using the "headers" and the "instance" parameters in the input arguments +of your function. The FastRPC instance requires only the "headers" parameters.
+// Java
+PostOffice po = new PostOffice(headers, instance);
+
+// Kotlin
+val fastRPC = FastRPC(headers);
+Reserved HTTP header names
+| Header | +Purpose | +
|---|---|
| X-Stream-Id | +Temporal route name for streaming content | +
| X-TTL | +Time to live in milliseconds for a streaming content | +
| X-Small-Payload-As-Bytes | +This header, if set to true, tells system to render stream content as bytes | +
| X-Event-Api | +The system uses this header to indicate that the request is sent over HTTP | +
| X-Async | +This header, if set to true, indicates it is a drop-n-forget request | +
| X-Trace-Id | +This allows the system to propagate trace ID | +
| X-Correlation-Id | +Alternative to X-Trace-Id | +
| X-Content-Length | +If present, it is the expected length of a streaming content | +
| X-Raw-Xml | +This header, if set to true, tells to system to skip XML rendering | +
| X-Flow-Id | +This tells the event manager to select a flow configuration by ID | +
| X-App-Instance | +This header is used by some protected actuator REST endpoints | +
To support traceId that is stored in X-Correlation-Id HTTP header, set this in application.properties.
+# list of supported traceId headers where the first one is the default label
+trace.http.header=X-Correlation-Id, X-Trace-Id
+| Appendix-I | +Home | +Appendix-III | +
|---|---|---|
| Application Configuration | +Table of Contents | +Actuators, HTTP client and More | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/APPENDIX-III/index.html b/docs/guides/APPENDIX-III/index.html
new file mode 100644
index 00000000..2f15ffe1
--- /dev/null
+++ b/docs/guides/APPENDIX-III/index.html
@@ -0,0 +1,533 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Actuators, HTTP client and More
+Actuator endpoints
+The following admin endpoints are available.
+GET /info
+GET /info/routes
+GET /info/lib
+GET /env
+GET /health
+GET /livenessprobe
+POST /shutdown
+| Endpoint | +Purpose | +
|---|---|
| /info | +Describe the application | +
| /info/routes | +Show public routing table | +
| /info/lib | +List libraries packed with this executable | +
| /env | +List all private and public function route names and selected environment variables | +
| /health | +Application health check endpoint | +
| /livenessprobe | +Check if application is running normally | +
| /shutdown | +Operator may use this endpoint to do a POST command to stop the application | +
For the shutdown endpoint, you must provide an X-App-Instance HTTP header where the value is the "origin ID"
+of the application. You can get the value from the "/info" endpoint.
Custom health services
+You can extend the "/health" endpoint by implementing and registering lambda functions to be added to the +"health check" dependencies.
+mandatory.health.dependencies=cloud.connector.health, demo.health
+optional.health.dependencies=other.service.health
+Your custom health service must respond to the following requests:
+-
+
- Info request (type=info) - it should return a map that includes service name and href (protocol, hostname and port) +
- Health check (type=health) - it should return a text string or a Map of the health check. e.g. read/write test result. + If health check fails, you can throw AppException with status code and error message. +
A sample health service is available in the DemoHealth class of the composable-example project as follows:
@PreLoad(route="demo.health", instances=5)
+public class DemoHealth implements LambdaFunction {
+
+ private static final String TYPE = "type";
+ private static final String INFO = "info";
+ private static final String HEALTH = "health";
+
+ @Override
+ public Object handleEvent(Map<String, String> headers, Object input, int instance) {
+ /*
+ * The interface contract for a health check service includes both INFO and HEALTH responses.
+ * It must return a Map.
+ */
+ if (INFO.equals(headers.get(TYPE))) {
+ Map<String, Object> about = new HashMap<>();
+ about.put("service", "demo.service");
+ about.put("href", "http://127.0.0.1");
+ return about;
+ }
+ if (HEALTH.equals(headers.get(TYPE))) {
+ /*
+ * This is a place-holder for checking a downstream service.
+ *
+ * Please implement your own logic to test if a downstream service is running fine.
+ * If running, just return health status as a String or a Map.
+ *
+ * Otherwise,
+ * throw new AppException(status, message)
+ */
+ return Map.of("demo", "I am running fine");
+ }
+ throw new IllegalArgumentException("type must be info or health");
+ }
+}
+AsyncHttpClient service
+The "async.http.request" function can be used as a non-blocking HTTP client.
+To make an HTTP request to an external REST endpoint, you can create an HTTP request object using the
+AsyncHttpRequest class and make an async RPC call to the "async.http.request" function like this:
PostOffice po = new PostOffice(headers, instance);
+AsyncHttpRequest req = new AsyncHttpRequest();
+req.setMethod("GET");
+req.setHeader("accept", "application/json");
+req.setUrl("/api/hello/world?hello world=abc");
+req.setQueryParameter("x1", "y");
+List<String> list = new ArrayList<>();
+list.add("a");
+list.add("b");
+req.setQueryParameter("x2", list);
+req.setTargetHost("http://127.0.0.1:8083");
+EventEnvelope request = new EventEnvelope().setTo("async.http.request").setBody(req);
+EventEnvelope res = po.request(request, 5000);
+// the result is in res.getBody()
+By default, your user function is running in a virtual thread. +While the RPC call looks like synchronous, the po.request API will run in non-blocking mode in the same fashion +as the "async/await" pattern.
+For reactive programming, you can use the "asyncRequest" API like this:
+PostOffice po = new PostOffice(headers, instance);
+AsyncHttpRequest req = new AsyncHttpRequest();
+req.setMethod("GET");
+req.setHeader("accept", "application/json");
+req.setUrl("/api/hello/world?hello world=abc");
+req.setQueryParameter("x1", "y");
+List<String> list = new ArrayList<>();
+list.add("a");
+list.add("b");
+req.setQueryParameter("x2", list);
+req.setTargetHost("http://127.0.0.1:8083");
+EventEnvelope request = new EventEnvelope().setTo("async.http.request").setBody(req);
+Future<EventEnvelope> res = po.asyncRequest(request, 5000);
+res.onSuccess(response -> {
+ // do something with the result
+});
+If you prefer writing in Kotlin, you can create a suspend function using KotlinLambdaFunction, +the same logic may look like this:
+val fastRPC = FastRPC(headers)
+val req = AsyncHttpRequest()
+req.setMethod("GET")
+req.setHeader("accept", "application/json")
+req.setUrl("/api/hello/world?hello world=abc")
+req.setQueryParameter("x1", "y")
+val list: MutableList<String> = ArrayList()
+list.add("a")
+list.add("b")
+req.setQueryParameter("x2", list)
+req.setTargetHost("http://127.0.0.1:8083")
+val request = EventEnvelope().setTo("async.http.request").setBody(req)
+val response = fastRPC.awaitRequest(request, 5000)
+// do something with the result
+Send HTTP request body for HTTP PUT, POST and PATCH methods
+For most cases, you can just set a HashMap into the request body and specify content-type as JSON or XML. +The system will perform serialization properly.
+Example code may look like this:
+AsyncHttpRequest req = new AsyncHttpRequest();
+req.setMethod("POST");
+req.setHeader("accept", "application/json");
+req.setHeader("content-type", "application/json");
+req.setUrl("/api/book");
+req.setTargetHost("https://service_provider_host");
+req.setBody(mapOfKeyValues);
+// where keyValues is a HashMap
+Send HTTP request body as a stream
+For larger payload, you may use the streaming method. See sample code below:
+int len;
+byte[] buffer = new byte[4096];
+FileInputStream in = new FileInputStream(myFile);
+EventPublisher publisher = new EventPublisher(timeoutInMIlls);
+while ((len = in.read(buffer, 0, buffer.length)) != -1) {
+ publisher.publish(buffer, 0, len);
+}
+// closing the output stream would send a EOF signal to the stream
+publisher.publishCompletion();
+// tell the HTTP client to read the input stream by setting the streamId in the AsyncHttpRequest object
+req.setStreamRoute(publisher.getStreamId());
+Read HTTP response body stream
+If content length is not given, the response body would arrive as a stream.
+Your application should check if the HTTP response header "stream" exists. Its value is the input "streamId".
+You can process the input stream using the FluxConsumer class like this:
+String streamId = headers.get("stream");
+long ttl = 10000; // anticipated time in milliseconds to stream the content
+FluxConsumer<Map<String, Object>> fc = new FluxConsumer<>(streamId, ttl);
+fc.consume(
+ data -> {
+ // handle incoming message
+ },
+ e -> {
+ // handle exception where e is a Throwable
+ },
+ () -> {
+ // handle stream completion
+ }
+);
+By default, a user function is executed in a virtual thread which effectively is an "async" function and +the PostOffice "request" API operates in the non-blocking "await" mode.
+If you prefers writing in Kotlin, it may look like this:
+val po = PostOffice(headers, instance)
+val fastRPC = FastRPC(headers)
+
+val req = EventEnvelope().setTo(streamId).setHeader("type", "read")
+while (true) {
+ val event = fastRPC.awaitRequest(req, 5000)
+ if (event.status == 408) {
+ // handle input stream timeout
+ break
+ }
+ if ("eof" == event.headers["type"]) {
+ po.send(streamId, Kv("type", "close"))
+ break
+ }
+ if ("data" == event.headers["type"]) {
+ val block = event.body
+ if (block is ByteArray) {
+ // handle the data block from the input stream
+ }
+ }
+}
+Rendering a small payload of streaming content
+If the streaming HTTP response is certain to be a small payload (i.e. Kilobytes), you can optimize +the rendering by adding the HTTP request header (X-Small-Payload-As-Bytes=true) in the AsyncHttpRequest object.
+AsyncHttpRequest req = new AsyncHttpRequest();
+req.setMethod("GET");
+req.setUrl("/api/some/binary/content");
+req.setTargetHost("https://service_provider_host");
+req.setHeader("X-Small-Payload-As-Bytes", "true");
+Note that the AsyncHttpClient will insert a custom HTTP response header "X-Content-Length" to show the size +of the payload.
+++IMPORTANT: This optimization does not validate the size of the streaming content. Therefore, it is possible for + the streaming content to trigger an "out of memory" exception. You must make sure the streaming content + is small enough before using the "X-Small-Payload-As-Bytes" HTTP request header.
+
Content length for HTTP request
+IMPORTANT: Do not set the "content-length" HTTP header because the system will automatically compute the +correct content-length for small payload. For large payload, it will use the chunking method.
+Starting a flow programmatically
+To start an "event" flow from a unit test, you may use the helper class "FlowExecutor" under the "Event Script" module.
+Examples of some APIs are as follows:
+// launch a flow asychronously
+public void launch(String originator, String flowId, Map<String, Object> dataset,
+ String correlationId) throws IOException;
+// launch a flow asychronously with tracing
+public void launch(String originator, String traceId, String tracePath, String flowId,
+ Map<String, Object> dataset, String correlationId) throws IOException
+// launch a flow asychronously and tracing
+public void launch(PostOffice po, String flowId, Map<String, Object> dataset,
+ String correlationId) throws IOException;
+// launch a flow with callback and tracing
+public void launch(PostOffice po, String flowId, Map<String, Object> dataset,
+ String replyTo, String correlationId) throws IOException;
+// launch a flow and expect a future response
+public Future<EventEnvelope> request(PostOffice po, String flowId, Map<String, Object> dataset,
+ String correlationId, long timeout) throws IOException;
+The following unit test emulates a HTTP request to the flow named "header-test".
+@Test
+public void internalFlowTest() throws IOException, ExecutionException, InterruptedException {
+ final long TIMEOUT = 8000;
+ String traceId = Utility.getInstance().getUuid();
+ String cid = Utility.getInstance().getUuid();
+ PostOffice po = new PostOffice("unit.test", traceId, "INTERNAL /flow/test");
+ String flowId = "header-test";
+ Map<String, Object> headers = new HashMap<>();
+ Map<String, Object> dataset = new HashMap<>();
+ dataset.put("header", headers);
+ dataset.put("body", Map.of("hello", "world"));
+ headers.put("user-agent", "internal-flow");
+ headers.put("accept", "application/json");
+ headers.put("x-flow-id", flowId);
+ FlowExecutor flowExecutor = FlowExecutor.getInstance();
+ EventEnvelope result = flowExecutor.request(po, flowId, dataset, cid, TIMEOUT).get();
+ assertInstanceOf(Map.class, result.getBody());
+ Map<String, Object> body = (Map<String, Object>) result.getBody();
+ // verify that input headers are mapped to the function's input body
+ assertEquals("header-test", body.get("x-flow-id"));
+ assertEquals("internal-flow", body.get("user-agent"));
+ assertEquals("application/json", body.get("accept"));
+}
+The dataset must contain at least the "body" key-value so that input data mapping is possible in a flow.
+For the built-in HTTP flow adapter, the dataset would contain the following:
+// convert HTTP context to flow "input" dataset
+Map<String, Object> dataset = new HashMap<>();
+dataset.put("header", request.getHeaders());
+dataset.put("body", request.getBody());
+dataset.put("cookie", request.getCookies());
+dataset.put("path_parameter", request.getPathParameters());
+dataset.put("method", request.getMethod());
+dataset.put("uri", request.getUrl());
+dataset.put("query", request.getQueryParameters());
+dataset.put("stream", request.getStreamRoute());
+dataset.put("ip", request.getRemoteIp());
+dataset.put("filename", request.getFileName());
+dataset.put("session", request.getSessionInfo());
+If you write your own Kafka flow adapter, the dataset should contain headers and body mapped with a Kafka event.
+For other flow adapters, you may use different set of key-values.
+| Appendix-II | +Home | +
|---|---|
| Reserved names and headers | +Table of Contents | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-1/index.html b/docs/guides/CHAPTER-1/index.html
new file mode 100644
index 00000000..1db835cd
--- /dev/null
+++ b/docs/guides/CHAPTER-1/index.html
@@ -0,0 +1,556 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Introduction
+Mercury Composable is a software development toolkit for writing composable applications.
+At the platform level, composable architecture refers to loosely coupled platform services, utilities, and +business applications. With modular design, you can assemble platform components and applications to create +new use cases or to adjust for ever-changing business environment and requirements. Domain driven design (DDD), +Command Query Responsibility Segregation (CQRS) and Microservices patterns are the popular tools that architects +use to build composable architecture. You may deploy application in container, serverless or other means.
+At the application level, a composable application means that an application is assembled from modular software +components or functions that are self-contained and pluggable. You can mix-n-match functions to form new applications. +You can retire outdated functions without adverse side effect to a production system. Multiple versions of a function +can exist, and you can decide how to route user requests to different versions of a function. Applications would be +easier to design, develop, maintain, deploy, and scale.
+Composable application architecture
+++Figure 1 - Composable application architecture
+
As shown in Figure 1, a composable application contains the following:
+-
+
- Flow adapters: Each flow adapter listens to requests for onwards delivery to an event manager. +
- Event Manager: it sends events to a set of user functions for them to work together as an application. +
- User functions: these are self-contained functions with clear input and output that are immutable. +
HTTP flow adapter
+A non-blocking HTTP flow adapter is built-in. For other external interface types, you can implement your own +flow adapters. e.g. Adapters for MQ, Kafka, Serverless, File based staging area, etc.
+The standard HTTP flow adapter leverages the underlying Mercury REST automation system to serve user facing REST +API endpoints. For example, a hypothetical "get profile" endpoint is created like this in the "rest.yaml" +configuration file:
+ - service: "http.flow.adapter"
+ methods: ['GET']
+ url: "/api/profile/{profile_id}"
+ flow: 'get-profile'
+ timeout: 10s
+ cors: cors_1
+ headers: header_1
+ tracing: true
+In this REST configuration entry, the system creates a REST API endpoint for "GET /api/profile/{profile_id}". +When a request arrives at this endpoint, the HTTP request will be converted to an incoming event by the flow adapter +that routes the event to the "event manager" to execute a new instance of the "get-profile" flow.
+Flow configuration example
+The event manager is driven by configuration instead of code. A hypothetical "get profile" flow is defined in +a YAML file like this:
+flow:
+ id: 'get-profile'
+ description: 'Get a user profile using profile ID'
+ ttl: 10s
+ exception: 'v1.hello.exception'
+
+first.task: 'v1.get.profile'
+
+tasks:
+ - input:
+ - 'input.path_parameter.profile_id -> header.profile_id'
+ process: 'v1.get.profile'
+ output:
+ - 'result -> model.profile'
+ description: 'Retrieve user profile from database using profile_id'
+ execution: sequential
+ next:
+ - 'v1.decrypt.fields'
+
+ - input:
+ - 'model.profile -> dataset'
+ - 'text(telephone, address) -> protected_fields'
+ process: 'v1.decrypt.fields'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+ description: 'Decrypt fields'
+ execution: end
+
+ - input:
+ - 'error.code -> status'
+ - 'error.message -> message'
+ - 'error.stack -> stack'
+ process: 'v1.hello.exception'
+ output:
+ - 'result.status -> output.status'
+ - 'result -> output.body'
+ description: 'Just a demo exception handler'
+ execution: end
+Note that the flow configuration is referring user functions by their "route" names. It is because all user functions +are self-contained with clearly defined input and output and the event manager would set their inputs and collect their +outputs accordingly. Note that you can map selected key-values or the whole event as a business object and this +decoupling promotes highly reusable user functional software.
+The event manager will create a "state machine" to manage each transaction flow because all user functions are +stateless. The "state machine" is referenced using the namespace "model".
+Assigning a route name to a user function
+You can assign a route name to a Java class using the PreLoad annotation like this:
@PreLoad(route="v1.get.profile", instances=100)
+public class GetProfile implements TypedLambdaFunction<Map<String, Object>, Profile> {
+
+ @Override
+ public Profile handleEvent(Map<String, String> headers, Map<String, Object> input, int instance) {
+ // your business logic here
+ return result;
+ }
+}
+Inside the "handleEvent" method, you can write regular Java code using your preferred coding style and +framework. You can define input/output as Map or PoJo.
+Building the Mercury libraries from source
+Mercury Composable leverages the best of Java 21 virtual threading technology. Therefore, you would need to +install Java JDK version 21 or higher. You also need maven version 3.9.7 or higher to build the libraries.
+Assuming you clone the Mercury repository into the "sandbox" directory, you may build the libraries like this.
+cd sandbox/mercury-composable
+mvn clean install
+The compiled libraries will be saved to your local ".m2" maven repository. For convenience, you may also publish +the Mercury libraries into your enterprise artifactory.
+We use "maven" build scripts. If your organization uses other build tools such as gradle, please convert them +accordingly.
+Things to avoid with Java 21
+By default, user functions are executed using Java 21 virtual threading technology. However, for performance reason, +there are two things that you MUST avoid:
+-
+
- Synchronized keyword: This will block the event loop in the Java VM, meaning that your whole application is + blocked when the synchronized block executes. +
- ThreadLocal: Java 21 virtual thread is designed to be very light weight. When you use ThreadLocal variables, + the "virtual thread" becomes heavy weighted and the Garbage Collector may have difficulties catching up. +
Since Mercury provides thread management abstraction, there is no need to use the Synchronized keyword and +ThreadLocal variables. The built-in "state machine" is a better place to keep your runtime variables for each +transaction.
+Interestingly, the "Thread" and "Future" APIs are safe to use in a virtual thread.
+If you are putting legacy code inside a new user function and the legacy code runs in blocking mode, you can +annotate the user function with the "KernelThreadRunner" class. This tells the system to turn on compatibility +mode to support the blocking code. The kernel thread would isolate the blocking code from the rest of the +application. However, kernel threads are limited resources. While virtual threads can support tens of thousands of +cooperative concurrent execution, kernel threads are limited to 250, depending on the number of CPU cores that the +target machine has.
+Composable application example
+Let's take a test drive of a composable application example in the "examples/composable-example" subproject.
+You can use your favorite IDE to run the example or execute it from a terminal using command line.
+To run it from the command line, you may do this:
+cd sandbox/mercury-composable/examples/composable-example
+java -jar target/composable-example-4.0.9.jar
+If you run the application from the IDE, you may execute the "main" method in the MainApp class under the
+"com.accenture.demo.start" package folder.
The first step in designing a composable application is to draw an event flow diagram. This is similar to +a data flow diagram where the arrows are labeled with the event objects. Note that event flow diagram is +not a flow chart and thus decision box is not required. If a user function (also known as a "task") contains +decision logic, you can draw two or more output from the task to connect to the next set of functions. +For example, label the arrows as true, false or a number starting from 1.
+The composable-example application is a hypothetical "profile management system" where you can create a profile, +browse or delete it.
+++Figure 2 - Event flow diagram
+
As shown in Figure 2, there are three event flows. One for "get profile", one for "delete profile" and the other +one for "create profile".
+The REST endpoints for the three use cases are shown in the "rest.yaml" configuration file under the "main/resources" +in the example subproject.
+You also find the following configuration parameters in "application.properties":
+rest.server.port=8100
+rest.automation=true
+yaml.rest.automation=classpath:/rest.yaml
+yaml.flow.automation=classpath:/flows.yaml
+The flow configuration files are shown in the "main/resources/flows" folder where you will find the flow configuration +files for the three event flows, namely get-profile.yml, delete-profile.yml and create-profile.yml.
+Starting the application
+When the application is started, you will see application log like this:
+CompileFlows:142 - Loaded create-profile
+CompileFlows:142 - Loaded delete-profile
+CompileFlows:142 - Loaded get-profile
+CompileFlows:144 - Event scripts deployed: 3
+...
+ServiceQueue:91 - PRIVATE v1.get.profile with 100 instances started as virtual threads
+...
+RoutingEntry:582 - GET /api/profile/{profile_id} -> [http.flow.adapter], timeout=10s, tracing=true, flow=get-profile
+...
+AppStarter:378 - Modules loaded in 663 ms
+AppStarter:365 - Reactive HTTP server running on port-8100
+Note that the above log is trimmed for presentation purpose.
+It shows that the 3 flow configuration files are compiled as objects for performance reason. The user functions are +loaded into the event system and the REST endpoints are rendered from the "rest.yaml" file.
+Testing the application
+You can create a test user profile with this python code. Alternatively, you can also use PostMan or other means +to do this.
+>>> import requests, json
+>>> d = { 'id': 12345, 'name': 'Hello World', 'address': '100 World Blvd', 'telephone': '123-456-7890' }
+>>> h = { 'content-type': 'application/json', 'accept': 'application/json' }
+>>> r = requests.post('http://127.0.0.1:8100/api/profile', data=json.dumps(d), headers=h)
+>>> print(r.status_code)
+201
+>>> print(r.text)
+{
+ "profile": {
+ "address": "***",
+ "name": "Hello World",
+ "telephone": "***",
+ "id": 12345
+ },
+ "type": "CREATE",
+ "secure": [
+ "address",
+ "telephone"
+ ]
+}
+To verify that the user profile has been created, you can point your browser to
+http://127.0.0.1:8100/api/profile/12345
+Your browser will return the following:
+{
+ "address": "100 World Blvd",
+ "name": "Hello World",
+ "telephone": "123-456-7890",
+ "id": 12345
+}
+You have successfully tested the two REST endpoints. Tracing information in the application log may look like this:
+DistributedTrace:76 - trace={path=POST /api/profile, service=http.flow.adapter, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:23.524Z,
+ exec_time=0.284, from=http.request, id=f6a6ae62340e43afb0a6f30445166e08}
+DistributedTrace:76 - trace={path=POST /api/profile, service=event.script.manager, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:23.525Z,
+ exec_time=0.57, from=http.flow.adapter, id=f6a6ae62340e43afb0a6f30445166e08}
+DistributedTrace:76 - trace={path=POST /api/profile, service=v1.create.profile, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:23.526Z,
+ exec_time=0.342, from=task.executor, id=f6a6ae62340e43afb0a6f30445166e08}
+DistributedTrace:76 - trace={path=POST /api/profile, service=async.http.response, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:23.528Z,
+ exec_time=0.294, from=task.executor, id=f6a6ae62340e43afb0a6f30445166e08}
+DistributedTrace:76 - trace={path=POST /api/profile, service=v1.encrypt.fields, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:23.528Z,
+ exec_time=3.64, from=task.executor, id=f6a6ae62340e43afb0a6f30445166e08}
+SaveProfile:52 - Profile 12345 saved
+TaskExecutor:186 - Flow create-profile (f6a6ae62340e43afb0a6f30445166e08) completed in 11 ms
+DistributedTrace:76 - trace={path=POST /api/profile, service=v1.save.profile, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:23.533Z,
+ exec_time=2.006, from=task.executor, id=f6a6ae62340e43afb0a6f30445166e08}
+
+DistributedTrace:76 - trace={path=GET /api/profile/12345, service=http.flow.adapter, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:52.089Z,
+ exec_time=0.152, from=http.request, id=1a29105044e94cc3ac68aee002f6f429}
+DistributedTrace:76 - trace={path=GET /api/profile/12345, service=event.script.manager, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:52.090Z,
+ exec_time=0.291, from=http.flow.adapter, id=1a29105044e94cc3ac68aee002f6f429}
+DistributedTrace:76 - trace={path=GET /api/profile/12345, service=v1.get.profile, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:52.091Z,
+ exec_time=1.137, from=task.executor, id=1a29105044e94cc3ac68aee002f6f429}
+DistributedTrace:76 - trace={path=GET /api/profile/12345, service=v1.decrypt.fields, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:52.093Z,
+ exec_time=1.22, from=task.executor, id=1a29105044e94cc3ac68aee002f6f429}
+TaskExecutor:186 - Flow get-profile (1a29105044e94cc3ac68aee002f6f429) completed in 4 ms
+DistributedTrace:76 - trace={path=GET /api/profile/12345, service=async.http.response, success=true,
+ origin=202406249aea0a481d46401d8379c8896a6698a2, start=2024-06-24T22:41:52.095Z,
+ exec_time=0.214, from=task.executor, id=1a29105044e94cc3ac68aee002f6f429}
+Main module
+Every application has an entry point. The MainApp in the example app contains the entry point like this:
+@MainApplication
+public class MainApp implements EntryPoint {
+ public static void main(String[] args) {
+ AutoStart.main(args);
+ }
+ @Override
+ public void start(String[] args) {
+ // your startup logic here
+ log.info("Started");
+ }
+}
+Since your application is event driven, the main application does not need any additional code in the above +example. However, this is a good place to put application initialization code if any.
+There is also a "BeforeApplication" annotation if you want to run some start up code before the event system +is started.
+Dependency management
+As a best practice, your user functions should not have any dependencies with other user functions.
+However, within a single user function, you may use your preferred framework or libraries.
+For maintainability, we do recommend to reduce library dependencies as much as you can. For example, you want to
+push JDBC or JPA dependency to a small number of user functions (for CRUD operation) so that the rest of the
+user functions do not need any DB dependencies.
Component scan
+Please update the following in the application.properties (or application.yml) to include packages of your own functions:
+web.component.scan=your.package.name
+++You should replace "your.package.name" with the real package name(s) that you use in your application. + Usually this is your organization software ID or "namespace". + "web.component.scan" is a comma separated list of package names.
+
Deploy your application
+Composable design can be used to create microservices. You can put related functions in a bounded context with +database persistence.
+Each composable application can be compiled and built into a single "executable" for deployment using
+mvn clean package.
The executable JAR is in the target folder.
+Composable application is by definition cloud native. It is designed to be deployable using Kubernetes or serverless.
+A sample Dockerfile for your executable JAR may look like this:
+FROM eclipse-temurin:21.0.1_12-jdk
+EXPOSE 8083
+WORKDIR /app
+COPY target/your-app-name.jar .
+ENTRYPOINT ["java","-jar","your-app-name.jar"]
+The above Dockerfile will fetch Openjdk 21 packaged in "Ubuntu 22.04 LTS".
+Event choreography by configuration
+The best practice for composable design is event choreography by configuration (Event Script) discussed above.
+We will examine the Event Script syntax in Chapter 4.
Generally, you do not need to use Mercury core APIs in your user functions.
+For composable applications that use Event Script, Mercury core APIs (Platform, PostOffice and FastRPC) are only +required for writing unit tests, "custom flow adapters", "legacy functional wrappers" or "external gateways".
+Orchestration by code
+Orchestration by code is strongly discouraged because it would result in tightly coupled code.
+For example, just an "Import" statement of another function would create tight coupling of two pieces of code, +even when using reactive or event-driven programming styles.
+However, if there is a use case that you prefer to write orchestration logic by code, you may use the Mercury core APIs +to do event-driven programming. API overview will be covered in Chapter 9.
+| Home | +Chapter-2 | +
|---|---|
| Table of Contents | +Function Execution Strategy | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-2/index.html b/docs/guides/CHAPTER-2/index.html
new file mode 100644
index 00000000..bfd5b1f0
--- /dev/null
+++ b/docs/guides/CHAPTER-2/index.html
@@ -0,0 +1,592 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Function Execution Strategies
+Define a function
+In a composable application, each function is self-contained with zero dependencies with other user functions.
+Only flow adapter, data adapter, notification function or gateway has a single external dependency such as +a network event system, a database or an external REST resource.
+A function is a class that implements the LambdaFunction, TypedLambdaFunction or KotlinLambdaFunction interface. +Within each function boundary, it may have private methods that are fully contained within the class.
+As discussed in Chapter-1, a function may look like this:
+@PreLoad(route = "my.first.function", instances = 10)
+public class MyFirstFunction implements TypedLambdaFunction<MyPoJo, AnotherPoJo> {
+
+ @Override
+ public AnotherPojo handleEvent(Map<String, String> headers, MyPoJo input, int instance) {
+ // your business logic here
+ return result;
+ }
+}
+A function is an event listener with the "handleEvent" method. The data structures of input and output are defined +by API interface contract during application design phase.
+In the above example, the input is MyPoJo and the output is AnotherPoJo.
+For event choreography, PoJos are treated as key-value Maps so that you can use the dot-bracket convention +to map subset of a PoJo from one function to another if needed.
+When the input is used for a PoJo, you may also pass parameters to the user function as headers. We will discuss +this in Chapter 3 "Event Script syntax".
+Non-blocking design
+While you can apply sequential, object oriented or reactive programming styles in your functions, you should pay +attention to making your function non-blocking and fast.
+In a virtual thread, if you use Java Future, the ".get()" method is synchronous but it is non-blocking behind the +curtain. This is like using the "await" keyword in other programming language.
+Virtual thread execution promotes high performance and high concurrency. However, it would be suboptimal +if you mix blocking code in a user function. It will block the whole event loop, resulting in substantial +degradation of application performance. We therefore recommend your user function to be implemented in non-blocking +or reactive styles.
+When you are using a reactive libaries in your function, your function can return a "Mono" or "Flux" reactive +response object using the Project-Reactor Core library. This feature is supported in Java and Kotlin.
+For simplicity, we support only the Mono and Flux reactive response objects. If you use other types of reactive APIs, +please convert them into a Mono or Flux in the return value.
+User function that returns a Mono object
+For Mono return value, a reactive user function may look like this:
+@PreLoad(route = "v1.reactive.mono.function")
+public class MonoUserFunction implements TypedLambdaFunction<Map<String, Object>, Mono<Map<String, Object>>> {
+ private static final Logger log = LoggerFactory.getLogger(MonoUserFunction.class);
+
+ private static final String EXCEPTION = "exception";
+
+ @Override
+ public Mono<Map<String, Object>> handleEvent(Map<String, String> headers, Map<String, Object> input, int instance) {
+ log.info("GOT {} {}", headers, input);
+ return Mono.create(callback -> {
+ if (headers.containsKey(EXCEPTION)) {
+ callback.error(new AppException(400, headers.get(EXCEPTION)));
+ } else {
+ callback.success(input);
+ }
+ });
+ }
+}
+When you use reactive API in your function to connect to external resources such as a database, please ensure that the +reactive API is non-blocking. For example, when subscribing to a Mono publisher, you may need to add a "Scheduler" +before your subscribe statement. It may look something like this:
+// obtain a virtual thread executor from the platform and apply it with the Mono's scheduler
+mono.subscribeOn(Schedulers.fromExecutor(Platform.getInstance().getVirtualThreadExecutor()))
+ .subscribe(responseConsumer, errorConsumer);
+Without the scheduler, the subscribe statement will be blocked. Your next statement will not be reachable until +the mono has completed with data or exception.
+User function that returns a Flux object
+For Flux return value, it may look like this:
+@PreLoad(route = "v1.reactive.flux.function")
+public class FluxUserFunction implements TypedLambdaFunction<Map<String, Object>, Flux<Map<String, Object>>> {
+ private static final Logger log = LoggerFactory.getLogger(FluxUserFunction.class);
+
+ private static final String EXCEPTION = "exception";
+ @Override
+ public Flux<Map<String, Object>> handleEvent(Map<String, String> headers, Map<String, Object> input, int instance) {
+ log.info("GOT {} {}", headers, input);
+ return Flux.create(emitter -> {
+ if (headers.containsKey(EXCEPTION)) {
+ emitter.error(new AppException(400, headers.get(EXCEPTION)));
+ } else {
+ // just generate two messages
+ emitter.next(Map.of("first", "message"));
+ emitter.next(input);
+ emitter.complete();
+ }
+ });
+ }
+}
+
+Handling a Flux stream
+When your function returns a Flux stream object, the system will pass the stream ID of the underlying event stream +to the calling function.
+The input arguments for the event stream ID and time-to-live parameters are provided in the event headers +to your function that implements the TypedLambdaFunction or LambdaFunction.
+The following event headers will be provided to the calling function:
+x-stream-id: streamId
+x-ttl: ttl
+In the calling function, you can create a FluxConsumer to handle the incoming event stream like this:
String streamId = headers.get("x-stream-id");
+long ttl = Utility.getInstance().str2long(headers.get("x-ttl"));
+FluxConsumer<Map<String, Object>> fc = new FluxConsumer<>(streamId, ttl);
+fc.consume(
+ data -> {
+ // handle incoming message
+ },
+ e -> {
+ // handle exception where e is a Throwable
+ },
+ () -> {
+ // handle stream completion
+ }
+);
+Object serialization consideration
+The system is designed to deliver Java primitive and HashMap through an event stream. If you pass Java +primitive such as String or byte[], you do not need to do any serialization.
+If the objects that your function streams over a Mono or Flux channel are not supported, you must perform +custom serialization.
+This can be achieved using the "map" method of the Mono or Flux class.
+For example, your function obtains a stream of Flux result objects from a database call. You can serialize +the objects using a custom serializer like this:
+// "source" is the original Flux object
+Flux<Map<String, Object> serializedStream = source.map(specialPoJo -> {
+ return myCustomSerializer.toMap(specialPoJo);
+});
+return serializedStream;
+Your customSerializer should implement the org.platformlambda.core.models.CustomSerializer interface.
+public interface CustomSerializer {
+ public Map<String, Object> toMap(Object obj);
+ public <T> T toPoJo(Object obj, Class<T> toValueType);
+}
+Extensible authentication function
+You can add authentication function using the optional authentication tag in a service. In "rest.yaml", a service
+for a REST endpoint refers to a function in your application.
An authentication function can be written using a TypedLambdaFunction that takes the input as a "AsyncHttpRequest". +Your authentication function can return a boolean value to indicate if the request should be accepted or rejected.
+A typical authentication function may validate an HTTP header or cookie. e.g. forward the "Bearer token" from the +"Authorization" header to your organization's OAuth 2.0 Identity Provider for validation.
+To approve an incoming request, your custom authentication function can return true.
Optionally, you can add "session" key-values by returning an EventEnvelope like this:
+return new EventEnvelope().setHeader("user_id", "A12345").setBody(true);
+The above example approves the incoming request and returns a "session" variable ("user_id": "A12345") to the next task.
+If your authentication function returns false, the user will receive a "HTTP-401 Unauthorized" error response.
You can also control the status code and error message by throwing an AppException like this:
throw new AppException(401, "Invalid credentials");
+Alternatively, you may implement authentication as a user function in the first step of an event flow. In this case, +the input to the function is defined by the "input data mapping" rules in the event flow configuration.
+The advantage of this approach is that authentication is shown as part of an event flow so that the application design +intention is clear.
+A composable application is assembled from a collection of self-contained functions that are highly reusable.
+Number of workers for a function
+@PreLoad(route = "my.first.function", instances = 10)
+In the above function, the parameter "instances" tells the system to reserve a number of workers for the function. +Workers are running on-demand to handle concurrent user requests.
+Note that you can use smaller number of workers to handle many concurrent users if your function finishes +processing very quickly. If not, you should reserve more workers to handle the work load.
+Concurrency requires careful planning for optimal performance and throughput. +Let's review the strategies for function execution.
+Three strategies for function execution
+A function is executed when an event arrives. There are three function execution strategies.
+| Strategy | +Advantage | +Disadvantage | +
|---|---|---|
| Virtual thread | +Highest throughput in terms of concurrent users. Functionally similar to a suspend function. |
+N/A | +
| Suspend function | +Sequential "non-blocking" for RPC (request-response) that makes code easier to read and maintain |
+Requires coding in Kotlin language | +
| Kernel threads | +Highest performance in terms of operations per seconds |
+Lower number of concurrent threads due to high context switching overheads |
+
Virtual thread
+By default, the system will run your function as a virtual thread because this is the most efficient execution +strategy.
+The "Thread" object in the standard library will operate in non-blocking mode. This means it is safe to use +the Thread.sleep() method. It will release control to the event loop when your function enters into sleep, thus +freeing CPU resources for other functions.
+We have added the "request" methods in the PostOffice API to support non-blocking RPC that leverages the virtual +thread resource suspend/resume functionality.
+Future<EventEnvelope> future = po.request(requestEvent, timeout);
+EventEnvelope result = future.get();
+
+// alternatively, you can do:
+EventEnvelope result = po.request(requestEvent, timeout).get();
+++The PostOffice API is used when you want to do orchestration by code. If you are using Event Script, you can + manage event flows using one or more configuration files.
+
Suspend function
+If you prefer writing business logic in Kotlin, you may use suspend function.
+Similar to virtual thread, a suspend function is a coroutine that can be suspended and resumed. The best use case +for a suspend function is for handling of "sequential non-blocking" request-response. This is the same as "async/await" +in node.js and other programming language.
+To implement a "suspend function", you must implement the KotlinLambdaFunction interface and write code in Kotlin.
+If you are new to Kotlin, please download and run JetBrains Intellij IDE. The quickest way to get productive in Kotlin +is to write a few statements of Java code in a placeholder class and then copy-n-paste the Java statements into the +KotlinLambdaFunction's handleEvent method. Intellij will automatically convert Java code into Kotlin.
+The automated code conversion is mostly accurate (roughly 90%). You may need some touch up to polish the converted +Kotlin code.
+In a suspend function, you can use a set of "await" methods to make non-blocking request-response (RPC) calls.
+For example, to make a RPC call to another function, you can use the awaitRequest method.
Please refer to the FileUploadDemo class in the "examples/lambda-example" project.
val po = PostOffice(headers, instance)
+val fastRPC = FastRPC(headers)
+
+val req = EventEnvelope().setTo(streamId).setHeader(TYPE, READ)
+while (true) {
+ val event = fastRPC.awaitRequest(req, 5000)
+ // handle the response event
+ if (EOF == event.headers[TYPE]) {
+ log.info("{} saved", file)
+ awaitBlocking {
+ out.close()
+ }
+ po.send(streamId, Kv(TYPE, CLOSE))
+ break;
+ }
+ if (DATA == event.headers[TYPE]) {
+ val block = event.body
+ if (block is ByteArray) {
+ total += block.size
+ log.info("Saving {} - {} bytes", filename, block.size)
+ awaitBlocking {
+ out.write(block)
+ }
+ }
+ }
+}
+In the above code segment, it has a "while" loop to make RPC calls to continuously "fetch" blocks of data +from a stream. The status of the stream is indicated in the event header "type". It will exit the "while" loop +when it detects the "End of Stream (EOF)" signal.
+Suspend function will be "suspended" when it is waiting for a response. When it is suspended, it does not +consume CPU resources, thus your application can handle a large number of concurrent users and requests.
+Coroutines run in a "cooperative multitasking" manner. Technically, each function is running sequentially. +However, when many functions are suspended during waiting, it appears that all functions are running concurrently.
+You may notice that there is an awaitBlocking wrapper in the code segment.
Sometimes, you cannot avoid blocking code. In the above example, the Java's FileOutputStream is a blocking method.
+To ensure that a small piece of blocking code in a coroutine does not slow down the "event loop",
+you can apply the awaitBlocking wrapper method. The system will run the blocking code in a separate worker thread
+without blocking the event loop.
In addition to the "await" sets of API, the delay(milliseconds) method puts your function into sleep in a
+non-blocking manner. The yield() method is useful when your function requires more time to execute complex
+business logic. You can add the yield() statement before you execute a block of code. The yield method releases
+control to the event loop so that other coroutines and suspend functions will not be blocked by a heavy weighted
+function.
++Do not block your function because it may block all coroutines since they run in a single kernel thread
+
Suspend function is a powerful way to write high throughput application. Your code is presented in a sequential +flow that is easier to write and maintain.
+You may want to try the demo "file upload" REST endpoint to see how suspend function behaves. If you follow Chapter-1, +your lambda example application is already running. To test the file upload endpoint, here is a simple Python script:
+import requests
+files = {'file': open('some_data_file.txt', 'rb')}
+r = requests.post('http://127.0.0.1:8085/api/upload', files=files)
+print(r.text)
+This assumes you have the python "requests" package installed. If not, please do pip install requests to install
+the dependency.
The uploaded file will be kept in the "/tmp/upload-download-demo" folder.
+To download the file, point your browser to http://127.0.0.1:8085/api/download/some_data_file.txt +Your browser will usually save the file in the "Downloads" folder.
+You may notice that the FileDownloadDemo class is written in Java using the interface
+TypedLambdaFunction<AsyncHttpRequest, EventEnvelope>. The FileDownloadDemo class will run using a kernel thread.
Note that each function is independent and the functions with different execution strategies can communicate in events.
+The output of your function is an "EventEnvelope" so that you can set the HTTP response header correctly. +e.g. content type and filename.
+When downloading a file, the FileDownloadDemo function will block if it is sending a large file. +Therefore, you want it to run as a kernel thread.
+For very large file download, you may want to write the FileDownloadDemo function using asynchronous programming
+with the EventInterceptor annotation or implement a suspend function using KotlinLambdaFunction. Suspend function
+is non-blocking.
++The FastRPC API is used when you want to do orchestration by code. If you are using Event Script, you can + manage event flows using one or more configuration files.
+
Kernel thread pool
+When you add the annotation "KernelThreadRunner" in a function declared as LambdaFunction or TypedLambdaFunction, +the function will be executed using a "kernel thread pool" and Java will run your function in native +"preemptive multitasking" mode.
+While preemptive multitasking fully utilizes the CPU, its context switching overheads may increase as the number of +kernel threads grow. As a rule of thumb, you should control the maximum number of kernel threads to less than 200.
+The parameter kernel.thread.pool is defined with a default value of 100. You can change this value to adjust to
+the actual CPU power in your environment. Keep the default value for best performance unless you have tested the
+limit in your environment.
++When you have more concurrent requests, your application may slow down because some functions + are blocked when the number of concurrent kernel threads is reached.
+
You should reduce the number of "instances" (i.e. worker pool) for a function to a small number so that your
+application does not exceed the maximum limit of the kernel.thread.pool parameter.
Kernel threads are precious and finite resources. When your function is computational intensive or making +external HTTP or database calls in a synchronous blocking manner, you may use it with a small number +of worker instances.
+To rapidly release kernel thread resources, you should write "asynchronous" code. i.e. for event-driven programming, +you can use send event to another function asynchronously, and you can create a callback function to listen +to responses.
+For RPC call, you can use the asyncRequest method to write asynchronous RPC calls. However, coding for asynchronous
+RPC pattern is more challenging. For example, you may want to return a "pending" result immediately using HTTP-202.
+Your code will move on to execute using a "future" that will execute callback methods (onSuccess and onFailure).
+Another approach is to annotate the function as an EventInterceptor so that your function can respond to the user
+in a "future" callback.
For ease of programming, we recommend using virtual thread or suspend function to handle synchronous RPC calls +in a non-blocking manner.
+Solving the puzzle of multithreading performance
+Before the availability of virtual thread technology, Java VM is using kernel threads for code execution. +If you have a lot of users hitting your service concurrently, multiple threads are created to serve concurrent +requests.
+When your code serving the requests make blocking call to other services, the kernel threads are busy while your +user functions wait for responses. Kernel threads that are in the wait state is consuming CPU time.
+If the blocking calls finish very quickly, this may not be an issue.
+However, when the blocking calls take longer to complete, a lot of outstanding kernel threads that are waiting +for responses would compete for CPU resources, resulting in higher internal friction in the JVM that makes your +application running slower. This is not a productive use of computer resources.
+This type of performance issue caused by internal friction is very difficult to avoid. While event driven and +reactive programming that uses asynchronous processing and callbacks would address this artificial bottleneck, +asynchronous code is harder to implement and maintain when the application complexity increases.
+It would be ideal if we can write sequential code that does not block. Sequential code is much easier to write +and read because it communicates the intent of the code clearly.
+Leveraging Java 21 virtual thread, Mercury Composable allows the developer to write code in a sequential manner. +When code in your function makes an RPC call to another service using the PostOffice's "request" API, it returns +a Java Future object but the "Future" object itself is running in a virtual thread. This means when your code +retrieves the RPC result using the "get" method, your code appears "blocked" while waiting for the response +from the target service.
+Although your code appears to be "blocked", the virtual thread is “suspended”. It will wake up when the response +arrives. When a virtual thread is suspended, it does not consume CPU time and the memory structure for keeping +the thread in suspend mode is very small. Virtual thread technology is designed to support tens of thousands, +if not millions, of concurrent RPC requests in a single compute machine, container or serverless instance.
+Mercury Composable supports mixed thread management - virtual threads, suspend functions and kernel threads.
+Functions running in different types of threads are connected loosely in events. This functional isolation +and encapsulation mean that you can precisely control how your application performs for each functional logic block.
+| Chapter-1 | +Home | +Chapter-3 | +
|---|---|---|
| Introduction | +Table of Contents | +REST Automation | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-3/index.html b/docs/guides/CHAPTER-3/index.html
new file mode 100644
index 00000000..b48581c4
--- /dev/null
+++ b/docs/guides/CHAPTER-3/index.html
@@ -0,0 +1,465 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ REST Automation
+The platform-core foundation library contains a built-in non-blocking HTTP server that you can use to create REST +endpoints. Behind the curtain, it is using the vertx web client and server libraries.
+The REST automation system is not a code generator. The REST endpoints in the rest.yaml file are handled by +the system directly - "Config is the code".
+We will use the "rest.yaml" sample configuration file in the "lambda-example" project to elaborate the configuration +approach.
+The rest.yaml configuration has three sections:
+-
+
- REST endpoint definition +
- CORS header processing +
- HTTP header transformation +
Turn on the REST automation engine
+REST automation is optional. To turn on REST automation, add or update the following parameters in the +application.properties file (or application.yml if you like).
+rest.server.port=8085
+rest.automation=true
+yaml.rest.automation=classpath:/rest.yaml
+When rest.automation=true, you can configure the server port using rest.server.port or server.port.
REST automation can co-exist with Spring Boot. Please use rest.server.port for REST automation and
+server.port for Spring Boot.
The yaml.rest.automation tells the system the location of the rest.yaml configuration file.
Support of multiple configuration files
+You can configure more than one location and the system will search and merge them sequentially. +The following example tells the system to merge the rest.yaml config files in the /tmp/config folder +and the project's resources folder.
+yaml.rest.automation=file:/tmp/config/rest.yaml, classpath:/rest.yaml
+Duplicated REST endpoints
+The system will detect duplicated REST endpoint configuation. If there is a duplicated entry, it will +abort the REST endpoint rendering. Your unit tests will fail because REST endpoints are not enabled.
+The application log may look like this:
+INFO - Loading config from classpath:/rest.yaml
+INFO - Loading config from classpath:/event-api.yaml
+ERROR - REST endpoint rendering aborted due to duplicated entry 'POST /api/event' in classpath:/event-api.yaml
+Please correct the rest.yaml configuration files and rebuild your application again.
+Duplicated static content, cors and headers sections
+When duplicated entry is detected, the subsequent one will replace the prior one. A warning will be +shown in the application log like this:
+WARN - Duplicated 'static-content' in classpath:/duplicated-endpoint.yaml will override a prior one
+WARN - Duplicated 'cors' in classpath:/duplicated-endpoint.yaml will override a prior one 'cors_1'
+WARN - Duplicated 'headers' in classpath:/duplicated-endpoint.yaml will override a prior one 'header_1'
+Defining a REST endpoint
+The "rest" section of the rest.yaml configuration file may contain one or more REST endpoints.
+A REST endpoint may look like this:
+ - service: ["hello.world"]
+ methods: ['GET', 'PUT', 'POST', 'HEAD', 'PATCH', 'DELETE']
+ url: "/api/hello/world"
+ timeout: 10s
+ cors: cors_1
+ headers: header_1
+ threshold: 30000
+ tracing: true
+In this example, the URL for the REST endpoint is "/api/hello/world" and it accepts a list of HTTP methods.
+When an HTTP request is sent to the URL, the HTTP event will be sent to the function declared with service route name
+"hello.world". The input event will be the "AsyncHttpRequest" object. Since the "hello.world" function is written
+as an inline LambdaFunction in the lambda-example application, the AsyncHttpRequest is converted to a HashMap.
To process the input as an AsyncHttpRequest object, the function must be written as a regular class. See the +"services" folder of the lambda-example for additional examples.
+The "timeout" value is the maximum time that REST endpoint will wait for a response from your function. +If there is no response within the specified time interval, the user will receive an HTTP-408 timeout exception.
+The "authentication" tag is optional. If configured, the route name given in the authentication tag will be used. +The input event will be delivered to a function with the authentication route name. In this example, it is +"v1.api.auth".
+Your custom authentication function may look like this:
+@PreLoad(route = "v1.api.auth", instances = 10)
+public class SimpleAuthentication implements TypedLambdaFunction<AsyncHttpRequest, Object> {
+
+ @Override
+ public Object handleEvent(Map<String, String> headers, AsyncHttpRequest input, int instance) {
+ // Your authentication logic here. The return value should be true or false.
+ return result;
+ }
+}
+Your authentication function can return a boolean value to indicate if the request should be accepted or rejected.
+If true, the system will send the HTTP request to the service. In this example, it is the "hello.world" function. +If false, the user will receive an "HTTP-401 Unauthorized" exception.
+Optionally, you can use the authentication function to return some session information after authentication. +For example, your authentication can forward the "Authorization" header of the incoming HTTP request to your +organization's OAuth 2.0 Identity Provider for authentication.
+To return session information to the next function, the authentication function can return an EventEnvelope. +It can set the session information as key-values in the response event headers.
+In the lambda-example application, there is a demo authentication function in the AuthDemo class with the +"v1.api.auth" route name. To demonstrate passing session information, the AuthDemo class set the header +"user=demo" in the result EventEnvelope.
+You can test this by visiting http://127.0.0.1:8085/api/hello/generic/1 to invoke the "hello.generic" function.
+The console will print:
+DistributedTrace:55 - trace={path=GET /api/hello/generic/1, service=v1.api.auth, success=true,
+ origin=20230326f84dd5f298b64be4901119ce8b6c18be, exec_time=0.056, start=2023-03-26T20:08:01.702Z,
+ from=http.request, id=aa983244cef7455cbada03c9c2132453, round_trip=1.347, status=200}
+HelloGeneric:56 - Got session information {user=demo}
+DistributedTrace:55 - trace={path=GET /api/hello/generic/1, service=hello.generic, success=true,
+ origin=20230326f84dd5f298b64be4901119ce8b6c18be, start=2023-03-26T20:08:01.704Z, exec_time=0.506,
+ from=v1.api.auth, id=aa983244cef7455cbada03c9c2132453, status=200}
+DistributedTrace:55 - trace={path=GET /api/hello/generic/1, service=async.http.response,
+ success=true, origin=20230326f84dd5f298b64be4901119ce8b6c18be, start=2023-03-26T20:08:01.705Z,
+ exec_time=0.431, from=hello.generic, id=aa983244cef7455cbada03c9c2132453, status=200}
+This illustrates that the HTTP request has been processed by the "v1.api.auth" function. The "hello.generic" function +is wired to the "/api/hello/generic/{id}" endpoint as follows:
+ - service: "hello.generic"
+ methods: ['GET']
+ url: "/api/hello/generic/{id}"
+ # Turn on authentication pointing to the "v1.api.auth" function
+ authentication: "v1.api.auth"
+ timeout: 20s
+ cors: cors_1
+ headers: header_1
+ tracing: true
+The tracing tag tells the system to turn on "distributed tracing". In the console log shown above, you see
+three lines of log from "distributed trace" showing that the HTTP request is processed by "v1.api.auth" and
+"hello.generic" before returning result to the browser using the "async.http.response" function.
++Note: the "async.http.response" is a built-in function to send the HTTP response to the browser.
+
The optional cors and headers tags point to the specific CORS and HEADERS sections respectively.
CORS section
+For ease of development, you can define CORS headers using the CORS section like this.
+This is a convenient feature for development. For cloud native production system, it is most likely that +CORS processing is done at the API gateway level.
+You can define different sets of CORS headers using different IDs.
+cors:
+ - id: cors_1
+ options:
+ - "Access-Control-Allow-Origin: ${api.origin:*}"
+ - "Access-Control-Allow-Methods: GET, DELETE, PUT, POST, PATCH, OPTIONS"
+ - "Access-Control-Allow-Headers: Origin, Authorization, X-Session-Id, X-Correlation-Id,
+ Accept, Content-Type, X-Requested-With"
+ - "Access-Control-Max-Age: 86400"
+ headers:
+ - "Access-Control-Allow-Origin: ${api.origin:*}"
+ - "Access-Control-Allow-Methods: GET, DELETE, PUT, POST, PATCH, OPTIONS"
+ - "Access-Control-Allow-Headers: Origin, Authorization, X-Session-Id, X-Correlation-Id,
+ Accept, Content-Type, X-Requested-With"
+ - "Access-Control-Allow-Credentials: true"
+HEADERS section
+The HEADERS section is used to do some simple transformation for HTTP request and response headers.
+You can add, keep or drop headers for HTTP request and response. Sample HEADERS section is shown below.
+headers:
+ - id: header_1
+ request:
+ #
+ # headers to be inserted
+ # add: ["hello-world: nice"]
+ #
+ # keep and drop are mutually exclusive where keep has precedent over drop
+ # i.e. when keep is not empty, it will drop all headers except those to be kept
+ # when keep is empty and drop is not, it will drop only the headers in the drop list
+ # e.g.
+ # keep: ['x-session-id', 'user-agent']
+ # drop: ['Upgrade-Insecure-Requests', 'cache-control', 'accept-encoding', 'connection']
+ #
+ drop: ['Upgrade-Insecure-Requests', 'cache-control', 'accept-encoding', 'connection']
+
+ response:
+ #
+ # the system can filter the response headers set by a target service,
+ # but it cannot remove any response headers set by the underlying servlet container.
+ # However, you may override non-essential headers using the "add" directive.
+ # i.e. don't touch essential headers such as content-length.
+ #
+ # keep: ['only_this_header_and_drop_all']
+ # drop: ['drop_only_these_headers', 'another_drop_header']
+ #
+ # add: ["server: mercury"]
+ #
+ # You may want to add cache-control to disable browser and CDN caching.
+ # add: ["Cache-Control: no-cache, no-store", "Pragma: no-cache",
+ # "Expires: Thu, 01 Jan 1970 00:00:00 GMT"]
+ #
+ add:
+ - "Strict-Transport-Security: max-age=31536000"
+ - "Cache-Control: no-cache, no-store"
+ - "Pragma: no-cache"
+ - "Expires: Thu, 01 Jan 1970 00:00:00 GMT"
+Static content
+Static content (HTML/CSS/JS bundle), if any, can be placed in the "resources/public" folder in your +application project root. It is because the default value for the "static.html.folder" parameter +in the application configuration is "classpath:/resources/public". If you want to place your +static content elsewhere, you may adjust this parameter. You may point it to the local file system +such as "file:/tmp/html".
+For security reason, you may add the following configuration in the rest.yaml. +The following example is shown in the unit test section of the platform-core library module.
+#
+# Optional static content handling for HTML/CSS/JS bundle
+# -------------------------------------------------------
+#
+# no-cache-pages - tells the browser not to cache some specific pages
+#
+# The "filter" section is a programmatic way to protect certain static content.
+#
+# The filter can be used to inspect HTTP path, headers and query parameters.
+# The typical use case is to check cookies and perform browser redirection
+# for SSO login. Another use case is to selectively add security HTTP
+# response headers such as cache control and X-Frame-Options. You can also
+# perform HTTP to HTTPS redirection.
+#
+# Syntax for the "no-cache-pages", "path" and "exclusion" parameters are:
+# 1. Exact match - complete path
+# 2. Match "startsWith" - use a single "*" as the suffix
+# 3. Match "endsWith" - use a single "*" as the prefix
+#
+# If filter is configured, the path and service parameters are mandatory
+# and the exclusion parameter is optional.
+#
+# In the following example, it will intercept the home page, all contents
+# under "/assets/" and any files with extensions ".html" and ".js".
+# It will ignore all CSS files.
+#
+static-content:
+ no-cache-pages: ["/", "/index.html"]
+ filter:
+ path: ["/", "/assets/*", "*.html", "*.js"]
+ exclusion: ["*.css"]
+ service: "http.request.filter"
+The sample request filter function is available in the platform-core project like this:
+@PreLoad(route="http.request.filter", instances=100)
+public class GetRequestFilter implements TypedLambdaFunction<AsyncHttpRequest, EventEnvelope> {
+
+ @Override
+ public EventEnvelope handleEvent(Map<String, String> headers, AsyncHttpRequest input, int instance) {
+ return new EventEnvelope().setHeader("x-filter", "demo");
+ }
+}
+In the above http.request.filter, it adds a HTTP response header "X-Filter" for the unit test +to validate.
+If you set status code in the return EventEnvelope to 302 and add a header "Location", the system +will redirect the browser to the given URL in the location header. Please be careful to avoid +HTTP redirection loop.
+Similarly, you can throw exception and the HTTP request will be rejected with the given status +code and error message accordingly.
+| Chapter-2 | +Home | +Chapter-4 | +
|---|---|---|
| Function Execution Strategies | +Table of Contents | +Event Script Syntax | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-4/index.html b/docs/guides/CHAPTER-4/index.html
new file mode 100644
index 00000000..30a004dd
--- /dev/null
+++ b/docs/guides/CHAPTER-4/index.html
@@ -0,0 +1,1449 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Event Script Syntax
+Event Script uses YAML to represent an end-to-end transaction flow. A transaction is a business use case, and +the flow can be an API service, a batch job or a real-time transaction.
+Flow list
+This configuration file sits in the project "resources" project and contains a list of filenames.
+The default flow list is "flows.yaml" under the "resources" folder. It may look like this.
+flows:
+ - 'get-profile.yml'
+ - 'create-profile.yml'
+ - 'delete-profile.yml'
+
+location: 'classpath:/flows/'
+The "location" tag is optional. If present, you can tell the system to load the flow config files from +another folder location.
+Multiple flow lists
+You can provide more than one flow list to your application and it can become very handy under different +situations. For instance, to achieve better modularity in complex application, flows can be grouped to +multiple categories based on development team's choice and these flows can be managed in multiple flow +lists. Another great place to use multiple flow list is to include external libraries which contain +pre-defined flow lists. The following example demonstrates that an application loads a list of flows +defined in "flows.yaml" and additional flows defined in "more-flows.yaml" file of a composable library.
+yaml.flow.automation=classpath:/flows.yaml, classpath:/more-flows.yaml
+Writing new REST endpoint and function
+You can use the "flow-demo" subproject as a template to write your own composable application.
+For each filename in the flows.yml, you should create a corresponding configuration file under the +"resources/flows" folder.
+Let's write a new flow called "greetings". You can copy-n-paste the following into a file called "greetings.yml" +under the "resources/flows" folder.
+flow:
+ id: 'greetings'
+ description: 'Simplest flow'
+ ttl: 10s
+
+first.task: 'greeting.demo'
+
+tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ process: 'greeting.demo'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+ description: 'Hello World'
+ execution: end
+In the application.properties, you can specify the following parameter:
+yaml.flow.automation=classpath:/flows.yaml
+and update the "flows.yaml" file in the resources folder as follows:
+flows:
+ - 'get-profile.yml'
+ - 'create-profile.yml'
+ - 'delete-profile.yml'
+ - 'greetings.yml'
+Then, you can add a new REST endpoint in the "rest.yaml" configuration file like this.
+ - service: "http.flow.adapter"
+ methods: ['GET']
+ url: "/api/greetings/{user}"
+ flow: 'greetings'
+ timeout: 10s
+ cors: cors_1
+ headers: header_1
+The above REST endpoint takes the path parameter "user". The task executor will map the path parameter to the +input arguments (headers and body) in your function. Now you can write your new function with the named route +"greeting.demo". Please copy-n-paste the following into a Java class called "Greetings" and save in the package +under "my.organization.tasks" in the source project.
+++Note: "my.organization" package name is an example. Please replace it with your organization package path.
+
@PreLoad(route="greeting.demo", instances=10, isPrivate = false)
+public class Greetings implements TypedLambdaFunction<Map<String, Object>, Map<String, Object>> {
+ private static final String USER = "user";
+
+ @Override
+ public Map<String, Object> handleEvent(Map<String, String> headers, Map<String, Object> input, int instance) {
+ if (input.containsKey(USER)) {
+ String user = input.get(USER).toString();
+ Map<String, Object> result = new HashMap<>();
+ result.put(USER, user);
+ result.put("message", "Welcome");
+ result.put("time", new Date());
+ return result;
+ } else {
+ throw new IllegalArgumentException("Missing path parameter 'user'");
+ }
+ }
+}
+For the flow-engine to find your new function, please update the key-value for "web.component.scan" in +application.properties:
+web.component.scan=my.organization
+To test your new REST endpoint, flow configuration and function, please point your browser to
+http://127.0.0.1:8100/api/greetings/my_name
+You can replace "my_name" with your first name to see the response to the browser.
+Flow configuration syntax
+In your "greetings.yml" file above, you find the following key-values:
+flow.id - Each flow must have a unique flow ID. The flow ID is usually originated from a user facing endpoint
+through an event adapter. For example, you may write an adapter to listen to a cloud event in a serverless deployment.
+In The most common one is the HTTP adapter.
The flow ID is originated from the "rest.yaml". The flow-engine will find the corresponding flow configuration
+and create a new flow instance to process the user request.
flow.description - this describes the purpose of the flow
flow.ttl - "Time to live (TTL)" timer for each flow. You can define the maximum time for a flow to finish processing.
+All events are delivered asynchronously and there is no timeout value for each event. The TTL defines the time budget
+for a complete end-to-end flow. Upon expiry, an unfinished flow will be aborted.
first.task - this points to the route name of a function (aka "task") to which the flow engine will deliver
+the incoming event.
The configuration file contains a list of task entries where each task is defined by "input", "process", "output" +and "execution" type. In the above example, the execution type is "end", meaning that it is the end of a transaction +and its result set will be delivered to the user.
+Underlying Event System
+The Event Script system uses platform-core as the event system where it encapsulates Java Virtual Threads, +Eclipse Vertx, Kotlin coroutine and suspend function.
+The integration points are intentionally minimalist. For most use cases, the user application does not need +to make any API calls to the underlying event system.
+REST automation and HTTP adapter
+The most common transaction entry point is a REST endpoint. The event flow may look like this:
+Request -> "http.request" -> "task.executor" -> user defined tasks
+ -> "async.http.response" -> Response
+REST automation is part of the Mercury platform-core library. It contains a non-blocking HTTP server that converts +HTTP requests and responses into events.
+It routes an HTTP request event to the HTTP adapter if the "flow" tag is provided.
+In the following example, the REST endpoint definition is declared in a "rest.yaml" configuration. It will route +the URI "/api/decision" to the HTTP adapter that exposes its service route name as "http.flow.adapter".
+rest:
+ - service: "http.flow.adapter"
+ methods: ['GET']
+ url: "/api/decision?decision=_"
+ flow: 'decision-test'
+ timeout: 10s
+ cors: cors_1
+ headers: header_1
+ tracing: true
+The "cors" and "headers" tags are optional. When specified, the REST endpoint will insert CORS headers and HTTP request +headers accordingly.
+For rest.yaml syntax, please refer to https://accenture.github.io/mercury-composable/guides/CHAPTER-3
+The HTTP adapter maps the HTTP request dataset and the flow ID into a standard event envelope for delivery to the +flow engine.
+The HTTP request dataset, addressable with the "input." namespace, contains the following:
+| Key | +Values | +
|---|---|
| method | +HTTP method | +
| uri | +URI path | +
| header | +HTTP headers | +
| cookie | +HTTP cookies | +
| path_parameter | +Path parameters if any | +
| query | +HTTP query parameters if any | +
| body | +HTTP request body if any | +
| stream | +input stream route ID if any | +
| ip | +remote IP address | +
| filename | +filename if request is a multipart file upload | +
| session | +authenticated session key-values if any | +
For easy matching, keys of headers, cookies, query and path parameters are case-insensitive.
+Regular API uses JSON and XML and they will be converted to a hashmap in the event's body.
+For special use cases like file upload/download, your application logic may invoke a streaming API to retrieve +the binary payload. Please refer to the following sections for details.
+https://accenture.github.io/mercury-composable/guides/APPENDIX-III/#send-http-request-body-as-a-stream
+https://accenture.github.io/mercury-composable/guides/APPENDIX-III/#read-http-response-body-stream
+Task and its corresponding function
+Each task in a flow must have a corresponding function. You can assign a task name to the function using the
+Preload annotation like this.
@PreLoad(route="greeting.demo", instances=10)
+public class Greetings implements TypedLambdaFunction<Map<String, Object>, Map<String, Object>> {
+ @Override
+ public Map<String, Object> handleEvent(Map<String, String> headers, Map<String, Object> input, int instance) {
+ // business logic here
+ return someOutput;
+ }
+}
+The "route" in the Preload annotation is the task name. The "instances" define the maximum number of "workers" that +the function can handle concurrently. The system is designed to be reactive and the function does not consume memory +and CPU resources until an event arrives.
+You may also define concurrency using environment variable. You can replace the "instances" with envInstances using
+standard environment variable syntax like ${SOME_ENV_VARIABLE:default_value}.
Unique task naming
+Composable functions are designed to be reusable. By changing some input data mapping to feed different parameters and +payload, your function can behave differently.
+Therefore, it is quite common to use the same function ("process") more than once in a single event flow.
+When a task is not named, the "process" tag is used to name the task.
+Since each task must have a unique name for event routing, we cannot use the same "process" name more than once in an +event flow. To handle this use case, you can create unique names for the same function (i.e. "process") like this:
+flow:
+ id: 'greetings'
+ description: 'Simplest flow'
+ ttl: 10s
+
+first.task: 'my.first.task'
+
+tasks:
+ - name: 'my.first.task'
+ input:
+ - 'input.path_parameter.user -> user'
+ process: 'greeting.demo'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+ description: 'Hello World'
+ execution: sequential
+ next:
+ - 'another.task'
+The above event flow configuration uses "my.first.task" as a named route for "greeting.demo" by adding the +"name" tag to the composable function.
+For configuration simplicity, the "name" tag is optional. If not provided, the process name is assumed to be +the unique "task" name.
+++Important: The Event Manager performs event choreography using the unique task name. + Therefore, when the "process" name for the function is not unique, you must create unique task "names" + for the same function to ensure correct routing.
+
Assigning multiple route names to a single function
+The built-in distributed tracing system tracks the actual composable functions using the "process" name +and not the task names.
+When there is a need to track the task names in distributed trace, you can tell the system to create +additional instances of the same function with different route names.
+You can use a comma separated list as the route name like this:
+@PreLoad(route="greeting.case.1, greeting.case.2", instances=10)
+public class Greetings implements TypedLambdaFunction<Map<String, Object>, Map<String, Object>> {
+
+ @Override
+ public Map<String, Object> handleEvent(Map<String, String> headers, Map<String, Object> input, int instance) {
+ // business logic here
+ return someResult;
+ }
+}
+++Note: The "unique task naming" method is more memory efficient than creating additional route names
+
Preload overrides
+Once a composable function is published as a reusable library in the artifactory, its route name and +number of instances are fixed using the "PreLoad" annotation in the function class.
+Without refactoring your libary, you can override its route name and instances using a preload override +file like this:
+preload:
+ - original: 'greeting.demo'
+ routes:
+ - 'greeting.case.1'
+ - 'greeting.case.2'
+ # the "instances" tag is optional
+ instances: 20
+ - original: 'v1.another.reusable.function'
+ keep-original: true
+ routes:
+ - 'v1.reusable.1'
+ - 'v1.reusable.2'
+In the above example, the function associated with "greeting.demo" will be preloaded as "greeting.case.1" +and "greeting.case.2". The number of maximum concurrent instances is also changed from 10 to 20.
+In the second example, "v1.another.reusable.function" is updated as "v1.reusable.1" and "v1.reusable.2" +and the number of concurrent instances is not changed. The original route "v1.another.reusable.function" is +preserved when the "keep-original" parameter is set to true.
+Assuming the above file is "preload-override.yaml" in the "resources" folder of the application source code +project, you should add the following parameter in application.properties to activate this preload override +feature.
+yaml.preload.override=classpath:/preload-override.yaml
+Multiple preload override config files
+When you publish a composable function as a library, you may want to ensure the route names of the functions are +merged properly. In this case, you can bundle a library specific preload override config file.
+For example, your library contains a "preload-kafka.yaml" to override some route names, you can add it to the +yaml.preload.override parameter like this:
+yaml.preload.override=classpath:/preload-override.yaml, classpath:/preload-kafka.yaml
+The system will then merge the two preload override config files.
+The concurrency value of a function is overwritten using the "instances" parameter in the first preload override file. +Subsequent override of the "instances" parameter is ignored. i.e. the first preload override file will take precedence.
+Hierarchy of flows
+Inside a flow, you can run one or more sub-flows.
+To do this, you can use the flow protocol identifier (flow://) to indicate that the task is a flow.
For example, when running the following task, "flow://my-sub-flow" will be executed like a regular task.
+tasks:
+ - input:
+ - 'input.path_parameter.user -> header.user'
+ - 'input.body -> body'
+ process: 'flow://my-sub-flow'
+ output:
+ - 'result -> model.pojo'
+ description: 'Execute a sub-flow'
+ execution: sequential
+ next:
+ - 'my.next.function'
+If the sub-flow is not available, the system will throw an error stating that it is not found.
+Hierarchy of flows would reduce the complexity of a single flow configuration file. The "time-to-live (TTL)" +value of the parent flow should be set to a value that covers the complete flow including the time used in +the sub-flows.
+For simplicity, the input data mapping for a sub-flow should contain only the "header" and "body" arguments.
+Tasks and data mapping
+All tasks for a flow are defined in the "tasks" section.
+Input/Output data mapping
+A function is self-contained. This modularity reduces application complexity because the developer only needs +interface contract details for a specific function.
+To handle this level of modularity, the system provides configurable input/output data mapping.
+Namespaces for I/O data mapping
+| Type | +Keyword and/or namespace | +LHS / RHS | +Mappings | +
|---|---|---|---|
| Flow input dataset | +input. |
+left | +input | +
| Flow output dataset | +output. |
+right | +output | +
| Function input body | +no namespace required | +right | +input | +
| Function input or output headers | +header or header. |
+right | +I/O | +
| Function output result set | +result. |
+left | +output | +
| Function output status code | +status |
+left | +output | +
| Decision value | +decision |
+right | +output | +
| State machine dataset | +model. |
+left/right | +I/O | +
| External state machine key-value | +ext: |
+right | +I/O | +
Note that external state machine namespace uses ":" to indicate that the key-value is external.
+Constants for input data mapping
+| Type | +Keyword for the left-hand-side argument | +
|---|---|
| String | +text(example_value) |
+
| Integer | +int(number) |
+
| Long | +long(number) |
+
| Float | +float(number) |
+
| Double | +double(number) |
+
| Boolean | +boolean(true or false) |
+
| Map | +map(k1=v1, k2=v2)map(base.config.parameter) |
+
| File | +file(text:file_path)file(binary:file_path) |
+
| Classpath | +classpath(text:file_path)classpath(binary:file_path) |
+
For input data mapping, the "file" constant type is used to load some file content as an argument of a user function. +You can tell the system to render the file as "text" or "binary". Similarly, the "classpath" constant type refers +to static file in the application source code's "resources" folder.
+The "map" constant type is used for two purposes:
+1. Map of key-values
+The following example illustrates creation of a map of key-values. In the first entry, a map of 2 key-values +is set as the input argument "myMap" of a user function. In the second entry, the map's values are retrieved +from the key "some.key" in base configuration and the environment variable "ENV_VAR_ONE".
+'map(k1=v1, k2=v2) -> myMap'
+'map(k1=${some.key}, k2=${ENV_VAR_ONE}) -> myMap'
+Note that the comma character is used as a separator for each key-value pair. If the value contains a comma, +the system cannot parse the key-values correctly. In this case, please use the 2nd method below.
+2. Mapping values from application.yml
+The following input data mapping sets the value of "my.key" from the application.yml base configuration file +to the input argument "myKey" of a user function.
+'map(my.key) -> myKey'
+Since the system uses both application.properties and application.yml as base configuration files, +you can use either configuration files depending on the data type of the value.
+For application.properties, "map(my.key)" is the same as "text(${my.key})".
+For application.yml, "map(my.key)" would set a primitive value (text, integer, float, boolean), +a hash map of key-values or an array of values.
+Special content type for output data mapping
+| Type | +Keyword for the right-hand-side argument | +
|---|---|
| File | +file(file_path) |
+
For output data mapping, the "file" content type is used to save some data from the output of a user function +to a file in the local file system.
+Decison value
+The "decision" keyword applies to "right hand side" of output data mapping statement in a decision task only +(See "Decision" in the task section).
+Each flow has its own input and output
+Each function has its input headers, input body and output result set. +Optionally, a function can return an EventEnvelope object to hold its result set in the "body", a "status" code +and one or more header key-values.
+Since each function is stateless, a state machine (with namespace model.) is available as a temporary memory store
+for transaction states that can be passed from one task to another.
All variables are addressable using the standard dot-bracket convention.
+For example, "hello.world" will retrieve the value 100 from this data structure:
{
+ "hello": {
+ "world": 100
+ }
+}
+and "numbers[1]" will retrieve the value 200 below:
{ "numbers": [100, 200] }
+The assignment is done using the -> syntax.
In the following example, the HTTP input query parameter 'amount' is passed as input body argument 'amount' +to the task 'simple.decision'. The result (function "return value") from the task will be mapped to the +special "decision" variable that the flow engine will evaluate. This assumes the result is a boolean or +numeric value.
+The "decision" value is also saved to the state machine (model) for subsequent tasks to evaluate.
- input:
+ - 'input.query.amount -> amount'
+ process: 'simple.decision'
+ output:
+ - 'result -> decision'
+ - 'result -> model.decision'
+Metadata for each flow instance
+For each flow instance, the state machine in the "model" namespace provides the following metadata that +you can use in the input/output data mapping. For example, you can set this for an exception handler to +log additional information.
+| Type | +Keyword | +Comment | +
|---|---|---|
| Flow ID | +model.flow |
+The ID of the event flow config | +
| Trace ID | +model.trace |
+Optional traceId when tracing is turned on | +
| Correlation ID | +model.cid |
+Correlation ID of the inbound request | +
Special handling for header
+When function input keyword header is specified in the "right hand side" of an input data mapping statement,
+it refers to the input event envelope's headers. Therefore, it assumes the "left hand side" to resolve into
+a Map object of key-values. Otherwise, it will reject the input data mapping statement with an error like this:
Invalid input mapping 'text(ok) -> header', expect: Map, Actual: String
+When function input namespace header. is used, the system will map the value resolved from the "left hand side"
+statement into the specific header.
For example, the input data mapping statement text(ok) -> header.demo will set "demo=ok" into input event
+envelope's headers.
When function output keyword header is specified in the "left hand side" of an output data mapping statement,
+it will resolve as a Map from the function output event envelope's headers.
Similarly, when function output namespace header. is used, the system will resolve the value from a specific
+key of the function output event envelope's headers.
Function input and output
+To support flexible input data mapping, the input to a function must be either Map<String, Object> or PoJo.
+The output (i.e. result set) of a function can be Map, PoJo or Java primitive.
Your function can implement the TypedLambdaFunction interface to configure input and output.
Since a data structure is passed to your function's input argument as key-values, you may create a PoJo class +to deserialize the data structure.
+To tell the system that your function is expecting input as a PoJo, you can use the special notation * in
+the right hand side.
For example, the following entry tells the system to set the value in "model.dataset" as a PoJo input.
+ - input:
+ - 'model.dataset -> *'
+++If the value from the left hand side is not a map, the system will ignore the input mapping command and +print out an error message in the application log.
+
Setting function input headers
+When function input body is used to hold a PoJo, we may use function input headers to pass other arguments +to the function without changing the data structure of a user defined PoJo.
+In the following example, the HTTP query parameter "userid" will be mappped to the function input header +key "user" and the HTTP request body will be mapped to the function input body.
+ - input:
+ - 'input.query.userid -> header.user'
+ - 'input.body -> *'
+ process: 'my.user.function'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+Task types
+Decision task
+A decision task makes decision to select the next task to execute. It has the tag execution=decision.
In the output data mapping section, it must map the corresponding result set or its key-value to the decision object.
The "next" tag contains a list of tasks to be selected based on the decision value.
+If decision value is boolean, a true value will select the first task. Otherwise, the second task will be selected.
If decision value is an integer, the number should start from 1 where the corresponding "next" task +will be selected.
+tasks:
+ - input:
+ - 'input.query.decision -> decision'
+ process: 'simple.decision'
+ output:
+ - 'result -> model.decision'
+ - 'result -> decision'
+ description: 'Simple decision test'
+ execution: decision
+ next:
+ - 'decision.case.one'
+ - 'decision.case.two'
+Response task
+A response task will provide result set as a flow output or "response". A response task allows the flow to respond +to the user or caller immediately and then move on to the next task asynchronously. For example, telling the user +that it has accepted a request and then moving on to process the request that may take longer time to run.
+A response task has the tag execution=response and a "next" task.
tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.seq -> sequence'
+ process: 'sequential.one'
+ output:
+ - 'result -> model.pojo'
+ - 'result -> output.body'
+ description: 'Pass a pojo to another task'
+ execution: response
+ next:
+ - 'sequential.two'
+End task
+An end task indicates that it is the last task of the transaction processing in a flow. If the flow has not executed +a response task, the end task will generate the response. Response is defined by output data mapping.
+This task has the tag execution=end.
For example, the greeting task in the unit tests is an end task.
+ - input:
+ - 'input.path_parameter.user -> user'
+ process: 'greeting.demo'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+ description: 'Hello World'
+ execution: end
+Sequential task
+Upon completion of a sequential task, the next task will be executed.
+This task has the tag execution=sequential.
In the following example, sequential.two will be executed after sequential.one.
tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.seq -> sequence'
+ process: 'sequential.one'
+ output:
+ - 'result -> model.pojo'
+ description: 'Pass a pojo to another task'
+ execution: sequential
+ next:
+ - 'sequential.two'
+Parallel task
+Upon completion of a parallel task, all tasks in the "next" task list will be executed in parallel.
This task has the tag execution=parallel.
In this example, parallel.one and parallel.two will run after begin.parallel.test
tasks:
+ - input:
+ - 'int(2) -> count'
+ process: 'begin.parallel.test'
+ output: []
+ description: 'Setup counter for two parallel tasks'
+ execution: parallel
+ next:
+ - 'parallel.one'
+ - 'parallel.two'
+Fork-n-join task
+Fork-n-join is a parallel processing pattern.
+A "fork" task will execute multiple "next" tasks in parallel and then consolidate the result sets before running +the "join" task.
+This task has the tag execution=fork. It must have a list of "next" tasks and a "join" task.
It may look like this:
+tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.seq -> sequence'
+ process: 'sequential.one'
+ output:
+ - 'result -> model.pojo'
+ description: 'Pass a pojo to another task'
+ execution: fork
+ next:
+ - 'echo.one'
+ - 'echo.two'
+ join: 'join.task'
+Sink task
+A sink task is a task without any next tasks. Sink tasks are used by fork-n-join and pipeline tasks as reusable modules.
+This task has the tag execution=sink.
- input:
+ - 'text(hello-world-two) -> key2'
+ process: 'echo.two'
+ output:
+ - 'result.key2 -> model.key2'
+ description: 'Hello world'
+ execution: sink
+Pipeline feature
+Pipeline is an advanced feature of Event Script.
+Pipeline task
+A pipeline is a list of tasks that will be executed orderly within the current task.
+When the pipeline is done, the system will execute the "next" task.
+This task has the tag execution=pipeline.
tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.seq -> sequence'
+ process: 'sequential.one'
+ output:
+ - 'result -> model.pojo'
+ description: 'Pass a pojo to another task'
+ execution: pipeline
+ pipeline:
+ - 'echo.one'
+ - 'echo.two'
+ next:
+ - 'echo.three'
+Some special uses of pipelines include "for/while-loop" and "continue/break" features.
+Simple for-loop
+In the following example, the loop.statement contains a for-loop that uses a variable in the state machine to
+evaluate the loop.
In this example, the pipeline will be executed three times before passing control to the "next" task.
+tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.seq -> sequence'
+ process: 'sequential.one'
+ output:
+ - 'result -> model.pojo'
+ description: 'Pass a pojo to another task'
+ execution: pipeline
+ loop:
+ statement: 'for (model.n = 0; model.n < 3; model.n++)'
+ pipeline:
+ - 'echo.one'
+ - 'echo.two'
+ - 'echo.three'
+ next:
+ - 'echo.four'
+Simple while loop
+The loop.statement may use a "while loop" syntax like this:
loop:
+ statement: 'while (model.running)'
+To exit the above while loop, one of the functions in the pipeline should return a boolean "false" value with
+output "data mapping" to the model.running variable.
For loop with break/continue decision
+In the following example, the system will evaluate if the model.quit variable is true.
+If yes, the break or continue condition will be executed.
The state variable is obtained after output data mapping and any task in the pipeline can set a key-value for +mapping into the state variable.
+tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.seq -> sequence'
+ process: 'sequential.one'
+ output:
+ - 'result -> model.pojo'
+ description: 'Pass a pojo to another task'
+ execution: pipeline
+ loop:
+ statement: 'for (model.n = 0; model.n < 3; model.n++)'
+ condition:
+ - 'if (model.quit) break'
+ pipeline:
+ - 'echo.one'
+ - 'echo.two'
+ - 'echo.three'
+ next:
+ - 'echo.four'
+Handling exception
+You can define exception handler at the top level or at the task level.
+Exception is said to occur when a user function throws exception or returns an EventEnvelope object with +a status code equals to or larger than 400.
+The event status uses the same numbering scheme as HTTP exception status code. +Therefore, status code less than 400 is not considered an exception.
+Top-level exception handler
+Top-level exception handler is a "catch-all" handler. You can define it like this:
+flow:
+ id: 'greetings'
+ description: 'Simplest flow of one task'
+ ttl: 10s
+ exception: 'v1.my.exception.handler'
+In this example, the v1.my.exception.handler should point to a corresponding exception handler that you provide.
The following input arguments will be delivered to your function when exception happens.
+| Key | +Description | +
|---|---|
| status | +Exception status code | +
| message | +Error message | +
| stack | +Stack trace in a text string | +
The exception handler function can be an "end" task to abort the transaction or a decision task +to take care of the exception. For example, the exception handler can be a "circuit-breaker" to retry a request.
+Task-level exception handler
+You can attach an exception handler to a task. One typical use is the "circuit breaker" pattern. +In the following example, the user function "breakable.function" may throw an exception for some error condition. +The exception will be caught by the "v1.circuit.breaker" function.
+ - input:
+ - 'input.path_parameter.accept -> accept'
+ - 'model.attempt -> attempt'
+ process: 'breakable.function'
+ output:
+ - 'int(0) -> model.attempt'
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+ description: 'This demo function will break until the "accept" number is reached'
+ execution: end
+ exception: 'v1.circuit.breaker'
+The configuration for the circuit breaker function may look like this:
+ - input:
+ - 'model.attempt -> attempt'
+ - 'int(2) -> max_attempts'
+ - 'error.code -> status'
+ - 'error.message -> message'
+ - 'error.stack -> stack'
+ process: 'v1.circuit.breaker'
+ output:
+ - 'result.attempt -> model.attempt'
+ - 'result.decision -> decision'
+ - 'result.status -> model.status'
+ - 'result.message -> model.message'
+ description: 'Just a demo circuit breaker'
+ execution: decision
+ next:
+ - 'breakable.function'
+ - 'abort.request'
+An exception handler will be provided with the "error" object that contains error code, error message and an exception +stack trace. The exception handler can inspect the error object to make decision of the next step.
+For circuit breaker, we can keep the number of retry attempts in the state machine under "model.attempt" or any +key name that you prefer. In the above example, it sets an integer constant of 2 for the maximum attempts.
+The circuit breaker can then evaluate if the number of attempts is less than the maximum attempts. If yes, it will +return a decision of "true" value to tell the system to route to the "breakable.function" again. Otherwise, it will +return a decision of "false" value to abort the request.
+A more sophisticated circuit breaker may be configured with "alternative execution paths" depending on the error +status and stack trace. In this case, the decision value can be a number from 1 to n that corresponds to the "next" +task list.
+Exception handlers may be used in both queries and transactions. For a complex transaction, the exception handler +may implement some data rollback logic or recovery mechanism.
+Best practice
+When a task-level exception handler throws exception, it will be caught by the top-level exception handler, if any.
+A top-level exception handler should not throw exception. Otherwise it may go into an exception loop.
+Therefore, we recommend that an exception handler should return regular result set in a PoJo or a Map object.
+An example of task-level exception handler is shown in the "HelloException.class" in the unit test section of +the event script engine where it set the status code in the result set so that the system can map the status code +from the result set to the next task or to the HTTP output status code.
+Advanced features
+Simple type matching and conversion
+Event script's state machine supports simple type matching and conversion. This "impedance matching" feature +allows us to accommodate minor interface contract changes without refactoring business logic of a user function.
+This is supported in both the left-hand-side and right-hand-side of both input and output data mappings.
+For the left-hand-side, the state machine's model value is matched or converted to the target data type before +setting the value of the right-hand-side. The state machine values are unchanged.
+For the right-hand-side, the matched or converted value is applied to the state machine's model value.
+The syntax is model.somekey:type where "type" is one of the following:
| Type | +Match value as | +Example | +
|---|---|---|
| text | +text string | +model.someKey:text | +
| binary | +byte array | +model.someKey:binary | +
| int | +integer or -1 if not numeric | +model.someKey:int | +
| long | +long or -1 if not numeric | +model.someKey:long | +
| float | +float or -1 if not numeric | +model.someKey:float | +
| double | +double or -1 if not numeric | +model.someKey:double | +
| boolean | +true or false | +model.someKey:boolean | +
| boolean(value) | +true if value matches | +model.someKey:boolean(positive) | +
| boolean(value=true) | +true if value matches | +model.someKey:boolean(positive=true) | +
| boolean(value=false) | +false if value matches | +model.someKey:boolean(negative=false) | +
| and(model.key) | +boolean AND of 2 model keys | +model.someKey:and(model.another) | +
| or(model.key) | +boolean OR of 2 model keys | +model.someKey:or(model.another) | +
| substring(start, end) | +extract a substring | +model.someKey:substring(0, 5) | +
| substring(start) | +extract a substring | +model.someKey:substring(5) | +
| b64 | +byte-array to Base64 text | +model.someKey:b64 | +
| b64 | +Base64 text to byte-array | +model.someKey:b64 | +
For boolean with value matching, the value can be null. This allows your function to test if the +key-value in the left-hand-side is a null value.
+For Base64 type matching, if the key-value is a text string, the system will assume it is a +Base64 text string and convert it to a byte-array. If the key-value is a byte-array, the system +will encode it into a Base64 text string.
+An interesting use case of type matching is a simple decision task using the built-in no-op function. +For example, when a control file for the application is not available, your application will switch +to run in dev mode.
+A sample task may look like this:
+first.task: 'no.op'
+
+tasks:
+- input:
+ - 'file(binary:/tmp/interesting-config-file) -> model.is-local:boolean(null=true)'
+ process: 'no.op'
+ output:
+ - 'model.is-local -> decision'
+ execution: decision
+ next:
+ - 'start.in.dev.mode'
+ - 'start.in.cloud'
+External state machine
+The in-memory state machine is created for each query or transaction flow and it is temporal.
+For complex transactions or long running work flows, you would typically want to externalize some transaction +states to a persistent store such as a distributed cache system or a high performance key-value data store.
+In these use cases, you can implement an external state machine function and configure it in a flow.
+Below is an example from a unit test. When you externalize a key-value to an external state machine, +you must configure the route name (aka level-3 functional topic) of the external state machine.
+Note that when passing a null value to a key of an external state machine means "removal".
external.state.machine: 'v1.ext.state.machine'
+
+tasks:
+ - input:
+ # A function can call an external state machine using input or output mapping.
+ # In this example, it calls external state machine from input data mapping.
+ - 'input.path_parameter.user -> ext:/${app.id}/user'
+ - 'input.body -> model.body'
+ # demonstrate saving constant to state machine and remove it using model.none
+ - 'text(world) -> ext:hello'
+ - 'model.none -> ext:hello'
+ process: 'no.op'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ # It calls external state machine again from output data mapping
+ - 'input.body -> ext:/${app.id}/body'
+ - 'input.body -> output.body'
+ - 'text(message) -> ext:test'
+ - 'model.none -> ext:test'
+ description: 'Hello World'
+ execution: end
+The "external.state.machine" parameter is optional.
+When present, the system will send a key-value from the current flow instance's state machine +to the function implementing the external state machine. The system uses the "ext:" namespace +to externalize a state machine's key-value.
+Note that the delivery of key-values to the external state machine is asynchronous. +Therefore, please assume eventual consistency.
+You should implement a user function as the external state machine.
+The input interface contract to the external state machine for saving a key-value is:
+header.type = 'put'
+header.key = key
+body = value
+Your function should save the input key-value to a persistent store.
+In another flow that requires the key-value, you can add an initial task +to retrieve from the persistent store and do "output data mapping" to +save to the in-memory state machine so that your transaction flow can +use the persisted key-values to continue processing.
+In the unit tests of the event-script-engine subproject, these two flows work together:
+externalize-put-key-value
+externalize-get-key-value
+IMPORTANT: Events to an external state machine are delivered asynchronously. If you want to guarantee +message sequencing, please do not set the "instances" parameter in the PreLoad annotation.
+To illustrate a minimalist implementation, below is an example of an external state machine in the +event-script-engine's unit test section.
+@PreLoad(route = "v1.ext.state.machine")
+public class ExternalStateMachine implements LambdaFunction {
+ private static final Logger log = LoggerFactory.getLogger(ExternalStateMachine.class);
+
+ private static final ManagedCache store = ManagedCache.createCache("state.machine", 5000);
+ private static final String TYPE = "type";
+ private static final String PUT = "put";
+ private static final String GET = "get";
+ private static final String REMOVE = "remove";
+ private static final String KEY = "key";
+
+ @Override
+ public Object handleEvent(Map<String, String> headers, Object input, int instance) {
+ if (!headers.containsKey(KEY)) {
+ throw new IllegalArgumentException("Missing key in headers");
+ }
+ String type = headers.get(TYPE);
+ String key = headers.get(KEY);
+ if (PUT.equals(type) && input != null) {
+ log.info("Saving {} to store", key);
+ store.put(key, input);
+ return true;
+ }
+ if (GET.equals(type)) {
+ Object v = store.get(key);
+ if (v != null) {
+ log.info("Retrieve {} from store", key);
+ return v;
+ } else {
+ return null;
+ }
+ }
+ if (REMOVE.equals(type)) {
+ if (store.exists(key)) {
+ store.remove(key);
+ log.info("Removed {} from store", key);
+ return true;
+ } else {
+ return false;
+ }
+ }
+ return false;
+ }
+}
+Future task scheduling
+You may add a “delay” tag in a task so that it will be executed later. +This feature is usually used for unit tests or "future task scheduling".
+Since the system is event-driven and non-blocking, the delay is simulated by event scheduling. +It does not block the processing flow.
+| Type | +Value | +Example | +
|---|---|---|
| Fixed delay | +Milliseconds | +delay=1000 | +
| Variable delay | +State machine variable | +delay=model.delay | +
When delay is set to a state variable that its value is not configured by a prior data mapping, +the delay command will be ignored.
+An example task that has an artificial delay of 2 seconds:
+tasks:
+ - input:
+ - 'input.path_parameter.user -> user'
+ - 'input.query.ex -> exception'
+ - 'text(hello world) -> greeting'
+ process: 'greeting.test'
+ output:
+ - 'text(application/json) -> output.header.content-type'
+ - 'result -> output.body'
+ description: 'Hello World'
+ execution: end
+ delay: 2000
+| Chapter-3 | +Home | +Chapter-5 | +
|---|---|---|
| REST Automation | +Table of Contents | +Build, Test and Deploy | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-5/index.html b/docs/guides/CHAPTER-5/index.html
new file mode 100644
index 00000000..42151222
--- /dev/null
+++ b/docs/guides/CHAPTER-5/index.html
@@ -0,0 +1,550 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Build, Test and Deploy
+The first step in writing an application is to create an entry point for your application.
+Main application
+A minimalist main application template is shown as follows:
+@MainApplication
+public class MainApp implements EntryPoint {
+ public static void main(String[] args) {
+ AutoStart.main(args);
+ }
+ @Override
+ public void start(String[] args) {
+ // your startup logic here
+ log.info("Started");
+ }
+}
+Note that MainApplication is mandatory. You must have at least one "main application" module.
+++Note: Please adjust the parameter "web.component.scan" in application.properties + to point to your user application package(s) in your source code project.
+
If your application does not require additional startup logic, you may just print a greeting message.
+The AutoStart.main() statement in the "main" method is used when you want to start your application within the IDE.
+You can "right-click" the main method and select "run".
You can also build and run the application from command line like this:
+cd sandbox/mercury-composable/examples/lambda-example
+mvn clean package
+java -jar target/lambda-example-4.0.16.jar
+The lambda-example is a sample application that you can use as a template to write your own code. Please review +the pom.xml and the source directory structure. The pom.xml is pre-configured to support Java and Kotlin.
+In the lambda-example project root, you will find the following directories:
+src/main/java
+src/main/kotlin
+src/test/java
+Note that kotlin unit test directory is not included because you can test all functions in Java unit tests.
+Since all functions are connected using the in-memory event bus, you can test any function by sending events +from a unit test module in Java. If you are comfortable with the Kotlin language, you may also set up Kotlin +unit tests accordingly. There is no harm having both types of unit tests in the same project.
+Source code documentation
+Since the source project contains both Java and Kotlin, we have replaced javadoc maven plugin with Jetbrains "dokka" +documentation engine for both Java and Kotlin. Javadoc is useful if you want to write and publish your own libraries.
+To generate Java and Kotlin source documentation, please run "mvn dokka:dokka". You may "cd" to the platform-core +project to try the maven dokka command to generate some source documentation. The home page will be available +in "target/dokka/index.html"
+Writing your functions
+Please follow the step-by-step learning guide in Chapter-1 to write your own functions. You can then +configure new REST endpoints to use your new functions.
+In Chapter-2, we have discussed the three function execution strategies to optimize your application +to the full potential of stability, performance and throughput.
+HTTP forwarding
+In Chapter-3, we have presented the configuration syntax for the "rest.yaml" REST automation
+definition file. Please review the sample rest.yaml file in the lambda-example project. You may notice that
+it has an entry for HTTP forwarding. The following entry in the sample rest.yaml file illustrates an HTTP
+forwarding endpoint. In HTTP forwarding, you can replace the "service" route name with a direct HTTP target host.
+You can do "URL rewrite" to change the URL path to the target endpoint path. In the below example,
+/api/v1/* will be mapped to /api/* in the target endpoint.
- service: "http://127.0.0.1:${rest.server.port}"
+ trust_all_cert: true
+ methods: ['GET', 'PUT', 'POST']
+ url: "/api/v1/*"
+ url_rewrite: ['/api/v1', '/api']
+ timeout: 20
+ cors: cors_1
+ headers: header_1
+ tracing: true
+Sending HTTP request event to more than one service
+One feature in REST automation "rest.yaml" configuration is that you can configure more than one function in the +"service" section. In the following example, there are two function route names ("hello.world" and "hello.copy"). +The first one "hello.world" is the primary service provider. The second one "hello.copy" will receive a copy of +the incoming event automatically.
+This feature allows you to write new version of a function without disruption to current functionality. Once you are +happy with the new version of function, you can route the endpoint directly to the new version by updating the +"rest.yaml" configuration file.
+ - service: ["hello.world", "hello.copy"]
+Writing your first unit test
+Please refer to "rpcTest" method in the "HelloWorldTest" class in the lambda-example to get started.
+In unit test, we want to start the main application so that all the functions are ready for tests.
+First, we write a "TestBase" class to use the BeforeClass setup method to start the main application like this:
+public class TestBase {
+
+ private static final AtomicInteger seq = new AtomicInteger(0);
+
+ @BeforeClass
+ public static void setup() {
+ if (seq.incrementAndGet() == 1) {
+ AutoStart.main(new String[0]);
+ }
+ }
+}
+The atomic integer "seq" is used to ensure the main application entry point is executed only once.
+Your first unit test may look like this:
+@SuppressWarnings("unchecked")
+@Test
+public void rpcTest() throws IOException, InterruptedException {
+ Utility util = Utility.getInstance();
+ BlockingQueue<EventEnvelope> bench = new ArrayBlockingQueue<>(1);
+ String name = "hello";
+ String address = "world";
+ String telephone = "123-456-7890";
+ DemoPoJo pojo = new DemoPoJo(name, address, telephone);
+ PostOffice po = new PostOffice("unit.test", "12345", "POST /api/hello/world");
+ EventEnvelope request = new EventEnvelope().setTo("hello.world")
+ .setHeader("a", "b").setBody(pojo.toMap());
+ po.asyncRequest(request, 800).onSuccess(bench::add);
+ EventEnvelope response = bench.poll(10, TimeUnit.SECONDS);
+ assert response != null;
+ assertEquals(HashMap.class, response.getBody().getClass());
+ MultiLevelMap map = new MultiLevelMap((Map<String, Object>) response.getBody());
+ assertEquals("b", map.getElement("headers.a"));
+ assertEquals(name, map.getElement("body.name"));
+ assertEquals(address, map.getElement("body.address"));
+ assertEquals(telephone, map.getElement("body.telephone"));
+ assertEquals(util.date2str(pojo.time), map.getElement("body.time"));
+}
+Note that the PostOffice instance can be created with tracing information in a Unit Test. The above example +tells the system that the sender is "unit.test", the trace ID is 12345 and the trace path is "POST /api/hello/world".
+For unit test, we need to convert the asynchronous code into "synchronous" execution so that unit test can run +sequentially. "BlockingQueue" is a good choice for this.
+The "hello.world" is an echo function. The above unit test sends an event containing a key-value {"a":"b"} and
+the payload of a HashMap from the DemoPoJo.
If the function is designed to handle PoJo, we can send PoJo directly instead of a Map.
+++IMPORTANT: blocking code should only be used for unit tests. DO NOT use blocking code in your + application code because it will block the event system and dramatically slow down + your application.
+
Convenient utility classes
+The Utility and MultiLevelMap classes are convenient tools for unit tests. In the above example, we use the +Utility class to convert a date object into a UTC timestamp. It is because date object is serialized as a UTC +timestamp in an event.
+The MultiLevelMap supports reading an element using the convenient "dot and bracket" format.
+For example, given a map like this:
+{
+ "body":
+ {
+ "time": "2023-03-27T18:10:34.234Z",
+ "hello": [1, 2, 3]
+ }
+}
+| Example | +Command | +Result | +
|---|---|---|
| 1 | +map.getElement("body.time") | +2023-03-27T18:10:34.234Z | +
| 2 | +map.getElement("body.hello[2]") | +3 | +
The second unit test
+Let's do a unit test for PoJo. In this second unit test, it sends a RPC request to the "hello.pojo" function that +is designed to return a SamplePoJo object with some mock data.
+Please refer to "pojoRpcTest" method in the "PoJoTest" class in the lambda-example for details.
+The unit test verifies that the "hello.pojo" has correctly returned the SamplePoJo object with the pre-defined +mock value.
+@Test
+public void pojoTest() throws IOException, InterruptedException {
+ Integer ID = 1;
+ String NAME = "Simple PoJo class";
+ String ADDRESS = "100 World Blvd, Planet Earth";
+ BlockingQueue<EventEnvelope> bench = new ArrayBlockingQueue<>(1);
+ PostOffice po = new PostOffice("unit.test", "20001", "GET /api/hello/pojo");
+ EventEnvelope request = new EventEnvelope().setTo("hello.pojo").setHeader("id", "1");
+ po.asyncRequest(request, 800).onSuccess(bench::add);
+ EventEnvelope response = bench.poll(10, TimeUnit.SECONDS);
+ assert response != null;
+ assertEquals(SamplePoJo.class, response.getBody().getClass());
+ SamplePoJo pojo = response.getBody(SamplePoJo.class);
+ assertEquals(ID, pojo.getId());
+ assertEquals(NAME, pojo.getName());
+ assertEquals(ADDRESS, pojo.getAddress());
+}
+Note that you can do class "casting" or use the built-in casting API as shown below:
+++SamplePoJo pojo = (SamplePoJo) response.getBody()
+SamplePoJo pojo = response.getBody(SamplePoJo.class)
+
The third unit test
+Testing Kotlin suspend functions is challenging. However, testing suspend function using events is straight forward +because of loose coupling.
+Let's do a unit test for the lambda-example's FileUploadDemo function. Its route name is "hello.upload".
+Please refer to "uploadTest" method in the "SuspendFunctionTest" class in the lambda-example for details.
+@SuppressWarnings("unchecked")
+@Test
+public void uploadTest() throws IOException, InterruptedException {
+ String FILENAME = "unit-test-data.txt";
+ BlockingQueue<EventEnvelope> bench = new ArrayBlockingQueue<>(1);
+ Utility util = Utility.getInstance();
+ String traceId = Utility.getInstance().getUuid();
+ PostOffice po = new PostOffice("unit.test", traceId, "/stream/upload/test");
+ int len = 0;
+ ByteArrayOutputStream bytes = new ByteArrayOutputStream();
+ EventPublisher publisher = new EventPublisher(10000);
+ for (int i=0; i < 10; i++) {
+ String line = "hello world "+i+"\n";
+ byte[] d = util.getUTF(line);
+ publisher.publish(d);
+ bytes.write(d);
+ len += d.length;
+ }
+ publisher.publishCompletion();
+ // emulate a multipart file upload
+ AsyncHttpRequest req = new AsyncHttpRequest();
+ req.setMethod("POST");
+ req.setUrl("/api/upload/demo");
+ req.setTargetHost("http://127.0.0.1:8080");
+ req.setHeader("accept", "application/json");
+ req.setHeader("content-type", "multipart/form-data");
+ req.setContentLength(len);
+ req.setFileName(FILENAME);
+ req.setStreamRoute(publisher.getStreamId());
+ // send the HTTP request event to the "hello.upload" function
+ EventEnvelope request = new EventEnvelope().setTo("hello.upload")
+ .setBody(req).setTrace("12345", "/api/upload/demo").setFrom("unit.test");
+ po.asyncRequest(request, 8000).onSuccess(bench::add);
+ EventEnvelope response = bench.poll(10, TimeUnit.SECONDS);
+ assert response != null;
+ assertEquals(HashMap.class, response.getBody().getClass());
+ Map<String, Object> map = (Map<String, Object>) response.getBody();
+ System.out.println(response.getBody());
+ assertEquals(len, map.get("expected_size"));
+ assertEquals(len, map.get("actual_size"));
+ assertEquals(FILENAME, map.get("filename"));
+ assertEquals("Upload completed", map.get("message"));
+ // finally check that "hello.upload" has saved the test file
+ File dir = new File("/tmp/upload-download-demo");
+ File file = new File(dir, FILENAME);
+ assertTrue(file.exists());
+ assertEquals(len, file.length());
+ // compare file content
+ byte[] b = Utility.getInstance().file2bytes(file);
+ assertArrayEquals(bytes.toByteArray(), b);
+}
+In the above unit test, we use the ObjectStreamIO to emulate a file stream and write 10 blocks of data into it. +The unit test then makes an RPC call to the "hello.upload" with the emulated HTTP request event.
+The "hello.upload" is a Kotlin suspend function. It will be executed when the event arrives. +After saving the test file, it will return an HTTP response object that the unit test can validate.
+In this fashion, you can create unit tests to test suspend functions in an event-driven manner.
+Deployment
+The pom.xml is pre-configured to generate an executable JAR. The following is extracted from the pom.xml.
+The main class is AutoStart that will load the "main application" and use it as the entry point to
+run the application.
<plugin>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-maven-plugin</artifactId>
+ <configuration>
+ <mainClass>org.platformlambda.core.system.AutoStart</mainClass>
+ </configuration>
+ <executions>
+ <execution>
+ <id>build-info</id>
+ <goals>
+ <goal>build-info</goal>
+ </goals>
+ </execution>
+ </executions>
+</plugin>
+Composable application is designed to be deployable using Kubernetes or serverless.
+A sample Dockerfile for an executable JAR may look like this:
+FROM mcr.microsoft.com/openjdk/jdk:21-ubuntu
+EXPOSE 8083
+WORKDIR /app
+COPY target/rest-spring-3-example-3.1.2.jar .
+ENTRYPOINT ["java","-jar","rest-spring-3-example-3.1.2.jar"]
+Distributed tracing
+The system has a built-in distributed tracing feature. You can enable tracing for any REST endpoint by adding +"tracing=true" in the endpoint definition in the "rest.yaml" configuration file.
+You may also upload performance metrics from the distributed tracing data to your favorite telemetry system dashboard.
+To do that, please implement a custom metrics function with the route name distributed.trace.forwarder.
The input to the function will be a HashMap like this:
+trace={path=/api/upload/demo, service=hello.upload, success=true,
+ origin=2023032731e2a5eeae8f4da09f3d9ac6b55fb0a4,
+ exec_time=77.462, start=2023-03-27T19:38:30.061Z,
+ from=http.request, id=12345, round_trip=132.296, status=200}
+The system will detect if distributed.trace.forwarder is available. If yes, it will forward performance metrics
+from distributed trace to your custom function.
Request-response journaling
+Optionally, you may also implement a custom audit function named transaction.journal.recorder to monitor
+request-response payloads.
To enable journaling, please add this to the application.properties file.
+journal.yaml=classpath:/journal.yaml
+and add the "journal.yaml" configuration file to the project's resources folder with content like this:
+journal:
+ - "my.test.function"
+ - "another.function"
+In the above example, the "my.test.function" and "another.function" will be monitored and their request-response +payloads will be forwarded to your custom audit function. The input to your audit function will be a HashMap +containing the performance metrics data and a "journal" section with the request and response payloads in clear form.
+++IMPORTANT: journaling may contain sensitive personally identifiable data and secrets. Please check + security compliance before storing them into access restricted audit data store.
+
| Chapter-4 | +Home | +Chapter-6 | +
|---|---|---|
| Event Script Syntax | +Table of Contents | +Spring Boot | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-6/index.html b/docs/guides/CHAPTER-6/index.html
new file mode 100644
index 00000000..d547d572
--- /dev/null
+++ b/docs/guides/CHAPTER-6/index.html
@@ -0,0 +1,393 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Spring Boot Integration
+While the platform-core foundation code includes a lightweight non-blocking HTTP server, you can also turn your +application into an executable Spring Boot application.
+There are two ways to do that:
+-
+
- Add dependency for Spring Boot version 3.2.1 and implement your Spring Boot main application +
- Add the
rest-spring-3add-on library for a pre-configured Spring Boot experience
+
Add platform-core to an existing Spring Boot application
+For option 1, the platform-core library can co-exist with Spring Boot. You can write code specific to Spring Boot +and the Spring framework ecosystem. Please make sure you add the following startup code to your Spring Boot +main application like this:
+@SpringBootApplication
+public class MyMainApp extends SpringBootServletInitializer {
+
+ public static void main(String[] args) {
+ AutoStart.main(args);
+ SpringApplication.run(MyMainApp.class, args);
+ }
+
+}
+We suggest running AutoStart.main before the SpringApplication.run statement. This would allow the platform-core
+foundation code to load the event-listener functions into memory before Spring Boot starts.
Use the rest-spring library in your application
+You can add the rest-spring-3 library in your application and turn it into a pre-configured
+Spring Boot 3 application.
The "rest-spring" library configures Spring Boot's serializers (XML and JSON) to behave consistently as the +built-in lightweight non-blocking HTTP server.
+If you want to disable the lightweight HTTP server, you can set rest.automation=false in application.properties.
+The REST automation engine and the lightweight HTTP server will be turned off.
++IMPORTANT: the platform-core library assumes the application configuration files to be either + application.yml or application.properties. If you use custom Spring profile, please keep the + application.yml or application.properties for the platform-core. If you use default Spring + profile, both platform-core and Spring Boot will use the same configuration files.
+
You can customize your error page using the default errorPage.html by copying it from the platform-core's or
+rest-spring's resources folder to your source project. The default page is shown below.
This is the HTML error page that the platform-core or rest-spring library uses. You can update it with +your corporate style guide. Please keep the parameters (status, message, path, warning) intact.
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <title>HTTP Error</title>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+</head>
+<body>
+
+<div>
+ <h3>HTTP-${status}</h3>
+ <div>${warning}</div><br/>
+ <table>
+ <tbody>
+ <tr><td style="font-style: italic; width: 100px">Type</td><td>error</td></tr>
+ <tr><td style="font-style: italic; width: 100px">Status</td><td>${status}</td></tr>
+ <tr><td style="font-style: italic; width: 100px">Message</td><td>${message}</td></tr>
+ <tr><td style="font-style: italic; width: 100px">Path</td><td>${path}</td></tr>
+ </tbody>
+ </table>
+
+</div>
+</body>
+</html>
+If you want to keep REST automation's lightweight HTTP server together with Spring Boot's Tomcat or other +application server, please add the following to your application.properties file:
+server.port=8083
+rest.server.port=8085
+rest.automation=true
+The platform-core and Spring Boot will use rest.server.port and server.port respectively.
The rest-spring-3-example demo application
+Let's review the rest-spring-3-example demo application in the "examples/rest-spring-3-example" project.
You can use the rest-spring-3-example as a template to create a Spring Boot application.
+In addition to the REST automation engine that let you create REST endpoints by configuration, you can also +programmatically create REST endpoints with the following approaches:
+-
+
- Spring RestControllers with Mono/Flux +
- Servlet 3.1 WebServlets +
We will examine asynchronous REST endpoint with the AsyncHelloWorld class.
@RestController
+public class AsyncHelloWorld {
+ private static final AtomicInteger seq = new AtomicInteger(0);
+
+ @GetMapping(value = "/api/hello/world", produces={"application/json", "application/xml"})
+ public Mono<Map<String, Object>> hello(HttpServletRequest request) {
+ String traceId = Utility.getInstance().getUuid();
+ PostOffice po = new PostOffice("hello.world.endpoint", traceId, "GET /api/hello/world");
+ Map<String, Object> forward = new HashMap<>();
+
+ Enumeration<String> headers = request.getHeaderNames();
+ while (headers.hasMoreElements()) {
+ String key = headers.nextElement();
+ forward.put(key, request.getHeader(key));
+ }
+ // As a demo, just put the incoming HTTP headers as a payload and a parameter showing the sequence counter.
+ // The echo service will return both.
+ int n = seq.incrementAndGet();
+ EventEnvelope req = new EventEnvelope();
+ req.setTo("hello.world").setBody(forward).setHeader("seq", n);
+ return Mono.create(callback -> {
+ try {
+ po.asyncRequest(req, 3000)
+ .onSuccess(event -> {
+ Map<String, Object> result = new HashMap<>();
+ result.put("status", event.getStatus());
+ result.put("headers", event.getHeaders());
+ result.put("body", event.getBody());
+ result.put("execution_time", event.getExecutionTime());
+ result.put("round_trip", event.getRoundTrip());
+ callback.success(result);
+ })
+ .onFailure(ex -> callback.error(new AppException(408, ex.getMessage())));
+ } catch (IOException e) {
+ callback.error(e);
+ }
+ });
+ }
+}
+In this hello world REST endpoint, Spring Reactor runs the "hello" method asynchronously without waiting for a response.
+The example code copies the HTTP requests and sends it as the request payload to the "hello.world" function. +The function is defined in the MainApp like this:
+Platform platform = Platform.getInstance();
+LambdaFunction echo = (headers, input, instance) -> {
+ Map<String, Object> result = new HashMap<>();
+ result.put("headers", headers);
+ result.put("body", input);
+ result.put("instance", instance);
+ result.put("origin", platform.getOrigin());
+ return result;
+};
+platform.register("hello.world", echo, 20);
+When "hello.world" responds, its result set will be returned to the onSuccess method as a "future response".
The "onSuccess" method then sends the response to the browser using the JAX-RS resume mechanism.
+The AsyncHelloConcurrent is the same as the AsyncHelloWorld except that it performs a "fork-n-join" operation
+to multiple instances of the "hello.world" function.
Unlike "rest.yaml" that defines tracing by configuration, you can turn on tracing programmatically in a JAX-RS +endpoint. To enable tracing, the function sets the trace ID and path in the PostOffice constructor.
+When you try the endpoint at http://127.0.0.1:8083/api/hello/world, it will echo your HTTP request headers. +In the command terminal, you will see tracing information in the console log like this:
+DistributedTrace:67 - trace={path=GET /api/hello/world, service=hello.world, success=true,
+ origin=20230403364f70ebeb54477f91986289dfcd7b75, exec_time=0.249, start=2023-04-03T04:42:43.445Z,
+ from=hello.world.endpoint, id=e12e871096ba4938b871ee72ef09aa0a, round_trip=20.018, status=200}
+Lightweight non-blocking websocket server
+If you want to turn on a non-blocking websocket server, you can add the following configuration to +application.properties.
+server.port=8083
+websocket.server.port=8085
+The above assumes Spring Boot runs on port 8083 and the websocket server runs on port 8085.
+++Note that "websocket.server.port" is an alias of "rest.server.port"
+
You can create a websocket service with a Java class like this:
+@WebSocketService("hello")
+public class WsEchoDemo implements LambdaFunction {
+
+ @Override
+ public Object handleEvent(Map<String, String> headers, Object body, int instance) {
+ // handle the incoming websocket events (type = open, close, bytes or string)
+ }
+}
+The above creates a websocket service at the URL "/ws/hello" server endpoint.
+Please review the example code in the WsEchoDemo class in the rest-spring-2-example project for details.
+If you want to use Spring Boot's Tomcat websocket server, you can disable the non-blocking websocket server feature
+by removing the websocket.server.port configuration and any websocket service classes with the WebSocketService
+annotation.
To try out the demo websocket server, visit http://127.0.0.1:8083 and select "Websocket demo".
+Spring Boot version 3
+The rest-spring-3 subproject is a pre-configured Spring Boot 3 library.
In "rest-spring-3", Spring WebFlux replaces JAX-RS as the asynchronous HTTP servlet engine.
+| Chapter-5 | +Home | +Chapter-7 | +
|---|---|---|
| Build, Test and Deploy | +Table of Contents | +Event over HTTP | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-7/index.html b/docs/guides/CHAPTER-7/index.html
new file mode 100644
index 00000000..8a08d2f0
--- /dev/null
+++ b/docs/guides/CHAPTER-7/index.html
@@ -0,0 +1,411 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Event over HTTP
+The in-memory event system allows functions to communicate with each other in the same application memory space.
+In composable architecture, applications are modular components in a network. Some transactions may require +the services of more than one application. "Event over HTTP" extends the event system beyond a single application.
+The Event API service (event.api.service) is a built-in function in the system.
The Event API endpoint
+To enable "Event over HTTP", you must first turn on the REST automation engine with the following parameters +in the application.properties file:
+rest.server.port=8085
+rest.automation=true
+and then check if the following entry is configured in the "rest.yaml" endpoint definition file. +If not, update "rest.yaml" accordingly. The "timeout" value is set to 60 seconds to fit common use cases.
+ - service: [ "event.api.service" ]
+ methods: [ 'POST' ]
+ url: "/api/event"
+ timeout: 60s
+ tracing: true
+This will expose the Event API endpoint at port 8085 and URL "/api/event".
+In kubernetes, The Event API endpoint of each application is reachable through internal DNS and there is no need +to create "ingress" for this purpose.
+Test drive Event API
+You may now test drive the Event API service.
+First, build and run the lambda-example application in port 8085.
+cd examples/lambda-example
+java -jar target/lambda-example-3.1.2.jar
+Second, build and run the rest-spring-example application.
+cd examples/rest-spring-example-3
+java -jar target/rest-spring-3-example-3.1.2.jar
+The rest-spring-3-example application will run as a Spring Boot application in port 8083 and 8086.
+These two applications will start independently.
+You may point your browser to http://127.0.0.1:8083/api/pojo/http/1 to invoke the HelloPojoEventOverHttp
+endpoint service that will in turn makes an Event API call to the lambda-example's "hello.pojo" service.
You will see the following response in the browser. This means the rest-spring-example application has successfully +made an event API call to the lambda-example application using the Event API endpoint.
+{
+ "id": 1,
+ "name": "Simple PoJo class",
+ "address": "100 World Blvd, Planet Earth",
+ "date": "2023-03-27T23:17:19.257Z",
+ "instance": 6,
+ "seq": 66,
+ "origin": "2023032791b6938a47614cf48779b1cf02fc89c4"
+}
+To examine how the application makes the Event API call, please refer to the HelloPojoEventOverHttp class
+in the rest-spring-example. The class is extracted below:
@RestController
+public class HelloPoJoEventOverHttp {
+
+ @GetMapping("/api/pojo/http/{id}")
+ public Mono<SamplePoJo> getPoJo(@PathVariable("id") Integer id) {
+ AppConfigReader config = AppConfigReader.getInstance();
+ String remotePort = config.getProperty("lambda.example.port", "8085");
+ String remoteEndpoint = "http://127.0.0.1:"+remotePort+"/api/event";
+ String traceId = Utility.getInstance().getUuid();
+ PostOffice po = new PostOffice("hello.pojo.endpoint", traceId, "GET /api/pojo/http");
+ EventEnvelope req = new EventEnvelope().setTo("hello.pojo").setHeader("id", id);
+ return Mono.create(callback -> {
+ try {
+ EventEnvelope response = po.request(req, 3000, Collections.emptyMap(), remoteEndpoint, true).get();
+ if (response.getBody() instanceof SamplePoJo result) {
+ callback.success(result);
+ } else {
+ callback.error(new AppException(response.getStatus(), response.getError()));
+ }
+ } catch (IOException | ExecutionException | InterruptedException e) {
+ callback.error(e);
+ }
+ });
+ }
+}
+The method signatures of the Event API is shown as follows:
+Asynchronous API (Java)
+// io.vertx.core.Future
+public Future<EventEnvelope> asyncRequest(final EventEnvelope event, long timeout,
+ Map<String, String> headers,
+ String eventEndpoint, boolean rpc) throws IOException;
+Sequential non-blocking API (virtual thread function)
+// java.util.concurrent.Future
+public Future<EventEnvelope> request(final EventEnvelope event, long timeout,
+ Map<String, String> headers,
+ String eventEndpoint, boolean rpc) throws IOException;
+Sequential non-blocking API (Kotlin suspend function)
+suspend fun awaitRequest(request: EventEnvelope?, timeout: Long,
+ headers: Map<String, String>,
+ eventEndpoint: String, rpc: Boolean): EventEnvelope
+}
+Optionally, you may add security headers in the "headers" argument. e.g. the "Authorization" header.
+The eventEndpoint is a fully qualified URL. e.g. http://peer/api/event
The "rpc" boolean value is set to true so that the response from the service of the peer application instance +will be delivered. For drop-n-forget use case, you can set the "rpc" value to false. It will immediately return +an HTTP-202 response.
+Event-over-HTTP using configuration
+While you can call the "Event-over-HTTP" APIs programmatically, it would be more convenient to automate it with a +configuration. This service abstraction means that user applications do not need to know where the target services are.
+You can enable Event-over-HTTP configuration by adding this parameter in application.properties:
+#
+# Optional event-over-http target maps
+#
+yaml.event.over.http=classpath:/event-over-http.yaml
+and then create the configuration file "event-over-http.yaml" like this:
+event:
+ http:
+ - route: 'hello.pojo2'
+ target: 'http://127.0.0.1:${lambda.example.port}/api/event'
+ - route: 'event.http.test'
+ target: 'http://127.0.0.1:${server.port}/api/event'
+ # optional security headers
+ headers:
+ authorization: 'demo'
+ - route: 'event.save.get'
+ target: 'http://127.0.0.1:${server.port}/api/event'
+ headers:
+ authorization: 'demo'
+In the above example, there are three routes (hello.pojo2, event.http.test and event.save.get) with target URLs. +If additional authentication is required for the peer's "/api/event" endpoint, you may add a set of security +headers in each route.
+When you send asynchronous event or make a RPC call to "event.save.get" service, it will be forwarded to the
+peer's "event-over-HTTP" endpoint (/api/event) accordingly.
You may also add variable references to the application.properties (or application.yaml) file, such as +"server.port" in this example.
+An example in the rest-spring-3-example subproject is shown below to illustrate this service abstraction. +In this example, the remote Event-over-HTTP endpoint address is resolved from the event-over-http.yaml +configuration.
+@RestController
+public class HelloPoJoEventOverHttpByConfig {
+
+ @GetMapping("/api/pojo2/http/{id}")
+ public Mono<SamplePoJo> getPoJo(@PathVariable("id") Integer id) {
+ String traceId = Utility.getInstance().getUuid();
+ PostOffice po = new PostOffice("hello.pojo.endpoint", traceId, "GET /api/pojo2/http");
+ /*
+ * "hello.pojo2" resides in the lambda-example and is reachable by "Event-over-HTTP".
+ * In HelloPojoEventOverHttp.java, it demonstrates the use of Event-over-HTTP API.
+ * In this example, it illustrates the use of the "Event-over-HTTP by configuration" feature.
+ * Please see application.properties and event-over-http.yaml files for more details.
+ */
+ EventEnvelope req = new EventEnvelope().setTo("hello.pojo2").setHeader("id", id);
+ return Mono.create(callback -> {
+ try {
+ EventEnvelope response = po.request(req, 3000, false).get();
+ if (response.getBody() instanceof SamplePoJo result) {
+ callback.success(result);
+ } else {
+ callback.error(new AppException(response.getStatus(), response.getError()));
+ }
+ } catch (IOException | ExecutionException | InterruptedException e) {
+ callback.error(e);
+ }
+ });
+ }
+}
+++Note: The configuration based "event-over-HTTP" feature does not support fork-n-join request API. + You can achieve similar parallel processing using multiple calls to "po.request API" + where each call returns a Java "Future".
+
Advantages
+The Event API exposes all public functions of an application instance to the network using a single REST endpoint.
+The advantages of Event API includes:
+-
+
- Convenient - you do not need to write or configure individual endpoint for each public service +
- Efficient - events are transported in binary format from one application to another +
- Secure - you can protect the Event API endpoint with an authentication service +
The following configuration adds authentication service to the Event API endpoint:
+ - service: [ "event.api.service" ]
+ methods: [ 'POST' ]
+ url: "/api/event"
+ timeout: 60s
+ authentication: "v1.api.auth"
+ tracing: true
+This enforces every incoming request to the Event API endpoint to be authenticated by the "v1.api.auth" service +before passing to the Event API service. You can plug in your own authentication service such as OAuth 2.0 +"bearer token" validation.
+Please refer to Chapter-3 - REST automation for details.
+
| Chapter-6 | +Home | +Chapter-8 | +
|---|---|---|
| Spring Boot | +Table of Contents | +Minimalist Service Mesh | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-8/index.html b/docs/guides/CHAPTER-8/index.html
new file mode 100644
index 00000000..7f35bdd2
--- /dev/null
+++ b/docs/guides/CHAPTER-8/index.html
@@ -0,0 +1,483 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Minimalist Service Mesh
+Service mesh is a dedicated infrastructure layer to facilitate inter-container communication using "sidecar" and +"control plane".
+Service mesh systems require additional administrative containers (PODs) for "control plane" and "service discovery."
+The additional infrastructure requirements vary among products.
+Using kafka as a minimalist service mesh
+We will discuss using Kafka as a minimalist service mesh.
+++Note: Service mesh is optional. You can use "event over HTTP" for inter-container + communication if service mesh is not suitable.
+
Typically, a service mesh system uses a "side-car" to sit next to the application container in the same POD to provide +service discovery and network proxy services.
+Instead of using a side-car proxy, the system maintains a distributed routing table in each application instance. +When a function requests the service of another function which is not in the same memory space, the "cloud.connector" +module will bridge the event to the peer application through a network event system like Kafka.
+As shown in the following table, if "service.1" and "service.2" are in the same memory space of an application, +they will communicate using the in-memory event bus.
+If they are in different applications and the applications are configured with Kafka, the two functions will +communicate via the "cloud.connector" service.
+| In-memory event bus | +Network event stream | +
|---|---|
| "service.1" -> "service.2" | +"service.1" -> "cloud.connector" -> "service.2" | +
The system supports Kafka out of the box. For example, to select kafka, you can configure application.properties like this:
+cloud.connector=kafka
+The "cloud.connector" parameter can be set to "none" or "kafka". +The default parameter of "cloud.connector" is "none". This means the application is not using +any network event system "connector", thus running independently.
+Let's set up a minimalist service mesh with Kafka to see how it works.
+Set up a standalone Kafka server for development
+You need a Kafka cluster as the network event stream system. For development and testing, you can build
+and run a standalone Kafka server like this. Note that the mvn clean package command is optional because
+the executable JAR should be available after the mvn clean install command in Chapter-1.
cd connectors/adapters/kafka/kafka-standalone
+mvn clean package
+java -jar target/kafka-standalone-3.1.2.jar
+The standalone Kafka server will start at port 9092. You may adjust the "server.properties" in the standalone-kafka +project when necessary.
+When the kafka server is started, it will create a temporary directory "/tmp/kafka-logs".
+++The kafka server is designed for development purpose only. The kafka message log store + will be cleared when the server is restarted.
+
Prepare the kafka-presence application
+The "kafka-presence" is a "presence monitor" application. It is a minimalist "control plane" in service mesh +terminology.
+What is a presence monitor? A presence monitor is the control plane that assigns unique "topic" for each +user application instance.
+It monitors the "presence" of each application. If an application fails or stops, the presence monitor will +advertise the event to the rest of the system so that each application container will update its corresponding +distributed routing table, thus bypassing the failed application and its services.
+If an application has more than one container instance deployed, they will work together to share load evenly.
+You will start the presence monitor like this:
+cd connectors/adapters/kafka/kafka-presence
+java -jar target/kafka-presence-3.1.2.jar
+By default, the kafka-connector will run at port 8080. Partial start-up log is shown below:
+AppStarter:344 - Modules loaded in 2,370 ms
+AppStarter:334 - Websocket server running on port-8080
+ServiceLifeCycle:73 - service.monitor, partition 0 ready
+HouseKeeper:72 - Registered monitor (me) 2023032896b12f9de149459f9c8b71ad8b6b49fa
+The presence monitor will use the topic "service.monitor" to connect to the Kafka server and register itself +as a presence monitor.
+Presence monitor is resilient. You can run more than one instance to back up each other. +If you are not using Docker or Kubernetes, you need to change the "server.port" parameter of the second instance +to 8081 so that the two application instances can run in the same laptop.
+Launch the rest-spring-2-example and lambda-example with kafka
+Let's run the rest-spring-2-example (rest-spring-3-example) and lambda-example applications with +Kafka connector turned on.
+For demo purpose, the rest-spring-2-example and lambda-example are pre-configured with "kafka-connector". +If you do not need these libraries, please remove them from the pom.xml built script.
+Since kafka-connector is pre-configured, we can start the two demo applications like this:
+cd examples/rest-spring-2-example
+java -Dcloud.connector=kafka -Dmandatory.health.dependencies=cloud.connector.health
+ -jar target/rest-spring-2-example-3.1.2.jar
+cd examples/lambda-example
+java -Dcloud.connector=kafka -Dmandatory.health.dependencies=cloud.connector.health
+ -jar target/lambda-example-3.1.2.jar
+The above command uses the "-D" parameters to configure the "cloud.connector" and "mandatory.health.dependencies".
+The parameter mandatory.health.dependencies=cloud.connector.health tells the system to turn on the health check
+endpoint for the application.
For the rest-spring-2-example, the start-up log may look like this:
+AppStarter:344 - Modules loaded in 2,825 ms
+PresenceConnector:155 - Connected pc.abb4a4de.in, 127.0.0.1:8080,
+ /ws/presence/202303282583899cf43a49b98f0522492b9ca178
+EventConsumer:160 - Subscribed multiplex.0001.0
+ServiceLifeCycle:73 - multiplex.0001, partition 0 ready
+This means that the rest-spring-2-example has successfully connected to the presence monitor at port 8080. +It has subscribed to the topic "multiplex.0001" partition 0.
+For the lambda-example, the log may look like this:
+AppStarter:344 - Modules loaded in 2,742 m
+PresenceConnector:155 - Connected pc.991a2be0.in, 127.0.0.1:8080,
+ /ws/presence/2023032808d82ebe2c0d4e5aa9ca96b3813bdd25
+EventConsumer:160 - Subscribed multiplex.0001.1
+ServiceLifeCycle:73 - multiplex.0001, partition 1 ready
+ServiceRegistry:242 - Peer 202303282583899cf43a49b98f0522492b9ca178 joins (rest-spring-2-example 3.0.0)
+ServiceRegistry:383 - hello.world (rest-spring-2-example, WEB.202303282583899cf43a49b98f0522492b9ca178) registered
+You notice that the lambda-example has discovered the rest-spring-2-example through Kafka and added the +"hello.world" to the distributed routing table.
+At this point, the rest-spring-2-example will find the lambda-example application as well:
+ServiceRegistry:242 - Peer 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25 joins (lambda-example 3.0.0)
+ServiceRegistry:383 - hello.world (lambda-example,
+ APP.2023032808d82ebe2c0d4e5aa9ca96b3813bdd25) registered
+ServiceRegistry:383 - hello.pojo (lambda-example,
+ APP.2023032808d82ebe2c0d4e5aa9ca96b3813bdd25) registered
+This is real-time service discovery coordinated by the "kafka-presence" monitor application.
+Now you have created a minimalist event-driven service mesh.
+Send an event request from rest-spring-2-example to lambda-example
+In Chapter-7, you have sent a request from the rest-spring-2-example to the lambda-example using +"Event over HTTP" without a service mesh.
+In this section, you can make the same request using service mesh.
+Please point your browser to http://127.0.0.1:8083/api/pojo/mesh/1 +You will see the following response in your browser.
+{
+ "id": 1,
+ "name": "Simple PoJo class",
+ "address": "100 World Blvd, Planet Earth",
+ "date": "2023-03-28T17:53:41.696Z",
+ "instance": 1,
+ "seq": 1,
+ "origin": "2023032808d82ebe2c0d4e5aa9ca96b3813bdd25"
+}
+Presence monitor info endpoint
+You can check the service mesh status from the presence monitor's "/info" endpoint.
+You can visit http://127.0.0.1:8080/info and it will show something like this:
+{
+ "app": {
+ "name": "kafka-presence",
+ "description": "Presence Monitor",
+ "version": "3.0.0"
+ },
+ "personality": "RESOURCES",
+ "additional_info": {
+ "total": {
+ "topics": 2,
+ "virtual_topics": 2,
+ "connections": 2
+ },
+ "topics": [
+ "multiplex.0001 (32)",
+ "service.monitor (11)"
+ ],
+ "virtual_topics": [
+ "multiplex.0001-000 -> 202303282583899cf43a49b98f0522492b9ca178, rest-spring-2-example v3.0.0",
+ "multiplex.0001-001 -> 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25, lambda-example v3.0.0"
+ ],
+ "connections": [
+ {
+ "elapsed": "25 minutes 12 seconds",
+ "created": "2023-03-28T17:43:13Z",
+ "origin": "2023032808d82ebe2c0d4e5aa9ca96b3813bdd25",
+ "name": "lambda-example",
+ "topic": "multiplex.0001-001",
+ "monitor": "2023032896b12f9de149459f9c8b71ad8b6b49fa",
+ "type": "APP",
+ "updated": "2023-03-28T18:08:25Z",
+ "version": "3.0.0",
+ "seq": 65,
+ "group": 1
+ },
+ {
+ "elapsed": "29 minutes 42 seconds",
+ "created": "2023-03-28T17:38:47Z",
+ "origin": "202303282583899cf43a49b98f0522492b9ca178",
+ "name": "rest-spring-2-example",
+ "topic": "multiplex.0001-000",
+ "monitor": "2023032896b12f9de149459f9c8b71ad8b6b49fa",
+ "type": "WEB",
+ "updated": "2023-03-28T18:08:29Z",
+ "version": "3.0.0",
+ "seq": 75,
+ "group": 1
+ }
+ ],
+ "monitors": [
+ "2023032896b12f9de149459f9c8b71ad8b6b49fa - 2023-03-28T18:08:46Z"
+ ]
+ },
+ "vm": {
+ "java_vm_version": "18.0.2.1+1",
+ "java_runtime_version": "18.0.2.1+1",
+ "java_version": "18.0.2.1"
+ },
+ "origin": "2023032896b12f9de149459f9c8b71ad8b6b49fa",
+ "time": {
+ "current": "2023-03-28T18:08:47.613Z",
+ "start": "2023-03-28T17:31:23.611Z"
+ }
+}
+In this example, it shows that there are two user applications (rest-spring-2-example and lambda-example) connected.
+Presence monitor health endpoint
+The presence monitor has a "/health" endpoint.
+You can visit http://127.0.0.1:8080/health and it will show something like this:
+{
+ "dependency": [
+ {
+ "route": "cloud.connector.health",
+ "status_code": 200,
+ "service": "kafka",
+ "topics": "on-demand",
+ "href": "127.0.0.1:9092",
+ "message": "Loopback test took 3 ms; System contains 2 topics",
+ "required": true
+ }
+ ],
+ "origin": "2023032896b12f9de149459f9c8b71ad8b6b49fa",
+ "name": "kafka-presence",
+ "status": "UP"
+}
+User application health endpoint
+Similarly, you can check the health status of the rest-spring-2-example application with http://127.0.0.1:8083/health
+{
+ "dependency": [
+ {
+ "route": "cloud.connector.health",
+ "status_code": 200,
+ "service": "kafka",
+ "topics": "on-demand",
+ "href": "127.0.0.1:9092",
+ "message": "Loopback test took 4 ms",
+ "required": true
+ }
+ ],
+ "origin": "202303282583899cf43a49b98f0522492b9ca178",
+ "name": "rest-spring-example",
+ "status": "UP"
+}
+It looks similar to the health status of the presence monitor. However, only the presence monitor shows the total +number of topics because it handles topic issuance to each user application instance.
+Actuator endpoints
+Additional actuator endpoints includes:
+-
+
- library endpoint ("/info/lib") - you can check the packaged libraries for each application +
- distributed routing table ("/info/routes") - this will display the distributed routing table for public functions +
- environment ("/env") - it shows all functions (public and private) with number of workers. +
- livenessproble ("/livenessprobe") - this should display "OK" to indicate the application is running +
Stop an application
+You can press "control-C" to stop an application. Let's stop the lambda-example application.
+Once you stopped lamdba-example from the command line, the rest-spring-2-example will detect it:
+ServiceRegistry:278 - Peer 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25 left (lambda-example 3.0.0)
+ServiceRegistry:401 - hello.world 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25 unregistered
+ServiceRegistry:401 - hello.pojo 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25 unregistered
+The rest-spring-2-example will update its distributed routing table automatically.
+You will also find log messages in the kafka-presence application like this:
+MonitorService:120 - Member 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25 left
+TopicController:250 - multiplex.0001-001 released by 2023032808d82ebe2c0d4e5aa9ca96b3813bdd25,
+ lambda-example, 3.0.0
+When an application instance stops, the presence monitor will detect the event, remove it from the registry and +release the topic associated with the disconnected application instance.
+The presence monitor is using the "presence" feature in websocket, thus we call it "presence" monitor.
+| Chapter-7 | +Home | +CHAPTER-9 | +
|---|---|---|
| Event over HTTP | +Table of Contents | +API Overview | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/CHAPTER-9/index.html b/docs/guides/CHAPTER-9/index.html
new file mode 100644
index 00000000..c469dff7
--- /dev/null
+++ b/docs/guides/CHAPTER-9/index.html
@@ -0,0 +1,786 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ API Overview
+Main Application
+Each application has an entry point. You may implement an entry point in a main application like this:
+@MainApplication
+public class MainApp implements EntryPoint {
+
+ public static void main(String[] args) {
+ AutoStart.main(args);
+ }
+
+ @Override
+ public void start(String[] args) {
+ // your startup logic here
+ log.info("Started");
+ }
+}
+In your main application, you must implement the EntryPoint interface to override the "start" method.
+Typically, a main application is used to initiate some application start up procedure.
In some case when your application does not need any start up logic, you can just print a message to indicate +that your application has started.
+You may want to keep the static "main" method which can be used to run your application inside an IDE.
+The pom.xml build script is designed to run the AutoStart class that will execute your main application's
+start method.
In some case, your application may have more than one main application module. You can decide the sequence of
+execution using the "sequence" parameter in the MainApplication annotation. The module with the smallest
+sequence number will run first. Duplicated sequence numbers are allowed. Normal startup sequence must be
+between 1 and 999.
Note: It is the "start" method of each EntryPoint implementation that follows the execution sequence of the
+MainApplication annotation. The optional "main" method is used only to kick off the application bootstrap and
+it must include only the following statement:
public static void main(String[] args) {
+ AutoStart.main(args);
+}
+Therefore, even when the default sequence of the MainApplication annotation is 10 and you invoke the "main"
+method from an IDE, the "start" method of each MainApplication modules will execute orderly.
Setup before the Main Application
+Sometimes, it may be required to set up some environment configuration before your main application starts.
+You can implement a BeforeApplication module. Its syntax is similar to the MainApplication.
@BeforeApplication
+public class EnvSetup implements EntryPoint {
+
+ @Override
+ public void start(String[] args) {
+ // your environment setup logic here
+ log.info("initialized");
+ }
+}
+The BeforeApplication logic will run before your MainApplication module(s). This is useful when you want to do
+special handling of environment variables. For example, decrypt an environment variable secret, construct an X.509
+certificate, and save it in the "/tmp" folder before your main application starts.
Normal startup sequence must be between 6 and 999. Sequence 5 is reserved by the AsyncHttpClientLoader. +If your startup code does not need the async HTTP client service and you want it to run first, you may use +sequence from 1 to 4.
+Event envelope
+Mercury is an event engine that encapsulates Eclipse Vertx and Kotlin coroutine and suspend function.
+A composable application is a collection of functions that communicate with each other in events. +Each event is transported by an event envelope. Let's examine the envelope.
+There are 3 elements in an event envelope:
+| Element | +Type | +Purpose | +
|---|---|---|
| 1 | +metadata | +Includes unique ID, target function name, reply address correlation ID, status, exception, trace ID and path |
+
| 2 | +headers | +User defined key-value pairs | +
| 3 | +body | +Event payload (primitive, hash map or PoJo) | +
Headers and body are optional, but you must provide at least one of them. If the envelope do not have any headers +or body, the system will send your event as a "ping" command to the target function. The response acknowledgements +that the target function exists. This ping/pong protocol tests the event loop or service mesh. This test mechanism +is useful for DevSecOps admin dashboard.
+PoJo transport
+Your function can implement the TypedLambdaFunction interface if you want to use PoJo as input and output.
+If you use the EventEnvelope as input, PoJo payload is provided as a HashMap in the event's body.
+The original class name of the PoJo payload is saved in the event's type attribute. +You can compare and restore the PoJo like this:
+if (SamplePoJo.class.getName().equals(input.getType())) {
+ SamplePoJo pojo = input.getBody(SamplePoJo.class);
+ // do something with your input PoJo
+}
+If you use the "untyped" LambdaFunction, the input "Object" is a HashMap and you would need to convert it back +to a PoJo using the SimpleMapper or a serializer of your choice.
+For example,
+SamplePoJo pojo = SimpleMapper.getInstance().getMapper().readValue((Map<String, Object>) input, SamplePoJo.class);
+Custom exception using AppException
+To reject an incoming request, you can throw an AppException like this:
+// example-1
+throw new AppException(400, "My custom error message");
+// example-2
+throw new AppException(400, "My custom error message", ex);
+Example-1 - a simple exception with status code (400) and an error message
+Example-2 - includes a nested exception
+As a best practice, we recommend using error codes that are compatible with HTTP status codes.
+Defining a user function in Java
+You can write a function in Java like this:
+@PreLoad(route = "hello.simple", instances = 10)
+public class SimpleDemoEndpoint implements TypedLambdaFunction<AsyncHttpRequest, Object> {
+ @Override
+ public Object handleEvent(Map<String, String> headers, AsyncHttpRequest input, int instance) {
+ // business logic here
+ return result;
+ }
+}
+The PreLoad annotation tells the system to preload the function into memory and register it into the event loop.
+You must provide a "route name" and configure the number of concurrent workers ("instances").
Route name is used by the event loop to find your function in memory. A route name must use lower letters and numbers, +and it must have at least one dot as a word separator. e.g. "hello.simple" is a proper route name but "HelloSimple" +is not.
+You can implement your function using the LambdaFunction or TypedLambdaFunction. The latter allows you to define +the input and output classes.
+The system will map the event body into the input argument and the event headers into the headers argument.
+The instance argument informs your function which worker is serving the current request.
Similarly, you can also write a "suspend function" in Kotlin like this:
+@PreLoad(route = "hello.world", instances = 10, isPrivate = false,
+ envInstances = "instances.hello.world")
+class HelloWorld : KotlinLambdaFunction<Any?, Map<String, Any>> {
+
+ @Throws(Exception::class)
+ override suspend fun handleEvent(headers: Map<String, String>, input: Any?,
+ instance: Int): Map<String, Any> {
+ // business logic here
+ return result;
+ }
+}
+In the suspend function example above, you may notice the optional envInstances parameter. This tells the system
+to use a parameter from the application.properties (or application.yml) to configure the number of workers for the
+function. When the parameter defined in "envInstances" is not found, the "instances" parameter is used as the
+default value.
Inspect event metadata
+There are some reserved metadata for route name ("my_route"), trace ID ("my_trace_id") and trace path ("my_trace_path") +in the "headers" argument. They do not exist in the incoming event envelope. Instead, the system automatically +insert them as read-only metadata.
+They are used when your code want to obtain an instance of PostOffice or FastRPC.
+To inspect all metadata, you can declare the input as "EventEnvelope". The system will map the whole event envelope +into the "input" argument. You can retrieve the replyTo address and other useful metadata.
+Note that the "replyTo" address is optional. It only exists when the caller is making an RPC call to your function. +If the caller sends an asynchronous request, the "replyTo" value is null.
+Platform API
+You can obtain a singleton instance of the Platform object to do the following:
+Register a function
+We recommend using the PreLoad annotation in a class to declare the function route name, number of worker instances
+and whether the function is public or private.
In some use cases where you want to create and destroy functions on demand, you can register them programmatically.
+In the following example, it registers "my.function" using the MyFunction class as a public function and +"another.function" with the AnotherFunction class as a private function. It then registers two kotlin functions +in public and private scope respectively.
+Platform platform = Platform.getInstance();
+
+// register a public function
+platform.register("my.function", new MyFunction(), 10);
+
+// register a private function
+platform.registerPrivate("another.function", new AnotherFunction(), 20);
+
+// register a public suspend function
+platform.registerKoltin("my.suspend.function", new MySuspendFunction(), 10);
+
+// register a private suspend function
+platform.registerKoltinPrivate("another.suspend.function", new AnotherSuspendFunction(), 10);
+What is a public function?
+A public function is visible by any application instances in the same network. When a function is declared as +"public", the function is reachable through the EventAPI REST endpoint or a service mesh.
+A private function is invisible outside the memory space of the application instance that it resides. +This allows application to encapsulate business logic according to domain boundary. You can assemble closely +related functions as a composable application that can be deployed independently.
+Release a function
+In some use cases, you want to release a function on-demand when it is no longer required.
+platform.release("another.function");
+The above API will unload the function from memory and release it from the "event loop".
+Check if a function is available
+You can check if a function with the named route has been deployed.
+if (platform.hasRoute("another.function")) {
+ // do something
+}
+Wait for a function to be ready
+Functions are registered asynchronously. For functions registered using the PreLoad annotation, they are available
+to your application when the MainApplication starts.
For functions that are registered on-demand, you can wait for the function to get ready like this:
+Future<Boolean> status = platform.waitForProvider("cloud.connector", 10);
+status.onSuccess(ready -> {
+ // business logic when "cloud.connector" is ready
+});
+Note that the "onFailure" method is not required. The onSuccess will return true or false. In the above example, +your application waits for up to 10 seconds. If the function (i.e. the "provider") is available, the API will invoke +the "onSuccess" method immediately.
+Obtain the unique application instance ID
+When an application instance starts, a unique ID is generated. We call this the "Origin ID".
+String originId = po.getOrigin();
+When running the application in a minimalist service mesh using Kafka or similar network event stream system, +the origin ID is used to uniquely identify the application instance.
+The origin ID is automatically appended to the "replyTo" address when making a RPC call over a network event stream +so that the system can send the response event back to the "originator" or "calling" application instance.
+Set application personality
+An application may have one of the following personality:
+-
+
- REST - the deployed application is user facing +
- APP - the deployed application serves business logic +
- RESOURCES - this is a resource-tier service. e.g. database service, MQ gateway, legacy service proxy, utility, etc. +
You can change the application personality like this:
+// the default value is "APP"
+ServerPersonality.getInstance().setType(ServerPersonality.Type.REST);
+The personality setting is for documentation purpose only. It does not affect the behavior of your application. +It will appear in the application "/info" endpoint.
+PostOffice API
+You can obtain an instance of the PostOffice from the input "headers" and "instance" parameters in the input +arguments of your function.
+PostOffice po = new PostOffice(headers, instance);
+The PostOffice is the event manager that you can use to send asynchronous events or to make RPC requests. +The constructor uses the READ only metadata in the "headers" argument in the "handleEvent" method of your function.
+Send an asynchronous event to a function
+You can send an asynchronous event like this.
+// example-1
+po.send("another.function", "test message");
+
+// example-2
+po.send("another.function", new Kv("some_key", "some_value"), new kv("another_key", "another_value"));
+
+// example-3
+po.send("another.function", somePoJo, new Kv("some_key", "some_value"));
+
+// example-4
+EventEnvelope event = new EventEnvelope().setTo("another.function")
+ .setHeader("some_key", "some_value").setBody(somePoJo);
+po.send(event)
+
+// example-5
+po.sendLater(event, new Date(System.currentTimeMillis() + 5000));
+-
+
- Example-1 sends the text string "test message" to the target service named "another.function". +
- Example-2 sends two key-values as "headers" parameters to the same service. +
- Example-3 sends a PoJo and a key-value pair to the same service. +
- Example-4 is the same as example-3. It is using an EventEnvelope to construct the request. +
- Example-5 schedules an event to be sent 5 seconds later. +
The first 3 APIs are convenient methods and the system will automatically create an EventEnvelope to hold the +target route name, key-values and/or event payload.
+Make an asynchronous RPC call
+You can make RPC call like this:
+// example-1
+EventEnvelope request = new EventEnvelope().setTo("another.function")
+ .setHeader("some_key", "some_value").setBody(somePoJo);
+Future<EventEnvelope> response = po.asyncRequest(request, 5000);
+response.onSuccess(result -> {
+ // result is the response event
+});
+response.onFailure(e -> {
+ // handle timeout exception
+});
+
+// example-2
+Future<EventEnvelope> response = po.asyncRequest(request, 5000, false);
+response.onSuccess(result -> {
+ // result is the response event
+ // Timeout exception is returned as a response event with status=408
+});
+
+// example-3 with the "rpc" boolean parameter set to true
+Future<EventEnvelope> response = po.asyncRequest(request, 5000, "http://peer/api/event", true);
+response.onSuccess(result -> {
+ // result is the response event
+});
+response.onFailure(e -> {
+ // handle timeout exception
+});
+-
+
- Example-1 makes a RPC call with a 5-second timeout to "another.function". +
- Example-2 sets the "timeoutException" to false, telling system to return timeout exception as a regular event. +
- Example-3 makes an "event over HTTP" RPC call to "another.function" in another application instance called "peer". +
"Event over HTTP" is an important topic. Please refer to Chapter 7 for more details.
+Perform a fork-n-join RPC call to multiple functions
+In a similar fashion, you can make a fork-n-join call that sends request events in parallel to more than one function.
+// example-1
+EventEnvelope request1 = new EventEnvelope().setTo("this.function")
+ .setHeader("hello", "world").setBody("test message");
+EventEnvelope request2 = new EventEnvelope().setTo("that.function")
+ .setHeader("good", "day").setBody(somePoJo);
+List<EventEnvelope> requests = new ArrayList<>();
+requests.add(request1);
+requests.add(request2);
+Future<List<EventEnvelope>> responses = po.asyncRequest(requests, 5000);
+response.onSuccess(results -> {
+ // results contains the response events
+});
+response.onFailure(e -> {
+ // handle timeout exception
+});
+
+// example-2
+Future<List<EventEnvelope>> responses = po.asyncRequest(requests, 5000, false);
+response.onSuccess(results -> {
+ // results contains the response events.
+ // Partial result list is returned if one or more functions did not respond.
+});
+Make a sequential non-blocking RPC call
+You can make a sequential non-blocking RPC call from one function to another.
+The most convenient method to make a sequential non-blocking RPC call is to use the PostOffice's request API.
+// for a single RPC call
+PostOffice po = new PostOffice(headers, instance);
+EventEnvelope result = po.request(requestEvent, timeoutInMills).get();
+
+// for a fork-n-join call
+PostOffice po = new PostOffice(headers, instance);
+List<EventEnvelope> result = po.request(requestEvents, timeoutInMills).get();
+If you prefer the Kotlin programming language, you may use the FastRPC API.
+It is the event manager for KotlinLambdaFunction. You can create an instance of the FastRPC using the "headers" +parameters in the input arguments of your function.
+val fastRPC = new FastRPC(headers)
+val request = EventEnvelope().setTo("another.function")
+ .setHeader("some_key", "some_value").setBody(somePoJo)
+// example-1
+val response = fastRPC.awaitRequest(request, 5000)
+// handle the response event
+
+// example-2 with the "rpc" boolean parameter set to true
+val response = fastRPC.awaitRequest(request, 5000, "http://peer/api/event", true)
+// handle the response event
+-
+
- Example-1 performs a non-blocking RPC call +
- Example-2 makes a non-blocking "Event Over HTTP" RPC call +
Note that timeout exception is returned as a regular event with status 408.
+Sequential non-blocking code is easier to read. Moreover, it handles more concurrent users and requests +without consuming a lot of CPU resources because it is "suspended" while waiting for a response from another function.
+Perform a sequential non-blocking fork-n-join call to multiple functions
+You can make a sequential non-blocking fork-n-join call using the FastRPC API like this:
+val fastRPC = FastRPC(headers)
+val template = EventEnvelope().setTo("hello.world").setHeader("someKey", "someValue")
+val requests = ArrayList<EventEnvelope>()
+// create a list of 4 request events
+for (i in 0..3) {
+ requests.add(EventEnvelope(template.toBytes()).setBody(i).setCorrelationId("cid-$i"))
+}
+val responses: List<EventEnvelope> = fastRPC.awaitRequest(requests, 5000)
+// handle the response events
+In the above example, the function creates a list of request events from a template event with target service +"hello.world". It sets the number 0 to 3 to the individual events with unique correlation IDs.
+The response events contain the same set of correlation IDs so that your business logic can decide how to +handle individual response event.
+The result may be a partial list of response events if one or more functions failed to respond on time.
+Check if a function with a named route exists
+The PostOffice provides the "exists()" method that is similar to the "platform.hasRoute()" command.
+The difference is that the "exists()" method can discover functions of another application instance when running +in the "service mesh" mode.
+If your application is not deployed in a service mesh, the PostOffice's "exists" and Platform's "hasRoute" APIs +will provide the same result.
+boolean found = po.exists("another.function");
+if (found) {
+ // do something
+}
+Retrieve trace ID and path
+If you want to know the route name and optional trace ID and path, you can use the following APIs.
+For example, if tracing is enabled, the trace ID will be available. You can put the trace ID in application log +messages. This would group log messages of the same transaction together when you search the trace ID from +a centralized logging dashboard such as Splunk.
+String myRoute = po.getRoute();
+String traceId = po.getTraceId();
+String tracePath = po.getTracePath();
+Trace annotation
+You can use the PostOffice instance to annotate a trace in your function like this:
+// annotate a trace with the key-value "hello:world"
+po.annotateTrace("hello", "world");
+This is useful when you want to attach transaction specific information in the performance metrics. +For example, the traces may be used in production transaction analytics.
+++IMPORTANT: do not annotate sensitive or secret information such as PII, PHI, PCI data because + the trace is visible in application log. It may also be forwarded to a centralized + telemetry dashboard.
+
Configuration API
+Your function can access the main application configuration from the platform like this:
+AppConfigReader config = AppConfigReader.getInstance();
+// the value can be string or a primitive
+Object value = config.get("my.parameter");
+// the return value will be converted to a string
+String text = config.getProperty("my.parameter");
+The system uses the standard dot-bracket format for a parameter name. e.g.
+hello.world
+some.key[2]
+You can override the main application configuration at run-time using the Java argument "-D". e.g.
+++java -Dserver.port=8080 -jar myApp.jar
+
Additional configuration files can be added with the ConfigReader API like this:
// filePath should have location prefix "classpath:/" or "file:/"
+ConfigReader reader = new ConfigReader();
+reader.load(filePath);
+The configuration system supports environment variable or reference to the main application configuration
+using the dollar-bracket syntax ${reference:default_value}. e.g.
some.key=${MY_ENV_VARIABLE}
+another.key=${my.key:12345}
+complex.key=first ${FIRST_ENV_VAR}, second ${SECOND_ENV_VAR}
+In the above example, a parameter may contain references to more than one environment variable.
+Default value, if not given, will be assumed to be an empty string.
+Custom serializer
+We are using GSON as the underlying serializer to handle common use cases. However, there may be +situation that you want to use your own custom serialization library.
+To do that, you may write a serializer that implements the CustomSerializer interface:
+public interface CustomSerializer {
+
+ public Map<String, Object> toMap(Object obj);
+
+ public <T> T toPoJo(Object obj, Class<T> toValueType);
+
+}
+You may configure a user function to use a custom serializer by adding the "customSerializer" parameter
+in the PreLoad annotation. For example,
@PreLoad(route="my.user.function", customSerializer = JacksonSerializer.class)
+public class MyUserFunction implements TypedLambdaFunction<SimplePoJo, SimplePoJo> {
+ @Override
+ public SimplePoJo handleEvent(Map<String, String> headers, SimplePoJo input, int instance) {
+ return input;
+ }
+}
+If you register your function dynamically in code, you can use the following platform API to assign
+a custom serializer.
public void setCustomSerializer(String route, CustomSerializer mapper);
+// e.g.
+// platform.setCustomSerializer("my.function", new JacksonSerializer());
+If you use the PostOffice to programmatically send event or make event RPC call and you need +custom serializer, you can create a PostOffice instance like this:
+// this should be the first statement in the "handleEvent" method.
+PostOffice po = new PostOffice(headers, instance, new MyCustomSerializer());
+The outgoing event using the PostOffice will use the custom serializer automatically.
+To interpret an event response from a RPC call, you can use the following PostOffice API:
+MyPoJo result = po.getResponseBodyAsPoJo(responseEvent, MyPoJo.class);
+Minimalist API design
+As a best practice, we advocate a minimalist approach in API integration. +To build powerful composable applications, the above set of APIs is sufficient to perform +"event orchestration" where you write code to coordinate how the various functions work together as a +single "executable". Please refer to Chapter-4 for more details about event orchestration.
+Since Mercury is used in production installations, we will exercise the best effort to keep the core API stable.
+Other APIs in the toolkits are used internally to build the engine itself, and they may change from time to time. +They are mostly convenient methods and utilities. The engine is fully encapsulated and any internal API changes +are not likely to impact your applications.
+Event Scripting
+To further reduce coding effort, you can perform "event choreography" by configuration using "Event Script". +Please refer to Event Script syntax in Chapter 4
+Co-existence with other development frameworks
+Mercury libraries are designed to co-exist with your favorite frameworks and tools. Inside a class implementing
+the LambdaFunction, TypedLambdaFunction or KotlinLambdaFunction, you can use any coding style and frameworks
+as you like, including sequential, object-oriented and reactive programming styles.
The core-engine has a built-in lightweight non-blocking HTTP server, but you can also use Spring Boot and other +application server framework with it.
+A sample Spring Boot integration is provided in the "rest-spring" project. It is an optional feature, and you can +decide to use a regular Spring Boot application with Mercury or to pick the customized Spring Boot in the +"rest-spring" library.
+Template application for quick start
+We recommend using the composable-example project as a template to start writing your Composable applications.
+You can follow the Composable methodology where you draw event flow diagrams to represent various use cases,
+convert them into event scripts that carry out event chorerography for your self-contained functions.
For more information, please refer to Event Script syntax in Chapter 4.
+If you prefer to do low-level event-driven programming, you can use the lambda-example project as a template.
+It is preconfigured to support kernel threads, coroutine and suspend function.
Source code update frequency
+This project is licensed under the Apache 2.0 open sources license. We will update the public codebase after +it passes regression tests and meets stability and performance benchmarks in our production systems.
+Mercury is developed as an engine for you to build the latest cloud native and composable applications. +While we are updating the technology frequently, the essential internals and the core APIs are stable.
+Technical support
+For enterprise clients, optional technical support is available. Please contact your Accenture representative
+for details.
+
| Chapter-8 | +Home | +
|---|---|
| Minimalist Service Mesh | +Table of Contents | +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/guides/TABLE-OF-CONTENTS/index.html b/docs/guides/TABLE-OF-CONTENTS/index.html
new file mode 100644
index 00000000..071e8873
--- /dev/null
+++ b/docs/guides/TABLE-OF-CONTENTS/index.html
@@ -0,0 +1,204 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Developer's Guide
+Mercury Composable is a software development toolkit for writing composable applications.
+ +Chapter 2 - Function Execution Strategies
+ +Chapter 4 - Event Script Syntax
+Chapter 5 - Build, Test and Deploy
+ + +Chapter 8 - Minimalist Service Mesh
+ +Appendix I - application.properties
+ + + +
+
+
+
+ « Previous
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/img/favicon.ico b/docs/img/favicon.ico
new file mode 100644
index 00000000..e85006a3
Binary files /dev/null and b/docs/img/favicon.ico differ
diff --git a/docs/index.html b/docs/index.html
new file mode 100644
index 00000000..59a7229c
--- /dev/null
+++ b/docs/index.html
@@ -0,0 +1,257 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Mercury Composable
+Since version 4.0, we have merged our enterprise extension ("Event Script") with the Mercury v3.1 foundation +codebase. It is a comprehensive toolkit to write composable applications including microservices and +serverless. This technology was filed under US Patent application 18/459,307. The source code is provided +as is under the Apache 2.0 license.
+August 2024
+Getting Started
+A composable application is designed in 3 steps:
+-
+
- Describe your use case as an event flow diagram +
- Create a configuration file to represent the event flow +
- Write a user story for each user function +
To get started, please visit Chapter 1, Developer Guide.
+We will illustrate the methodology with a composable application example.
+Conquer Complexity: Embrace Composable Design
+Introduction
+Software development is an ongoing battle against complexity. Over time, codebases can become tangled and unwieldy, +hindering innovation and maintenance. This article introduces composable design patterns, a powerful approach to +build applications that are modular, maintainable, and scalable.
+The Perils of Spaghetti Code
+We have all encountered it: code that resembles a plate of spaghetti – tangled dependencies, hidden logic, +and a general sense of dread when approaching modifications. These codebases are difficult to test, debug, +and update. Composable design patterns offer a solution.
+Evolution of Design Patterns
+Software development methodologies have evolved alongside hardware advancements. In the early days, developers +prized efficiency, writing code from scratch due to limited libraries. The rise of frameworks brought structure +and boilerplate code, but also introduced potential rigidity.
+Functional Programming and Event-Driven Architecture
+Functional programming, with its emphasis on pure functions and immutable data, paved the way for composable design. +This approach encourages building applications as chains of well-defined functions, each with a clear input and output.
+Event-driven architecture complements this approach by using events to trigger functions. This loose coupling +promotes modularity and scalability.
+The Power of Composable Design
+At its core, composable design emphasizes two principles:
+-
+
- Self-Contained Functions: Each function is a well-defined unit, handling its own logic and transformations + with minimal dependencies. +
- Event Choreography: Functions communicate through events, allowing for loose coupling and independent + execution. +
Benefits of Composable Design
+-
+
- Enhanced Maintainability: Isolated functions are easier to understand, test, and modify. +
- Improved Reusability: Self-contained functions can be easily reused across different parts of your application. +
- Superior Performance: Loose coupling reduces bottlenecks and encourages asynchronous execution. +
- Streamlined Testing: Well-defined functions facilitate unit testing and isolate potential issues. +
- Simplified Debugging: Independent functions make it easier to pinpoint the source of errors. +
- Technology Agnostic: You may use your preferred frameworks and tools to write composable code, + allowing for easier future adaptations. +
Implementing Composable Design
+While seemingly simple, implementing composable design can involve some initial complexity.
+Here's a breakdown of the approach:
+-
+
- Function Design: Each function serves a specific purpose, with clearly defined inputs and outputs. +
- Event Communication: Functions communicate through well-defined events, avoiding direct dependencies. +
- Choreography: An event manager, with a state machine and event flow configuration, sequences and triggers functions + based on events. +
Conclusion
+Composable design patterns offer a powerful paradigm for building maintainable, scalable, and future-proof applications. +By embracing the principles of self-contained functions and event-driven communication, you can conquer complexity and +write code that is a joy to work with.
+Are you ready to take your development practices to the next level? Embrace composable design now!
+ +
+
+
+
+
+ Next »
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/js/html5shiv.min.js b/docs/js/html5shiv.min.js
new file mode 100644
index 00000000..1a01c94b
--- /dev/null
+++ b/docs/js/html5shiv.min.js
@@ -0,0 +1,4 @@
+/**
+* @preserve HTML5 Shiv 3.7.3 | @afarkas @jdalton @jon_neal @rem | MIT/GPL2 Licensed
+*/
+!function(a,b){function c(a,b){var c=a.createElement("p"),d=a.getElementsByTagName("head")[0]||a.documentElement;return c.innerHTML="x",d.insertBefore(c.lastChild,d.firstChild)}function d(){var a=t.elements;return"string"==typeof a?a.split(" "):a}function e(a,b){var c=t.elements;"string"!=typeof c&&(c=c.join(" ")),"string"!=typeof a&&(a=a.join(" ")),t.elements=c+" "+a,j(b)}function f(a){var b=s[a[q]];return b||(b={},r++,a[q]=r,s[r]=b),b}function g(a,c,d){if(c||(c=b),l)return c.createElement(a);d||(d=f(c));var e;return e=d.cache[a]?d.cache[a].cloneNode():p.test(a)?(d.cache[a]=d.createElem(a)).cloneNode():d.createElem(a),!e.canHaveChildren||o.test(a)||e.tagUrn?e:d.frag.appendChild(e)}function h(a,c){if(a||(a=b),l)return a.createDocumentFragment();c=c||f(a);for(var e=c.frag.cloneNode(),g=0,h=d(),i=h.length;i>g;g++)e.createElement(h[g]);return e}function i(a,b){b.cache||(b.cache={},b.createElem=a.createElement,b.createFrag=a.createDocumentFragment,b.frag=b.createFrag()),a.createElement=function(c){return t.shivMethods?g(c,a,b):b.createElem(c)},a.createDocumentFragment=Function("h,f","return function(){var n=f.cloneNode(),c=n.createElement;h.shivMethods&&("+d().join().replace(/[\w\-:]+/g,function(a){return b.createElem(a),b.frag.createElement(a),'c("'+a+'")'})+");return n}")(t,b.frag)}function j(a){a||(a=b);var d=f(a);return!t.shivCSS||k||d.hasCSS||(d.hasCSS=!!c(a,"article,aside,dialog,figcaption,figure,footer,header,hgroup,main,nav,section{display:block}mark{background:#FF0;color:#000}template{display:none}")),l||i(a,d),a}var k,l,m="3.7.3",n=a.html5||{},o=/^<|^(?:button|map|select|textarea|object|iframe|option|optgroup)$/i,p=/^(?:a|b|code|div|fieldset|h1|h2|h3|h4|h5|h6|i|label|li|ol|p|q|span|strong|style|table|tbody|td|th|tr|ul)$/i,q="_html5shiv",r=0,s={};!function(){try{var a=b.createElement("a");a.innerHTML="
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Search Results
+ + + +
+ Searching...
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/search/lunr.js b/docs/search/lunr.js
new file mode 100644
index 00000000..aca0a167
--- /dev/null
+++ b/docs/search/lunr.js
@@ -0,0 +1,3475 @@
+/**
+ * lunr - http://lunrjs.com - A bit like Solr, but much smaller and not as bright - 2.3.9
+ * Copyright (C) 2020 Oliver Nightingale
+ * @license MIT
+ */
+
+;(function(){
+
+/**
+ * A convenience function for configuring and constructing
+ * a new lunr Index.
+ *
+ * A lunr.Builder instance is created and the pipeline setup
+ * with a trimmer, stop word filter and stemmer.
+ *
+ * This builder object is yielded to the configuration function
+ * that is passed as a parameter, allowing the list of fields
+ * and other builder parameters to be customised.
+ *
+ * All documents _must_ be added within the passed config function.
+ *
+ * @example
+ * var idx = lunr(function () {
+ * this.field('title')
+ * this.field('body')
+ * this.ref('id')
+ *
+ * documents.forEach(function (doc) {
+ * this.add(doc)
+ * }, this)
+ * })
+ *
+ * @see {@link lunr.Builder}
+ * @see {@link lunr.Pipeline}
+ * @see {@link lunr.trimmer}
+ * @see {@link lunr.stopWordFilter}
+ * @see {@link lunr.stemmer}
+ * @namespace {function} lunr
+ */
+var lunr = function (config) {
+ var builder = new lunr.Builder
+
+ builder.pipeline.add(
+ lunr.trimmer,
+ lunr.stopWordFilter,
+ lunr.stemmer
+ )
+
+ builder.searchPipeline.add(
+ lunr.stemmer
+ )
+
+ config.call(builder, builder)
+ return builder.build()
+}
+
+lunr.version = "2.3.9"
+/*!
+ * lunr.utils
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * A namespace containing utils for the rest of the lunr library
+ * @namespace lunr.utils
+ */
+lunr.utils = {}
+
+/**
+ * Print a warning message to the console.
+ *
+ * @param {String} message The message to be printed.
+ * @memberOf lunr.utils
+ * @function
+ */
+lunr.utils.warn = (function (global) {
+ /* eslint-disable no-console */
+ return function (message) {
+ if (global.console && console.warn) {
+ console.warn(message)
+ }
+ }
+ /* eslint-enable no-console */
+})(this)
+
+/**
+ * Convert an object to a string.
+ *
+ * In the case of `null` and `undefined` the function returns
+ * the empty string, in all other cases the result of calling
+ * `toString` on the passed object is returned.
+ *
+ * @param {Any} obj The object to convert to a string.
+ * @return {String} string representation of the passed object.
+ * @memberOf lunr.utils
+ */
+lunr.utils.asString = function (obj) {
+ if (obj === void 0 || obj === null) {
+ return ""
+ } else {
+ return obj.toString()
+ }
+}
+
+/**
+ * Clones an object.
+ *
+ * Will create a copy of an existing object such that any mutations
+ * on the copy cannot affect the original.
+ *
+ * Only shallow objects are supported, passing a nested object to this
+ * function will cause a TypeError.
+ *
+ * Objects with primitives, and arrays of primitives are supported.
+ *
+ * @param {Object} obj The object to clone.
+ * @return {Object} a clone of the passed object.
+ * @throws {TypeError} when a nested object is passed.
+ * @memberOf Utils
+ */
+lunr.utils.clone = function (obj) {
+ if (obj === null || obj === undefined) {
+ return obj
+ }
+
+ var clone = Object.create(null),
+ keys = Object.keys(obj)
+
+ for (var i = 0; i < keys.length; i++) {
+ var key = keys[i],
+ val = obj[key]
+
+ if (Array.isArray(val)) {
+ clone[key] = val.slice()
+ continue
+ }
+
+ if (typeof val === 'string' ||
+ typeof val === 'number' ||
+ typeof val === 'boolean') {
+ clone[key] = val
+ continue
+ }
+
+ throw new TypeError("clone is not deep and does not support nested objects")
+ }
+
+ return clone
+}
+lunr.FieldRef = function (docRef, fieldName, stringValue) {
+ this.docRef = docRef
+ this.fieldName = fieldName
+ this._stringValue = stringValue
+}
+
+lunr.FieldRef.joiner = "/"
+
+lunr.FieldRef.fromString = function (s) {
+ var n = s.indexOf(lunr.FieldRef.joiner)
+
+ if (n === -1) {
+ throw "malformed field ref string"
+ }
+
+ var fieldRef = s.slice(0, n),
+ docRef = s.slice(n + 1)
+
+ return new lunr.FieldRef (docRef, fieldRef, s)
+}
+
+lunr.FieldRef.prototype.toString = function () {
+ if (this._stringValue == undefined) {
+ this._stringValue = this.fieldName + lunr.FieldRef.joiner + this.docRef
+ }
+
+ return this._stringValue
+}
+/*!
+ * lunr.Set
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * A lunr set.
+ *
+ * @constructor
+ */
+lunr.Set = function (elements) {
+ this.elements = Object.create(null)
+
+ if (elements) {
+ this.length = elements.length
+
+ for (var i = 0; i < this.length; i++) {
+ this.elements[elements[i]] = true
+ }
+ } else {
+ this.length = 0
+ }
+}
+
+/**
+ * A complete set that contains all elements.
+ *
+ * @static
+ * @readonly
+ * @type {lunr.Set}
+ */
+lunr.Set.complete = {
+ intersect: function (other) {
+ return other
+ },
+
+ union: function () {
+ return this
+ },
+
+ contains: function () {
+ return true
+ }
+}
+
+/**
+ * An empty set that contains no elements.
+ *
+ * @static
+ * @readonly
+ * @type {lunr.Set}
+ */
+lunr.Set.empty = {
+ intersect: function () {
+ return this
+ },
+
+ union: function (other) {
+ return other
+ },
+
+ contains: function () {
+ return false
+ }
+}
+
+/**
+ * Returns true if this set contains the specified object.
+ *
+ * @param {object} object - Object whose presence in this set is to be tested.
+ * @returns {boolean} - True if this set contains the specified object.
+ */
+lunr.Set.prototype.contains = function (object) {
+ return !!this.elements[object]
+}
+
+/**
+ * Returns a new set containing only the elements that are present in both
+ * this set and the specified set.
+ *
+ * @param {lunr.Set} other - set to intersect with this set.
+ * @returns {lunr.Set} a new set that is the intersection of this and the specified set.
+ */
+
+lunr.Set.prototype.intersect = function (other) {
+ var a, b, elements, intersection = []
+
+ if (other === lunr.Set.complete) {
+ return this
+ }
+
+ if (other === lunr.Set.empty) {
+ return other
+ }
+
+ if (this.length < other.length) {
+ a = this
+ b = other
+ } else {
+ a = other
+ b = this
+ }
+
+ elements = Object.keys(a.elements)
+
+ for (var i = 0; i < elements.length; i++) {
+ var element = elements[i]
+ if (element in b.elements) {
+ intersection.push(element)
+ }
+ }
+
+ return new lunr.Set (intersection)
+}
+
+/**
+ * Returns a new set combining the elements of this and the specified set.
+ *
+ * @param {lunr.Set} other - set to union with this set.
+ * @return {lunr.Set} a new set that is the union of this and the specified set.
+ */
+
+lunr.Set.prototype.union = function (other) {
+ if (other === lunr.Set.complete) {
+ return lunr.Set.complete
+ }
+
+ if (other === lunr.Set.empty) {
+ return this
+ }
+
+ return new lunr.Set(Object.keys(this.elements).concat(Object.keys(other.elements)))
+}
+/**
+ * A function to calculate the inverse document frequency for
+ * a posting. This is shared between the builder and the index
+ *
+ * @private
+ * @param {object} posting - The posting for a given term
+ * @param {number} documentCount - The total number of documents.
+ */
+lunr.idf = function (posting, documentCount) {
+ var documentsWithTerm = 0
+
+ for (var fieldName in posting) {
+ if (fieldName == '_index') continue // Ignore the term index, its not a field
+ documentsWithTerm += Object.keys(posting[fieldName]).length
+ }
+
+ var x = (documentCount - documentsWithTerm + 0.5) / (documentsWithTerm + 0.5)
+
+ return Math.log(1 + Math.abs(x))
+}
+
+/**
+ * A token wraps a string representation of a token
+ * as it is passed through the text processing pipeline.
+ *
+ * @constructor
+ * @param {string} [str=''] - The string token being wrapped.
+ * @param {object} [metadata={}] - Metadata associated with this token.
+ */
+lunr.Token = function (str, metadata) {
+ this.str = str || ""
+ this.metadata = metadata || {}
+}
+
+/**
+ * Returns the token string that is being wrapped by this object.
+ *
+ * @returns {string}
+ */
+lunr.Token.prototype.toString = function () {
+ return this.str
+}
+
+/**
+ * A token update function is used when updating or optionally
+ * when cloning a token.
+ *
+ * @callback lunr.Token~updateFunction
+ * @param {string} str - The string representation of the token.
+ * @param {Object} metadata - All metadata associated with this token.
+ */
+
+/**
+ * Applies the given function to the wrapped string token.
+ *
+ * @example
+ * token.update(function (str, metadata) {
+ * return str.toUpperCase()
+ * })
+ *
+ * @param {lunr.Token~updateFunction} fn - A function to apply to the token string.
+ * @returns {lunr.Token}
+ */
+lunr.Token.prototype.update = function (fn) {
+ this.str = fn(this.str, this.metadata)
+ return this
+}
+
+/**
+ * Creates a clone of this token. Optionally a function can be
+ * applied to the cloned token.
+ *
+ * @param {lunr.Token~updateFunction} [fn] - An optional function to apply to the cloned token.
+ * @returns {lunr.Token}
+ */
+lunr.Token.prototype.clone = function (fn) {
+ fn = fn || function (s) { return s }
+ return new lunr.Token (fn(this.str, this.metadata), this.metadata)
+}
+/*!
+ * lunr.tokenizer
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * A function for splitting a string into tokens ready to be inserted into
+ * the search index. Uses `lunr.tokenizer.separator` to split strings, change
+ * the value of this property to change how strings are split into tokens.
+ *
+ * This tokenizer will convert its parameter to a string by calling `toString` and
+ * then will split this string on the character in `lunr.tokenizer.separator`.
+ * Arrays will have their elements converted to strings and wrapped in a lunr.Token.
+ *
+ * Optional metadata can be passed to the tokenizer, this metadata will be cloned and
+ * added as metadata to every token that is created from the object to be tokenized.
+ *
+ * @static
+ * @param {?(string|object|object[])} obj - The object to convert into tokens
+ * @param {?object} metadata - Optional metadata to associate with every token
+ * @returns {lunr.Token[]}
+ * @see {@link lunr.Pipeline}
+ */
+lunr.tokenizer = function (obj, metadata) {
+ if (obj == null || obj == undefined) {
+ return []
+ }
+
+ if (Array.isArray(obj)) {
+ return obj.map(function (t) {
+ return new lunr.Token(
+ lunr.utils.asString(t).toLowerCase(),
+ lunr.utils.clone(metadata)
+ )
+ })
+ }
+
+ var str = obj.toString().toLowerCase(),
+ len = str.length,
+ tokens = []
+
+ for (var sliceEnd = 0, sliceStart = 0; sliceEnd <= len; sliceEnd++) {
+ var char = str.charAt(sliceEnd),
+ sliceLength = sliceEnd - sliceStart
+
+ if ((char.match(lunr.tokenizer.separator) || sliceEnd == len)) {
+
+ if (sliceLength > 0) {
+ var tokenMetadata = lunr.utils.clone(metadata) || {}
+ tokenMetadata["position"] = [sliceStart, sliceLength]
+ tokenMetadata["index"] = tokens.length
+
+ tokens.push(
+ new lunr.Token (
+ str.slice(sliceStart, sliceEnd),
+ tokenMetadata
+ )
+ )
+ }
+
+ sliceStart = sliceEnd + 1
+ }
+
+ }
+
+ return tokens
+}
+
+/**
+ * The separator used to split a string into tokens. Override this property to change the behaviour of
+ * `lunr.tokenizer` behaviour when tokenizing strings. By default this splits on whitespace and hyphens.
+ *
+ * @static
+ * @see lunr.tokenizer
+ */
+lunr.tokenizer.separator = /[\s\-]+/
+/*!
+ * lunr.Pipeline
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * lunr.Pipelines maintain an ordered list of functions to be applied to all
+ * tokens in documents entering the search index and queries being ran against
+ * the index.
+ *
+ * An instance of lunr.Index created with the lunr shortcut will contain a
+ * pipeline with a stop word filter and an English language stemmer. Extra
+ * functions can be added before or after either of these functions or these
+ * default functions can be removed.
+ *
+ * When run the pipeline will call each function in turn, passing a token, the
+ * index of that token in the original list of all tokens and finally a list of
+ * all the original tokens.
+ *
+ * The output of functions in the pipeline will be passed to the next function
+ * in the pipeline. To exclude a token from entering the index the function
+ * should return undefined, the rest of the pipeline will not be called with
+ * this token.
+ *
+ * For serialisation of pipelines to work, all functions used in an instance of
+ * a pipeline should be registered with lunr.Pipeline. Registered functions can
+ * then be loaded. If trying to load a serialised pipeline that uses functions
+ * that are not registered an error will be thrown.
+ *
+ * If not planning on serialising the pipeline then registering pipeline functions
+ * is not necessary.
+ *
+ * @constructor
+ */
+lunr.Pipeline = function () {
+ this._stack = []
+}
+
+lunr.Pipeline.registeredFunctions = Object.create(null)
+
+/**
+ * A pipeline function maps lunr.Token to lunr.Token. A lunr.Token contains the token
+ * string as well as all known metadata. A pipeline function can mutate the token string
+ * or mutate (or add) metadata for a given token.
+ *
+ * A pipeline function can indicate that the passed token should be discarded by returning
+ * null, undefined or an empty string. This token will not be passed to any downstream pipeline
+ * functions and will not be added to the index.
+ *
+ * Multiple tokens can be returned by returning an array of tokens. Each token will be passed
+ * to any downstream pipeline functions and all will returned tokens will be added to the index.
+ *
+ * Any number of pipeline functions may be chained together using a lunr.Pipeline.
+ *
+ * @interface lunr.PipelineFunction
+ * @param {lunr.Token} token - A token from the document being processed.
+ * @param {number} i - The index of this token in the complete list of tokens for this document/field.
+ * @param {lunr.Token[]} tokens - All tokens for this document/field.
+ * @returns {(?lunr.Token|lunr.Token[])}
+ */
+
+/**
+ * Register a function with the pipeline.
+ *
+ * Functions that are used in the pipeline should be registered if the pipeline
+ * needs to be serialised, or a serialised pipeline needs to be loaded.
+ *
+ * Registering a function does not add it to a pipeline, functions must still be
+ * added to instances of the pipeline for them to be used when running a pipeline.
+ *
+ * @param {lunr.PipelineFunction} fn - The function to check for.
+ * @param {String} label - The label to register this function with
+ */
+lunr.Pipeline.registerFunction = function (fn, label) {
+ if (label in this.registeredFunctions) {
+ lunr.utils.warn('Overwriting existing registered function: ' + label)
+ }
+
+ fn.label = label
+ lunr.Pipeline.registeredFunctions[fn.label] = fn
+}
+
+/**
+ * Warns if the function is not registered as a Pipeline function.
+ *
+ * @param {lunr.PipelineFunction} fn - The function to check for.
+ * @private
+ */
+lunr.Pipeline.warnIfFunctionNotRegistered = function (fn) {
+ var isRegistered = fn.label && (fn.label in this.registeredFunctions)
+
+ if (!isRegistered) {
+ lunr.utils.warn('Function is not registered with pipeline. This may cause problems when serialising the index.\n', fn)
+ }
+}
+
+/**
+ * Loads a previously serialised pipeline.
+ *
+ * All functions to be loaded must already be registered with lunr.Pipeline.
+ * If any function from the serialised data has not been registered then an
+ * error will be thrown.
+ *
+ * @param {Object} serialised - The serialised pipeline to load.
+ * @returns {lunr.Pipeline}
+ */
+lunr.Pipeline.load = function (serialised) {
+ var pipeline = new lunr.Pipeline
+
+ serialised.forEach(function (fnName) {
+ var fn = lunr.Pipeline.registeredFunctions[fnName]
+
+ if (fn) {
+ pipeline.add(fn)
+ } else {
+ throw new Error('Cannot load unregistered function: ' + fnName)
+ }
+ })
+
+ return pipeline
+}
+
+/**
+ * Adds new functions to the end of the pipeline.
+ *
+ * Logs a warning if the function has not been registered.
+ *
+ * @param {lunr.PipelineFunction[]} functions - Any number of functions to add to the pipeline.
+ */
+lunr.Pipeline.prototype.add = function () {
+ var fns = Array.prototype.slice.call(arguments)
+
+ fns.forEach(function (fn) {
+ lunr.Pipeline.warnIfFunctionNotRegistered(fn)
+ this._stack.push(fn)
+ }, this)
+}
+
+/**
+ * Adds a single function after a function that already exists in the
+ * pipeline.
+ *
+ * Logs a warning if the function has not been registered.
+ *
+ * @param {lunr.PipelineFunction} existingFn - A function that already exists in the pipeline.
+ * @param {lunr.PipelineFunction} newFn - The new function to add to the pipeline.
+ */
+lunr.Pipeline.prototype.after = function (existingFn, newFn) {
+ lunr.Pipeline.warnIfFunctionNotRegistered(newFn)
+
+ var pos = this._stack.indexOf(existingFn)
+ if (pos == -1) {
+ throw new Error('Cannot find existingFn')
+ }
+
+ pos = pos + 1
+ this._stack.splice(pos, 0, newFn)
+}
+
+/**
+ * Adds a single function before a function that already exists in the
+ * pipeline.
+ *
+ * Logs a warning if the function has not been registered.
+ *
+ * @param {lunr.PipelineFunction} existingFn - A function that already exists in the pipeline.
+ * @param {lunr.PipelineFunction} newFn - The new function to add to the pipeline.
+ */
+lunr.Pipeline.prototype.before = function (existingFn, newFn) {
+ lunr.Pipeline.warnIfFunctionNotRegistered(newFn)
+
+ var pos = this._stack.indexOf(existingFn)
+ if (pos == -1) {
+ throw new Error('Cannot find existingFn')
+ }
+
+ this._stack.splice(pos, 0, newFn)
+}
+
+/**
+ * Removes a function from the pipeline.
+ *
+ * @param {lunr.PipelineFunction} fn The function to remove from the pipeline.
+ */
+lunr.Pipeline.prototype.remove = function (fn) {
+ var pos = this._stack.indexOf(fn)
+ if (pos == -1) {
+ return
+ }
+
+ this._stack.splice(pos, 1)
+}
+
+/**
+ * Runs the current list of functions that make up the pipeline against the
+ * passed tokens.
+ *
+ * @param {Array} tokens The tokens to run through the pipeline.
+ * @returns {Array}
+ */
+lunr.Pipeline.prototype.run = function (tokens) {
+ var stackLength = this._stack.length
+
+ for (var i = 0; i < stackLength; i++) {
+ var fn = this._stack[i]
+ var memo = []
+
+ for (var j = 0; j < tokens.length; j++) {
+ var result = fn(tokens[j], j, tokens)
+
+ if (result === null || result === void 0 || result === '') continue
+
+ if (Array.isArray(result)) {
+ for (var k = 0; k < result.length; k++) {
+ memo.push(result[k])
+ }
+ } else {
+ memo.push(result)
+ }
+ }
+
+ tokens = memo
+ }
+
+ return tokens
+}
+
+/**
+ * Convenience method for passing a string through a pipeline and getting
+ * strings out. This method takes care of wrapping the passed string in a
+ * token and mapping the resulting tokens back to strings.
+ *
+ * @param {string} str - The string to pass through the pipeline.
+ * @param {?object} metadata - Optional metadata to associate with the token
+ * passed to the pipeline.
+ * @returns {string[]}
+ */
+lunr.Pipeline.prototype.runString = function (str, metadata) {
+ var token = new lunr.Token (str, metadata)
+
+ return this.run([token]).map(function (t) {
+ return t.toString()
+ })
+}
+
+/**
+ * Resets the pipeline by removing any existing processors.
+ *
+ */
+lunr.Pipeline.prototype.reset = function () {
+ this._stack = []
+}
+
+/**
+ * Returns a representation of the pipeline ready for serialisation.
+ *
+ * Logs a warning if the function has not been registered.
+ *
+ * @returns {Array}
+ */
+lunr.Pipeline.prototype.toJSON = function () {
+ return this._stack.map(function (fn) {
+ lunr.Pipeline.warnIfFunctionNotRegistered(fn)
+
+ return fn.label
+ })
+}
+/*!
+ * lunr.Vector
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * A vector is used to construct the vector space of documents and queries. These
+ * vectors support operations to determine the similarity between two documents or
+ * a document and a query.
+ *
+ * Normally no parameters are required for initializing a vector, but in the case of
+ * loading a previously dumped vector the raw elements can be provided to the constructor.
+ *
+ * For performance reasons vectors are implemented with a flat array, where an elements
+ * index is immediately followed by its value. E.g. [index, value, index, value]. This
+ * allows the underlying array to be as sparse as possible and still offer decent
+ * performance when being used for vector calculations.
+ *
+ * @constructor
+ * @param {Number[]} [elements] - The flat list of element index and element value pairs.
+ */
+lunr.Vector = function (elements) {
+ this._magnitude = 0
+ this.elements = elements || []
+}
+
+
+/**
+ * Calculates the position within the vector to insert a given index.
+ *
+ * This is used internally by insert and upsert. If there are duplicate indexes then
+ * the position is returned as if the value for that index were to be updated, but it
+ * is the callers responsibility to check whether there is a duplicate at that index
+ *
+ * @param {Number} insertIdx - The index at which the element should be inserted.
+ * @returns {Number}
+ */
+lunr.Vector.prototype.positionForIndex = function (index) {
+ // For an empty vector the tuple can be inserted at the beginning
+ if (this.elements.length == 0) {
+ return 0
+ }
+
+ var start = 0,
+ end = this.elements.length / 2,
+ sliceLength = end - start,
+ pivotPoint = Math.floor(sliceLength / 2),
+ pivotIndex = this.elements[pivotPoint * 2]
+
+ while (sliceLength > 1) {
+ if (pivotIndex < index) {
+ start = pivotPoint
+ }
+
+ if (pivotIndex > index) {
+ end = pivotPoint
+ }
+
+ if (pivotIndex == index) {
+ break
+ }
+
+ sliceLength = end - start
+ pivotPoint = start + Math.floor(sliceLength / 2)
+ pivotIndex = this.elements[pivotPoint * 2]
+ }
+
+ if (pivotIndex == index) {
+ return pivotPoint * 2
+ }
+
+ if (pivotIndex > index) {
+ return pivotPoint * 2
+ }
+
+ if (pivotIndex < index) {
+ return (pivotPoint + 1) * 2
+ }
+}
+
+/**
+ * Inserts an element at an index within the vector.
+ *
+ * Does not allow duplicates, will throw an error if there is already an entry
+ * for this index.
+ *
+ * @param {Number} insertIdx - The index at which the element should be inserted.
+ * @param {Number} val - The value to be inserted into the vector.
+ */
+lunr.Vector.prototype.insert = function (insertIdx, val) {
+ this.upsert(insertIdx, val, function () {
+ throw "duplicate index"
+ })
+}
+
+/**
+ * Inserts or updates an existing index within the vector.
+ *
+ * @param {Number} insertIdx - The index at which the element should be inserted.
+ * @param {Number} val - The value to be inserted into the vector.
+ * @param {function} fn - A function that is called for updates, the existing value and the
+ * requested value are passed as arguments
+ */
+lunr.Vector.prototype.upsert = function (insertIdx, val, fn) {
+ this._magnitude = 0
+ var position = this.positionForIndex(insertIdx)
+
+ if (this.elements[position] == insertIdx) {
+ this.elements[position + 1] = fn(this.elements[position + 1], val)
+ } else {
+ this.elements.splice(position, 0, insertIdx, val)
+ }
+}
+
+/**
+ * Calculates the magnitude of this vector.
+ *
+ * @returns {Number}
+ */
+lunr.Vector.prototype.magnitude = function () {
+ if (this._magnitude) return this._magnitude
+
+ var sumOfSquares = 0,
+ elementsLength = this.elements.length
+
+ for (var i = 1; i < elementsLength; i += 2) {
+ var val = this.elements[i]
+ sumOfSquares += val * val
+ }
+
+ return this._magnitude = Math.sqrt(sumOfSquares)
+}
+
+/**
+ * Calculates the dot product of this vector and another vector.
+ *
+ * @param {lunr.Vector} otherVector - The vector to compute the dot product with.
+ * @returns {Number}
+ */
+lunr.Vector.prototype.dot = function (otherVector) {
+ var dotProduct = 0,
+ a = this.elements, b = otherVector.elements,
+ aLen = a.length, bLen = b.length,
+ aVal = 0, bVal = 0,
+ i = 0, j = 0
+
+ while (i < aLen && j < bLen) {
+ aVal = a[i], bVal = b[j]
+ if (aVal < bVal) {
+ i += 2
+ } else if (aVal > bVal) {
+ j += 2
+ } else if (aVal == bVal) {
+ dotProduct += a[i + 1] * b[j + 1]
+ i += 2
+ j += 2
+ }
+ }
+
+ return dotProduct
+}
+
+/**
+ * Calculates the similarity between this vector and another vector.
+ *
+ * @param {lunr.Vector} otherVector - The other vector to calculate the
+ * similarity with.
+ * @returns {Number}
+ */
+lunr.Vector.prototype.similarity = function (otherVector) {
+ return this.dot(otherVector) / this.magnitude() || 0
+}
+
+/**
+ * Converts the vector to an array of the elements within the vector.
+ *
+ * @returns {Number[]}
+ */
+lunr.Vector.prototype.toArray = function () {
+ var output = new Array (this.elements.length / 2)
+
+ for (var i = 1, j = 0; i < this.elements.length; i += 2, j++) {
+ output[j] = this.elements[i]
+ }
+
+ return output
+}
+
+/**
+ * A JSON serializable representation of the vector.
+ *
+ * @returns {Number[]}
+ */
+lunr.Vector.prototype.toJSON = function () {
+ return this.elements
+}
+/* eslint-disable */
+/*!
+ * lunr.stemmer
+ * Copyright (C) 2020 Oliver Nightingale
+ * Includes code from - http://tartarus.org/~martin/PorterStemmer/js.txt
+ */
+
+/**
+ * lunr.stemmer is an english language stemmer, this is a JavaScript
+ * implementation of the PorterStemmer taken from http://tartarus.org/~martin
+ *
+ * @static
+ * @implements {lunr.PipelineFunction}
+ * @param {lunr.Token} token - The string to stem
+ * @returns {lunr.Token}
+ * @see {@link lunr.Pipeline}
+ * @function
+ */
+lunr.stemmer = (function(){
+ var step2list = {
+ "ational" : "ate",
+ "tional" : "tion",
+ "enci" : "ence",
+ "anci" : "ance",
+ "izer" : "ize",
+ "bli" : "ble",
+ "alli" : "al",
+ "entli" : "ent",
+ "eli" : "e",
+ "ousli" : "ous",
+ "ization" : "ize",
+ "ation" : "ate",
+ "ator" : "ate",
+ "alism" : "al",
+ "iveness" : "ive",
+ "fulness" : "ful",
+ "ousness" : "ous",
+ "aliti" : "al",
+ "iviti" : "ive",
+ "biliti" : "ble",
+ "logi" : "log"
+ },
+
+ step3list = {
+ "icate" : "ic",
+ "ative" : "",
+ "alize" : "al",
+ "iciti" : "ic",
+ "ical" : "ic",
+ "ful" : "",
+ "ness" : ""
+ },
+
+ c = "[^aeiou]", // consonant
+ v = "[aeiouy]", // vowel
+ C = c + "[^aeiouy]*", // consonant sequence
+ V = v + "[aeiou]*", // vowel sequence
+
+ mgr0 = "^(" + C + ")?" + V + C, // [C]VC... is m>0
+ meq1 = "^(" + C + ")?" + V + C + "(" + V + ")?$", // [C]VC[V] is m=1
+ mgr1 = "^(" + C + ")?" + V + C + V + C, // [C]VCVC... is m>1
+ s_v = "^(" + C + ")?" + v; // vowel in stem
+
+ var re_mgr0 = new RegExp(mgr0);
+ var re_mgr1 = new RegExp(mgr1);
+ var re_meq1 = new RegExp(meq1);
+ var re_s_v = new RegExp(s_v);
+
+ var re_1a = /^(.+?)(ss|i)es$/;
+ var re2_1a = /^(.+?)([^s])s$/;
+ var re_1b = /^(.+?)eed$/;
+ var re2_1b = /^(.+?)(ed|ing)$/;
+ var re_1b_2 = /.$/;
+ var re2_1b_2 = /(at|bl|iz)$/;
+ var re3_1b_2 = new RegExp("([^aeiouylsz])\\1$");
+ var re4_1b_2 = new RegExp("^" + C + v + "[^aeiouwxy]$");
+
+ var re_1c = /^(.+?[^aeiou])y$/;
+ var re_2 = /^(.+?)(ational|tional|enci|anci|izer|bli|alli|entli|eli|ousli|ization|ation|ator|alism|iveness|fulness|ousness|aliti|iviti|biliti|logi)$/;
+
+ var re_3 = /^(.+?)(icate|ative|alize|iciti|ical|ful|ness)$/;
+
+ var re_4 = /^(.+?)(al|ance|ence|er|ic|able|ible|ant|ement|ment|ent|ou|ism|ate|iti|ous|ive|ize)$/;
+ var re2_4 = /^(.+?)(s|t)(ion)$/;
+
+ var re_5 = /^(.+?)e$/;
+ var re_5_1 = /ll$/;
+ var re3_5 = new RegExp("^" + C + v + "[^aeiouwxy]$");
+
+ var porterStemmer = function porterStemmer(w) {
+ var stem,
+ suffix,
+ firstch,
+ re,
+ re2,
+ re3,
+ re4;
+
+ if (w.length < 3) { return w; }
+
+ firstch = w.substr(0,1);
+ if (firstch == "y") {
+ w = firstch.toUpperCase() + w.substr(1);
+ }
+
+ // Step 1a
+ re = re_1a
+ re2 = re2_1a;
+
+ if (re.test(w)) { w = w.replace(re,"$1$2"); }
+ else if (re2.test(w)) { w = w.replace(re2,"$1$2"); }
+
+ // Step 1b
+ re = re_1b;
+ re2 = re2_1b;
+ if (re.test(w)) {
+ var fp = re.exec(w);
+ re = re_mgr0;
+ if (re.test(fp[1])) {
+ re = re_1b_2;
+ w = w.replace(re,"");
+ }
+ } else if (re2.test(w)) {
+ var fp = re2.exec(w);
+ stem = fp[1];
+ re2 = re_s_v;
+ if (re2.test(stem)) {
+ w = stem;
+ re2 = re2_1b_2;
+ re3 = re3_1b_2;
+ re4 = re4_1b_2;
+ if (re2.test(w)) { w = w + "e"; }
+ else if (re3.test(w)) { re = re_1b_2; w = w.replace(re,""); }
+ else if (re4.test(w)) { w = w + "e"; }
+ }
+ }
+
+ // Step 1c - replace suffix y or Y by i if preceded by a non-vowel which is not the first letter of the word (so cry -> cri, by -> by, say -> say)
+ re = re_1c;
+ if (re.test(w)) {
+ var fp = re.exec(w);
+ stem = fp[1];
+ w = stem + "i";
+ }
+
+ // Step 2
+ re = re_2;
+ if (re.test(w)) {
+ var fp = re.exec(w);
+ stem = fp[1];
+ suffix = fp[2];
+ re = re_mgr0;
+ if (re.test(stem)) {
+ w = stem + step2list[suffix];
+ }
+ }
+
+ // Step 3
+ re = re_3;
+ if (re.test(w)) {
+ var fp = re.exec(w);
+ stem = fp[1];
+ suffix = fp[2];
+ re = re_mgr0;
+ if (re.test(stem)) {
+ w = stem + step3list[suffix];
+ }
+ }
+
+ // Step 4
+ re = re_4;
+ re2 = re2_4;
+ if (re.test(w)) {
+ var fp = re.exec(w);
+ stem = fp[1];
+ re = re_mgr1;
+ if (re.test(stem)) {
+ w = stem;
+ }
+ } else if (re2.test(w)) {
+ var fp = re2.exec(w);
+ stem = fp[1] + fp[2];
+ re2 = re_mgr1;
+ if (re2.test(stem)) {
+ w = stem;
+ }
+ }
+
+ // Step 5
+ re = re_5;
+ if (re.test(w)) {
+ var fp = re.exec(w);
+ stem = fp[1];
+ re = re_mgr1;
+ re2 = re_meq1;
+ re3 = re3_5;
+ if (re.test(stem) || (re2.test(stem) && !(re3.test(stem)))) {
+ w = stem;
+ }
+ }
+
+ re = re_5_1;
+ re2 = re_mgr1;
+ if (re.test(w) && re2.test(w)) {
+ re = re_1b_2;
+ w = w.replace(re,"");
+ }
+
+ // and turn initial Y back to y
+
+ if (firstch == "y") {
+ w = firstch.toLowerCase() + w.substr(1);
+ }
+
+ return w;
+ };
+
+ return function (token) {
+ return token.update(porterStemmer);
+ }
+})();
+
+lunr.Pipeline.registerFunction(lunr.stemmer, 'stemmer')
+/*!
+ * lunr.stopWordFilter
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * lunr.generateStopWordFilter builds a stopWordFilter function from the provided
+ * list of stop words.
+ *
+ * The built in lunr.stopWordFilter is built using this generator and can be used
+ * to generate custom stopWordFilters for applications or non English languages.
+ *
+ * @function
+ * @param {Array} token The token to pass through the filter
+ * @returns {lunr.PipelineFunction}
+ * @see lunr.Pipeline
+ * @see lunr.stopWordFilter
+ */
+lunr.generateStopWordFilter = function (stopWords) {
+ var words = stopWords.reduce(function (memo, stopWord) {
+ memo[stopWord] = stopWord
+ return memo
+ }, {})
+
+ return function (token) {
+ if (token && words[token.toString()] !== token.toString()) return token
+ }
+}
+
+/**
+ * lunr.stopWordFilter is an English language stop word list filter, any words
+ * contained in the list will not be passed through the filter.
+ *
+ * This is intended to be used in the Pipeline. If the token does not pass the
+ * filter then undefined will be returned.
+ *
+ * @function
+ * @implements {lunr.PipelineFunction}
+ * @params {lunr.Token} token - A token to check for being a stop word.
+ * @returns {lunr.Token}
+ * @see {@link lunr.Pipeline}
+ */
+lunr.stopWordFilter = lunr.generateStopWordFilter([
+ 'a',
+ 'able',
+ 'about',
+ 'across',
+ 'after',
+ 'all',
+ 'almost',
+ 'also',
+ 'am',
+ 'among',
+ 'an',
+ 'and',
+ 'any',
+ 'are',
+ 'as',
+ 'at',
+ 'be',
+ 'because',
+ 'been',
+ 'but',
+ 'by',
+ 'can',
+ 'cannot',
+ 'could',
+ 'dear',
+ 'did',
+ 'do',
+ 'does',
+ 'either',
+ 'else',
+ 'ever',
+ 'every',
+ 'for',
+ 'from',
+ 'get',
+ 'got',
+ 'had',
+ 'has',
+ 'have',
+ 'he',
+ 'her',
+ 'hers',
+ 'him',
+ 'his',
+ 'how',
+ 'however',
+ 'i',
+ 'if',
+ 'in',
+ 'into',
+ 'is',
+ 'it',
+ 'its',
+ 'just',
+ 'least',
+ 'let',
+ 'like',
+ 'likely',
+ 'may',
+ 'me',
+ 'might',
+ 'most',
+ 'must',
+ 'my',
+ 'neither',
+ 'no',
+ 'nor',
+ 'not',
+ 'of',
+ 'off',
+ 'often',
+ 'on',
+ 'only',
+ 'or',
+ 'other',
+ 'our',
+ 'own',
+ 'rather',
+ 'said',
+ 'say',
+ 'says',
+ 'she',
+ 'should',
+ 'since',
+ 'so',
+ 'some',
+ 'than',
+ 'that',
+ 'the',
+ 'their',
+ 'them',
+ 'then',
+ 'there',
+ 'these',
+ 'they',
+ 'this',
+ 'tis',
+ 'to',
+ 'too',
+ 'twas',
+ 'us',
+ 'wants',
+ 'was',
+ 'we',
+ 'were',
+ 'what',
+ 'when',
+ 'where',
+ 'which',
+ 'while',
+ 'who',
+ 'whom',
+ 'why',
+ 'will',
+ 'with',
+ 'would',
+ 'yet',
+ 'you',
+ 'your'
+])
+
+lunr.Pipeline.registerFunction(lunr.stopWordFilter, 'stopWordFilter')
+/*!
+ * lunr.trimmer
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * lunr.trimmer is a pipeline function for trimming non word
+ * characters from the beginning and end of tokens before they
+ * enter the index.
+ *
+ * This implementation may not work correctly for non latin
+ * characters and should either be removed or adapted for use
+ * with languages with non-latin characters.
+ *
+ * @static
+ * @implements {lunr.PipelineFunction}
+ * @param {lunr.Token} token The token to pass through the filter
+ * @returns {lunr.Token}
+ * @see lunr.Pipeline
+ */
+lunr.trimmer = function (token) {
+ return token.update(function (s) {
+ return s.replace(/^\W+/, '').replace(/\W+$/, '')
+ })
+}
+
+lunr.Pipeline.registerFunction(lunr.trimmer, 'trimmer')
+/*!
+ * lunr.TokenSet
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * A token set is used to store the unique list of all tokens
+ * within an index. Token sets are also used to represent an
+ * incoming query to the index, this query token set and index
+ * token set are then intersected to find which tokens to look
+ * up in the inverted index.
+ *
+ * A token set can hold multiple tokens, as in the case of the
+ * index token set, or it can hold a single token as in the
+ * case of a simple query token set.
+ *
+ * Additionally token sets are used to perform wildcard matching.
+ * Leading, contained and trailing wildcards are supported, and
+ * from this edit distance matching can also be provided.
+ *
+ * Token sets are implemented as a minimal finite state automata,
+ * where both common prefixes and suffixes are shared between tokens.
+ * This helps to reduce the space used for storing the token set.
+ *
+ * @constructor
+ */
+lunr.TokenSet = function () {
+ this.final = false
+ this.edges = {}
+ this.id = lunr.TokenSet._nextId
+ lunr.TokenSet._nextId += 1
+}
+
+/**
+ * Keeps track of the next, auto increment, identifier to assign
+ * to a new tokenSet.
+ *
+ * TokenSets require a unique identifier to be correctly minimised.
+ *
+ * @private
+ */
+lunr.TokenSet._nextId = 1
+
+/**
+ * Creates a TokenSet instance from the given sorted array of words.
+ *
+ * @param {String[]} arr - A sorted array of strings to create the set from.
+ * @returns {lunr.TokenSet}
+ * @throws Will throw an error if the input array is not sorted.
+ */
+lunr.TokenSet.fromArray = function (arr) {
+ var builder = new lunr.TokenSet.Builder
+
+ for (var i = 0, len = arr.length; i < len; i++) {
+ builder.insert(arr[i])
+ }
+
+ builder.finish()
+ return builder.root
+}
+
+/**
+ * Creates a token set from a query clause.
+ *
+ * @private
+ * @param {Object} clause - A single clause from lunr.Query.
+ * @param {string} clause.term - The query clause term.
+ * @param {number} [clause.editDistance] - The optional edit distance for the term.
+ * @returns {lunr.TokenSet}
+ */
+lunr.TokenSet.fromClause = function (clause) {
+ if ('editDistance' in clause) {
+ return lunr.TokenSet.fromFuzzyString(clause.term, clause.editDistance)
+ } else {
+ return lunr.TokenSet.fromString(clause.term)
+ }
+}
+
+/**
+ * Creates a token set representing a single string with a specified
+ * edit distance.
+ *
+ * Insertions, deletions, substitutions and transpositions are each
+ * treated as an edit distance of 1.
+ *
+ * Increasing the allowed edit distance will have a dramatic impact
+ * on the performance of both creating and intersecting these TokenSets.
+ * It is advised to keep the edit distance less than 3.
+ *
+ * @param {string} str - The string to create the token set from.
+ * @param {number} editDistance - The allowed edit distance to match.
+ * @returns {lunr.Vector}
+ */
+lunr.TokenSet.fromFuzzyString = function (str, editDistance) {
+ var root = new lunr.TokenSet
+
+ var stack = [{
+ node: root,
+ editsRemaining: editDistance,
+ str: str
+ }]
+
+ while (stack.length) {
+ var frame = stack.pop()
+
+ // no edit
+ if (frame.str.length > 0) {
+ var char = frame.str.charAt(0),
+ noEditNode
+
+ if (char in frame.node.edges) {
+ noEditNode = frame.node.edges[char]
+ } else {
+ noEditNode = new lunr.TokenSet
+ frame.node.edges[char] = noEditNode
+ }
+
+ if (frame.str.length == 1) {
+ noEditNode.final = true
+ }
+
+ stack.push({
+ node: noEditNode,
+ editsRemaining: frame.editsRemaining,
+ str: frame.str.slice(1)
+ })
+ }
+
+ if (frame.editsRemaining == 0) {
+ continue
+ }
+
+ // insertion
+ if ("*" in frame.node.edges) {
+ var insertionNode = frame.node.edges["*"]
+ } else {
+ var insertionNode = new lunr.TokenSet
+ frame.node.edges["*"] = insertionNode
+ }
+
+ if (frame.str.length == 0) {
+ insertionNode.final = true
+ }
+
+ stack.push({
+ node: insertionNode,
+ editsRemaining: frame.editsRemaining - 1,
+ str: frame.str
+ })
+
+ // deletion
+ // can only do a deletion if we have enough edits remaining
+ // and if there are characters left to delete in the string
+ if (frame.str.length > 1) {
+ stack.push({
+ node: frame.node,
+ editsRemaining: frame.editsRemaining - 1,
+ str: frame.str.slice(1)
+ })
+ }
+
+ // deletion
+ // just removing the last character from the str
+ if (frame.str.length == 1) {
+ frame.node.final = true
+ }
+
+ // substitution
+ // can only do a substitution if we have enough edits remaining
+ // and if there are characters left to substitute
+ if (frame.str.length >= 1) {
+ if ("*" in frame.node.edges) {
+ var substitutionNode = frame.node.edges["*"]
+ } else {
+ var substitutionNode = new lunr.TokenSet
+ frame.node.edges["*"] = substitutionNode
+ }
+
+ if (frame.str.length == 1) {
+ substitutionNode.final = true
+ }
+
+ stack.push({
+ node: substitutionNode,
+ editsRemaining: frame.editsRemaining - 1,
+ str: frame.str.slice(1)
+ })
+ }
+
+ // transposition
+ // can only do a transposition if there are edits remaining
+ // and there are enough characters to transpose
+ if (frame.str.length > 1) {
+ var charA = frame.str.charAt(0),
+ charB = frame.str.charAt(1),
+ transposeNode
+
+ if (charB in frame.node.edges) {
+ transposeNode = frame.node.edges[charB]
+ } else {
+ transposeNode = new lunr.TokenSet
+ frame.node.edges[charB] = transposeNode
+ }
+
+ if (frame.str.length == 1) {
+ transposeNode.final = true
+ }
+
+ stack.push({
+ node: transposeNode,
+ editsRemaining: frame.editsRemaining - 1,
+ str: charA + frame.str.slice(2)
+ })
+ }
+ }
+
+ return root
+}
+
+/**
+ * Creates a TokenSet from a string.
+ *
+ * The string may contain one or more wildcard characters (*)
+ * that will allow wildcard matching when intersecting with
+ * another TokenSet.
+ *
+ * @param {string} str - The string to create a TokenSet from.
+ * @returns {lunr.TokenSet}
+ */
+lunr.TokenSet.fromString = function (str) {
+ var node = new lunr.TokenSet,
+ root = node
+
+ /*
+ * Iterates through all characters within the passed string
+ * appending a node for each character.
+ *
+ * When a wildcard character is found then a self
+ * referencing edge is introduced to continually match
+ * any number of any characters.
+ */
+ for (var i = 0, len = str.length; i < len; i++) {
+ var char = str[i],
+ final = (i == len - 1)
+
+ if (char == "*") {
+ node.edges[char] = node
+ node.final = final
+
+ } else {
+ var next = new lunr.TokenSet
+ next.final = final
+
+ node.edges[char] = next
+ node = next
+ }
+ }
+
+ return root
+}
+
+/**
+ * Converts this TokenSet into an array of strings
+ * contained within the TokenSet.
+ *
+ * This is not intended to be used on a TokenSet that
+ * contains wildcards, in these cases the results are
+ * undefined and are likely to cause an infinite loop.
+ *
+ * @returns {string[]}
+ */
+lunr.TokenSet.prototype.toArray = function () {
+ var words = []
+
+ var stack = [{
+ prefix: "",
+ node: this
+ }]
+
+ while (stack.length) {
+ var frame = stack.pop(),
+ edges = Object.keys(frame.node.edges),
+ len = edges.length
+
+ if (frame.node.final) {
+ /* In Safari, at this point the prefix is sometimes corrupted, see:
+ * https://github.com/olivernn/lunr.js/issues/279 Calling any
+ * String.prototype method forces Safari to "cast" this string to what
+ * it's supposed to be, fixing the bug. */
+ frame.prefix.charAt(0)
+ words.push(frame.prefix)
+ }
+
+ for (var i = 0; i < len; i++) {
+ var edge = edges[i]
+
+ stack.push({
+ prefix: frame.prefix.concat(edge),
+ node: frame.node.edges[edge]
+ })
+ }
+ }
+
+ return words
+}
+
+/**
+ * Generates a string representation of a TokenSet.
+ *
+ * This is intended to allow TokenSets to be used as keys
+ * in objects, largely to aid the construction and minimisation
+ * of a TokenSet. As such it is not designed to be a human
+ * friendly representation of the TokenSet.
+ *
+ * @returns {string}
+ */
+lunr.TokenSet.prototype.toString = function () {
+ // NOTE: Using Object.keys here as this.edges is very likely
+ // to enter 'hash-mode' with many keys being added
+ //
+ // avoiding a for-in loop here as it leads to the function
+ // being de-optimised (at least in V8). From some simple
+ // benchmarks the performance is comparable, but allowing
+ // V8 to optimize may mean easy performance wins in the future.
+
+ if (this._str) {
+ return this._str
+ }
+
+ var str = this.final ? '1' : '0',
+ labels = Object.keys(this.edges).sort(),
+ len = labels.length
+
+ for (var i = 0; i < len; i++) {
+ var label = labels[i],
+ node = this.edges[label]
+
+ str = str + label + node.id
+ }
+
+ return str
+}
+
+/**
+ * Returns a new TokenSet that is the intersection of
+ * this TokenSet and the passed TokenSet.
+ *
+ * This intersection will take into account any wildcards
+ * contained within the TokenSet.
+ *
+ * @param {lunr.TokenSet} b - An other TokenSet to intersect with.
+ * @returns {lunr.TokenSet}
+ */
+lunr.TokenSet.prototype.intersect = function (b) {
+ var output = new lunr.TokenSet,
+ frame = undefined
+
+ var stack = [{
+ qNode: b,
+ output: output,
+ node: this
+ }]
+
+ while (stack.length) {
+ frame = stack.pop()
+
+ // NOTE: As with the #toString method, we are using
+ // Object.keys and a for loop instead of a for-in loop
+ // as both of these objects enter 'hash' mode, causing
+ // the function to be de-optimised in V8
+ var qEdges = Object.keys(frame.qNode.edges),
+ qLen = qEdges.length,
+ nEdges = Object.keys(frame.node.edges),
+ nLen = nEdges.length
+
+ for (var q = 0; q < qLen; q++) {
+ var qEdge = qEdges[q]
+
+ for (var n = 0; n < nLen; n++) {
+ var nEdge = nEdges[n]
+
+ if (nEdge == qEdge || qEdge == '*') {
+ var node = frame.node.edges[nEdge],
+ qNode = frame.qNode.edges[qEdge],
+ final = node.final && qNode.final,
+ next = undefined
+
+ if (nEdge in frame.output.edges) {
+ // an edge already exists for this character
+ // no need to create a new node, just set the finality
+ // bit unless this node is already final
+ next = frame.output.edges[nEdge]
+ next.final = next.final || final
+
+ } else {
+ // no edge exists yet, must create one
+ // set the finality bit and insert it
+ // into the output
+ next = new lunr.TokenSet
+ next.final = final
+ frame.output.edges[nEdge] = next
+ }
+
+ stack.push({
+ qNode: qNode,
+ output: next,
+ node: node
+ })
+ }
+ }
+ }
+ }
+
+ return output
+}
+lunr.TokenSet.Builder = function () {
+ this.previousWord = ""
+ this.root = new lunr.TokenSet
+ this.uncheckedNodes = []
+ this.minimizedNodes = {}
+}
+
+lunr.TokenSet.Builder.prototype.insert = function (word) {
+ var node,
+ commonPrefix = 0
+
+ if (word < this.previousWord) {
+ throw new Error ("Out of order word insertion")
+ }
+
+ for (var i = 0; i < word.length && i < this.previousWord.length; i++) {
+ if (word[i] != this.previousWord[i]) break
+ commonPrefix++
+ }
+
+ this.minimize(commonPrefix)
+
+ if (this.uncheckedNodes.length == 0) {
+ node = this.root
+ } else {
+ node = this.uncheckedNodes[this.uncheckedNodes.length - 1].child
+ }
+
+ for (var i = commonPrefix; i < word.length; i++) {
+ var nextNode = new lunr.TokenSet,
+ char = word[i]
+
+ node.edges[char] = nextNode
+
+ this.uncheckedNodes.push({
+ parent: node,
+ char: char,
+ child: nextNode
+ })
+
+ node = nextNode
+ }
+
+ node.final = true
+ this.previousWord = word
+}
+
+lunr.TokenSet.Builder.prototype.finish = function () {
+ this.minimize(0)
+}
+
+lunr.TokenSet.Builder.prototype.minimize = function (downTo) {
+ for (var i = this.uncheckedNodes.length - 1; i >= downTo; i--) {
+ var node = this.uncheckedNodes[i],
+ childKey = node.child.toString()
+
+ if (childKey in this.minimizedNodes) {
+ node.parent.edges[node.char] = this.minimizedNodes[childKey]
+ } else {
+ // Cache the key for this node since
+ // we know it can't change anymore
+ node.child._str = childKey
+
+ this.minimizedNodes[childKey] = node.child
+ }
+
+ this.uncheckedNodes.pop()
+ }
+}
+/*!
+ * lunr.Index
+ * Copyright (C) 2020 Oliver Nightingale
+ */
+
+/**
+ * An index contains the built index of all documents and provides a query interface
+ * to the index.
+ *
+ * Usually instances of lunr.Index will not be created using this constructor, instead
+ * lunr.Builder should be used to construct new indexes, or lunr.Index.load should be
+ * used to load previously built and serialized indexes.
+ *
+ * @constructor
+ * @param {Object} attrs - The attributes of the built search index.
+ * @param {Object} attrs.invertedIndex - An index of term/field to document reference.
+ * @param {Object' + noResultsText + '
'); + } +} + +function doSearch () { + var query = document.getElementById('mkdocs-search-query').value; + if (query.length > min_search_length) { + if (!window.Worker) { + displayResults(search(query)); + } else { + searchWorker.postMessage({query: query}); + } + } else { + // Clear results for short queries + displayResults([]); + } +} + +function initSearch () { + var search_input = document.getElementById('mkdocs-search-query'); + if (search_input) { + search_input.addEventListener("keyup", doSearch); + } + var term = getSearchTermFromLocation(); + if (term) { + search_input.value = term; + doSearch(); + } +} + +function onWorkerMessage (e) { + if (e.data.allowSearch) { + initSearch(); + } else if (e.data.results) { + var results = e.data.results; + displayResults(results); + } else if (e.data.config) { + min_search_length = e.data.config.min_search_length-1; + } +} + +if (!window.Worker) { + console.log('Web Worker API not supported'); + // load index in main thread + $.getScript(joinUrl(base_url, "search/worker.js")).done(function () { + console.log('Loaded worker'); + init(); + window.postMessage = function (msg) { + onWorkerMessage({data: msg}); + }; + }).fail(function (jqxhr, settings, exception) { + console.error('Could not load worker.js'); + }); +} else { + // Wrap search in a web worker + var searchWorker = new Worker(joinUrl(base_url, "search/worker.js")); + searchWorker.postMessage({init: true}); + searchWorker.onmessage = onWorkerMessage; +} diff --git a/docs/search/search_index.json b/docs/search/search_index.json new file mode 100644 index 00000000..3e608029 --- /dev/null +++ b/docs/search/search_index.json @@ -0,0 +1 @@ +{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Mercury Composable Since version 4.0, we have merged our enterprise extension (\"Event Script\") with the Mercury v3.1 foundation codebase. It is a comprehensive toolkit to write composable applications including microservices and serverless. This technology was filed under US Patent application 18/459,307. The source code is provided as is under the Apache 2.0 license. August 2024 Getting Started A composable application is designed in 3 steps: Describe your use case as an event flow diagram Create a configuration file to represent the event flow Write a user story for each user function To get started, please visit Chapter 1, Developer Guide . We will illustrate the methodology with a composable application example. Conquer Complexity: Embrace Composable Design Introduction Software development is an ongoing battle against complexity. Over time, codebases can become tangled and unwieldy, hindering innovation and maintenance. This article introduces composable design patterns, a powerful approach to build applications that are modular, maintainable, and scalable. The Perils of Spaghetti Code We have all encountered it: code that resembles a plate of spaghetti \u2013 tangled dependencies, hidden logic, and a general sense of dread when approaching modifications. These codebases are difficult to test, debug, and update. Composable design patterns offer a solution. Evolution of Design Patterns Software development methodologies have evolved alongside hardware advancements. In the early days, developers prized efficiency, writing code from scratch due to limited libraries. The rise of frameworks brought structure and boilerplate code, but also introduced potential rigidity. Functional Programming and Event-Driven Architecture Functional programming, with its emphasis on pure functions and immutable data, paved the way for composable design. This approach encourages building applications as chains of well-defined functions, each with a clear input and output. Event-driven architecture complements this approach by using events to trigger functions. This loose coupling promotes modularity and scalability. The Power of Composable Design At its core, composable design emphasizes two principles: Self-Contained Functions : Each function is a well-defined unit, handling its own logic and transformations with minimal dependencies. Event Choreography : Functions communicate through events, allowing for loose coupling and independent execution. Benefits of Composable Design Enhanced Maintainability : Isolated functions are easier to understand, test, and modify. Improved Reusability : Self-contained functions can be easily reused across different parts of your application. Superior Performance : Loose coupling reduces bottlenecks and encourages asynchronous execution. Streamlined Testing : Well-defined functions facilitate unit testing and isolate potential issues. Simplified Debugging : Independent functions make it easier to pinpoint the source of errors. Technology Agnostic : You may use your preferred frameworks and tools to write composable code, allowing for easier future adaptations. Implementing Composable Design While seemingly simple, implementing composable design can involve some initial complexity. Here's a breakdown of the approach: Function Design : Each function serves a specific purpose, with clearly defined inputs and outputs. Event Communication : Functions communicate through well-defined events, avoiding direct dependencies. Choreography : An event manager, with a state machine and event flow configuration, sequences and triggers functions based on events. Conclusion Composable design patterns offer a powerful paradigm for building maintainable, scalable, and future-proof applications. By embracing the principles of self-contained functions and event-driven communication, you can conquer complexity and write code that is a joy to work with. Are you ready to take your development practices to the next level? Embrace composable design now!","title":"Home"},{"location":"#mercury-composable","text":"Since version 4.0, we have merged our enterprise extension (\"Event Script\") with the Mercury v3.1 foundation codebase. It is a comprehensive toolkit to write composable applications including microservices and serverless. This technology was filed under US Patent application 18/459,307. The source code is provided as is under the Apache 2.0 license. August 2024","title":"Mercury Composable"},{"location":"#getting-started","text":"A composable application is designed in 3 steps: Describe your use case as an event flow diagram Create a configuration file to represent the event flow Write a user story for each user function To get started, please visit Chapter 1, Developer Guide . We will illustrate the methodology with a composable application example.","title":"Getting Started"},{"location":"#conquer-complexity-embrace-composable-design","text":"","title":"Conquer Complexity: Embrace Composable Design"},{"location":"#introduction","text":"Software development is an ongoing battle against complexity. Over time, codebases can become tangled and unwieldy, hindering innovation and maintenance. This article introduces composable design patterns, a powerful approach to build applications that are modular, maintainable, and scalable.","title":"Introduction"},{"location":"#the-perils-of-spaghetti-code","text":"We have all encountered it: code that resembles a plate of spaghetti \u2013 tangled dependencies, hidden logic, and a general sense of dread when approaching modifications. These codebases are difficult to test, debug, and update. Composable design patterns offer a solution.","title":"The Perils of Spaghetti Code"},{"location":"#evolution-of-design-patterns","text":"Software development methodologies have evolved alongside hardware advancements. In the early days, developers prized efficiency, writing code from scratch due to limited libraries. The rise of frameworks brought structure and boilerplate code, but also introduced potential rigidity.","title":"Evolution of Design Patterns"},{"location":"#functional-programming-and-event-driven-architecture","text":"Functional programming, with its emphasis on pure functions and immutable data, paved the way for composable design. This approach encourages building applications as chains of well-defined functions, each with a clear input and output. Event-driven architecture complements this approach by using events to trigger functions. This loose coupling promotes modularity and scalability.","title":"Functional Programming and Event-Driven Architecture"},{"location":"#the-power-of-composable-design","text":"At its core, composable design emphasizes two principles: Self-Contained Functions : Each function is a well-defined unit, handling its own logic and transformations with minimal dependencies. Event Choreography : Functions communicate through events, allowing for loose coupling and independent execution.","title":"The Power of Composable Design"},{"location":"#benefits-of-composable-design","text":"Enhanced Maintainability : Isolated functions are easier to understand, test, and modify. Improved Reusability : Self-contained functions can be easily reused across different parts of your application. Superior Performance : Loose coupling reduces bottlenecks and encourages asynchronous execution. Streamlined Testing : Well-defined functions facilitate unit testing and isolate potential issues. Simplified Debugging : Independent functions make it easier to pinpoint the source of errors. Technology Agnostic : You may use your preferred frameworks and tools to write composable code, allowing for easier future adaptations.","title":"Benefits of Composable Design"},{"location":"#implementing-composable-design","text":"While seemingly simple, implementing composable design can involve some initial complexity. Here's a breakdown of the approach: Function Design : Each function serves a specific purpose, with clearly defined inputs and outputs. Event Communication : Functions communicate through well-defined events, avoiding direct dependencies. Choreography : An event manager, with a state machine and event flow configuration, sequences and triggers functions based on events.","title":"Implementing Composable Design"},{"location":"#conclusion","text":"Composable design patterns offer a powerful paradigm for building maintainable, scalable, and future-proof applications. By embracing the principles of self-contained functions and event-driven communication, you can conquer complexity and write code that is a joy to work with. Are you ready to take your development practices to the next level? Embrace composable design now!","title":"Conclusion"},{"location":"CHANGELOG/","text":"Changelog Release notes All notable changes to this project will be documented in this file. The format is based on Keep a Changelog , and this project adheres to Semantic Versioning . Version 4.1.1, 12/18/2024 Added Added \"map\" constant type in input data mapping. Removed N/A Changed Updated Chapter-4 for the new \"map\" constant feature. Version 4.1.0, 12/11/2024 This milestone version achieves ideal event choreography by removing additional event routing to and from the Event Manager. This would boost internal event routing performance by 50 percent. Added Performance optimization for Event Script Removed N/A Changed The platform-core module uses virtual threads to execute event.script.manager and task.executor directly to eliminate additional serialization overheads since the two functions are event routers themselves. Version 4.0.33, 12/11/2024 Added Support of custom content types in application.yml Removed N/A Changed Improved websocket housekeeping logic Use bench.add to replace bench.offer API Version 4.0.32, 12/9/2024 Added For completeness, added Boolean AND and OR operations for simple type mapping. Added traceId as metadata for a flow instance Removed N/A Changed Update Chapter-4 for the new AND/OR type mapping feature Consistent custom HTTP headers for event over http protocol and streaming content Version 4.0.31, 12/5/2024 Added N/A Removed N/A Changed The \"keep.original\" key is renamed as \"keep-original\" to comply with convention. Continue processing if some preload override config files are missing. Version 4.0.30, 12/5/2024 Added Implemented unique task naming feature for event flow configuration. Removed N/A Changed The \"keep_original\" key is renamed as \"keep.original\" in preload override Chapter-4 of developer guide updated with the new task alias feature Version 4.0.29, 12/3/2024 Added Added integer, long, float, double and boolean type matching for state machine. Removed N/A Changed N/A Version 4.0.28, 11/29/2024 Added Support for simple data type matching processing (text, substring, binary and b64) Optional external state machine Removed Removed \"http.input.\" and \"http.output.\" aliases from event script. Instead, use the generic \"input.\" and \"output.\" namespaces. Changed Bugfix for AsyncHttpClient to allow missing HTTP request body in POST, PUT or PATCH request Mono reactive flow control Version 4.0.27, 11/27/2024 Added Support for Mono/Flux return type for KotlinLambdaFunction Implemented Websocket handshake handler to adjust to API changes in vertx 4.5.11 Removed N/A Changed N/A Version 4.0.26, 11/26/2024 Added N/A Removed Remove pom.xml version override for netty and spring framework because Spring Boot 3.4.0 fetches the correct versions of netty and spring framework. Earlier override was done to avoid security vulnerabilities of older versions of netty and spring framework. Changed Handle the case that Mono will not return payload if the payload is null OSS update: Classgraph 4.8.179, Vertx 4.5.11, Spring Boot 3.4.0, Kafka Client 3.9.0 Version 4.0.25, 11/21/2024 Added Support more than one REST configuration files. When a duplicated REST entry is detected, the system will abort REST endpoint rendering and print out an error message in application log. If you have unit tests to cover the REST endpoints, the unit tests will fail accordingly. Removed N/A Changed Improved environment variable parsing in config reader. System will skip entries with invalid environment variable reference syntax. Version 4.0.24, 11/20/2024 Added N/A Removed N/A Changed Bugfix for an edge case in config reader to handle control character of brackets inside an environment variable reference. e.g. some.key=${ENV_VAR:something/{test1}/{test2}} Version 4.0.23, 11/19/2024 Added N/A Removed ObjectStreamWriter and AsyncObjectStreamReader are removed Changed Replace ObjectStreamWriter with FluxPublisher Replace AsyncObjectStreamReader with FluxConsumer Bugfix for FluxConsumer expiry - change type from \"data\" to \"exception\". Version 4.0.22, 11/18/2024 Added FluxPublisher and FluxConsumer for integration with Flux reactive response object Removed N/A Changed Unit tests in event streaming and post office to support Flux integration Select reactor-core version 3.7.0 using dependency management (reactor-bom version 2024.0.0) Version 4.0.21, 11/14/2024 Added Support for user function to return a Mono reactive response object Removed N/A Changed Update netty to version 4.1.115.Final to address security vulnerability in 4.1.114 Move reactor-core library from rest-spring-3 to platform-core Version 4.0.20, 11/13/2024 Added For ease of configuration, added \"com.accenture\" to the base packages so that user applications do not need to include it to use the event-script-engine module. Removed if-then-else pipeline feature in event-script Changed Update Event Script syntax for consistency Fix error in counting number of compiled flows Version 4.0.16, 11/10/2024 Added Generate unique flow instance ID as reference during flow execution. Removed N/A Changed Save the original correlation-ID from the calling party in a flow instance and return this value to the calling party at the end of flow execution. Version 4.0.15, 11/7/2024 Added N/A Removed N/A Changed renamed StartFlow to FlowExecutor Version 4.0.14, 11/7/2024 Added N/A Removed N/A Changed Health check function can return either a text string or a Map StartFlow API updates Version 4.0.13, 11/5/2024 Added Added helper class \"StartFlow\" to start a flow, including internal flows without HTTP or Kafka. Removed N/A Changed Bugfix for empty YAML file to avoid null pointer exception Sort event scripts for orderly logging in the CompileFlows validation process Version 4.0.12, 10/31/2024 Added New feature to support resolution of more than one environment variable for a parameter using the ConfigReader Removed N/A Changed Update OSS modules 1. classgraph version 4.8.177 2. kotlin version 2.0.21 3. guava version 33.3.1-jre 4. jUnit version 5 jupiter Adjusted all unit tests to use jUnit 5 Version 4.0.11, 10/28/2024 Added New features to support: 1. multiple preload override config file 2. multiple flow list config files Removed unused class \"UnauthorizedObj\" in platform-core commons-io dependency in Kafka-Standalone subproject Changed Unit test for the preload override feature JavaDoc for the MainApplication Version 4.0.10, 10/24/2024 Added N/A Removed N/A Changed OSS update - Spring Boot 3.3.5 Security patch for CR/LF exploit for HTTP cookie Version 4.0.9, 10/18/2024 Added Added Kafka Raft for the Kafka-standalone app. Removed Removed zookeeper from Kafka-standalone app. Changed Update spring framework verison 6.1.14 to avoid vulnerability in webflux Version 4.0.8, 10/9/2024 Added Partial support of Active Profile using the \"spring.profiles.active\" parameter Hierarchy of flows Removed N/A Changed N/A Version 4.0.7, 10/1/2024 Added A generic \"no-op\" function for use in event scripts. Removed Feature to ping a function without payload and headers. Changed Simplified api-playground application Version 4.0.6, 9/27/2024 Added HTTP request Cookie value filtering using RFC-6265 strict syntax Removed Automatic index page redirection filter for Spring Boot Changed Upgrade SHA-1 to SHA-512 algorithm in CryptoAPI utility Fix security vulnerability associated with HTTP request header and cookie manipulation Version 4.0.5, 9/24/2024 Added N/A Removed Feature for automatic PoJo transport in EventEnvelope and MsgPack Feature for safe.data.model deserialization Benchmark-server is no longer required Changed Update OSS versions - vertx 4.5.10, kotlin 2.0.20, spring boot 3.3.4 Version 4.0.4, 9/5/2024 Added New feature for AsyncHttpClient to render small streaming HTTP response (i.e. chunked binary data) as byte array. For details, Please refer to Appendix III, Developer Guide Removed N/A Changed Bugfix for parsing default value of environment variable in ConfigReader. This resolves an issue when the special character colon (\":\") is used more than once in the default value. Version 4.0.3, 9/4/2024 Added The \"preload override\" feature is added. This allows overriding a reusable composable library with a set of new route names that are unique for use in an event flow configuration script. For details, Please refer to Chapter 4, Developer Guide Removed N/A Changed N/A Version 4.0.2, 8/31/2024 Added New \"classpath\" namespace for input data mapping Support for input data mapping to handle subset of input request body as a Map or PoJo Removed N/A Changed Remove the class \"type\" variable from AsyncHttpRequest Improve the \"removeElement\" method in MultiLevelMap Make HTTP input request header labels key-insensitive Update Spring Boot to version 3.3.3 Version 4.0.1, 8/19/2024 Added new File read/write feature in Event Script's I/O data mapping Removed N/A Changed Update Spring Boot to version 3.3.2 Update Guava to version 33.3.0-jre Update Vertx to version 4.5.9 Update Kotlin to version 2.0.10 Change \"upstream\" to \"dependency\" in the \"/health\" endpoint Version 4.0.0, 6/24/2024 This version merges Event Script into the Mercury Composable repository. Added N/A Removed N/A Changed Update Spring Boot to version 3.3.1 Update Guava to version 33.2.1-jre Update Vertx to version 4.5.8 Update Kotlin to version 2.0.0 Update classgraph to version 4.8.174 Optional reply event for a flow configuration Kafka-standalone is still using Spring Boot 3.2.5 due to compatibility issue Version 3.1.5, 5/1/2024 This version supercedes 3.1.4 due to updated data structure for static content handling. Added Added optional static-content.no-cache-pages in rest.yaml AsyncHttpClientLoader Removed N/A Changed Updated data structure for static-content section in rest.yaml Fixed bug for setting multiple HTTP cookies Unified configuration file prefix \"yaml.\" Version 3.1.4, 4/28/2024 Added Added optional static content HTTP-GET request filter in rest.yaml Removed N/A Changed Updated syntax for static-content-filter Version 3.1.3, 4/24/2024 Added N/A Removed N/A Changed Enhanced OptionalService annotation. Version 3.1.2, 4/17/2024 Added Added \"app-config-reader.yml\" file in the resources folder so that you can override the default application configuration files. Removed N/A Changed Open sources library update (Spring Boot 3.2.5, Vertx 4.5.7) Improve AppConfigReader and ConfigReader to use the app-config-reader.yml file. Enhanced OptionalService annotation. Version 3.1.1, 2/8/2024 Added AutoStart to run application as Spring Boot if the rest-spring-3 library is packaged in app Configurable \"Event over HTTP\" - automatic forward events over HTTP using a configuration Support user defined serializer with PreLoad annotation and platform API Removed Bugfix: removed websocket client connection timeout that causes the first connection to drop after one minute Changed Open sources library update (Spring Boot 3.2.2, Vertx 4.5.3 and MsgPack 0.9.8) Rename application parameter \"event.worker.pool\" to \"kernel.thread.pool\" Version 3.1.0, 1/5/2024 Added Full integration with Java 21 Virtual Thread Default execution mode is set to \"virtual thread\" KernelThreadRunner annotation added to provide optional support of kernel threads Removed Retired Spring Boot version 2 Hazelcast and ActiveMQ network connectors Changed platform-core engine updated with virtual thread Version 3.0.7, 12/23/2023 Added Print out basic JVM information before startup for verification of base container image. Removed Removed Maven Shade packager Changed Updated open sources libraries to address security vulnerabilities Spring Boot 2/3 to version 2.7.18 and 3.2.1 respectively Tomcat 9.0.84 Vertx 4.5.1 Classgraph 4.8.165 Netty 4.1.104.Final slf4j API 2.0.9 log4j2 2.22.0 Kotlin 1.9.22 Artemis 2.31.2 Hazelcast 5.3.6 Guava 33.0.0-jre Version 3.0.6, 10/26/2023 Added Enhanced Benchmark tool to support \"Event over HTTP\" protocol to evaluate performance efficiency for commmunication between application containers using HTTP. Removed N/A Changed Updated open sources libraries Spring Boot 2/3 to version 2.7.17 and 3.1.5 respectively Kafka-client 3.6.0 Version 3.0.5, 10/21/2023 Added Support two executable JAR packaging system: 1. Maven Shade packager 2. Spring Boot packager Starting from version 3.0.5, we have replaced Spring Boot packager with Maven Shade. This avoids a classpath edge case for Spring Boot packager when running kafka-client under Java 11 or higher. Maven Shade also results in smaller executable JAR size. Removed N/A Changed Updated open sources libraries Spring-Boot 2.7.16 / 3.1.4 classgraph 4.8.163 snakeyaml 2.2 kotlin 1.9.10 vertx 4.4.6 guava 32.1.3-jre msgpack 0.9.6 slj4j 2.0.9 zookeeper 3.7.2 The \"/info/lib\" admin endpoint has been enhanced to list library dependencies for executable JAR generated by either Maven Shade or Spring Boot Packager. Improved ConfigReader to recognize both \".yml\" and \".yaml\" extensions and their uses are interchangeable. Version 3.0.4, 8/6/2023 Added N/A Removed N/A Changed Updated open sources libraries Spring-Boot 2.7.14 / 3.1.2 Kafka-client 3.5.1 classgraph 4.8.161 guava 32.1.2-jre msgpack 0.9.5 Version 3.0.3, 6/27/2023 Added File extension to MIME type mapping for static HTML file handling Removed N/A Changed Open sources library update - Kotlin version 1.9.0 Version 3.0.2, 6/9/2023 Added N/A Removed N/A Changed Consistent exception handling for Event API endpoint Open sources lib update - Vertx 4.4.4, Spring Boot 2.7.13, Spring Boot 3.1.1, classgraph 4.8.160, guava 32.0.1-jre Version 3.0.1, 6/5/2023 In this release, we have replace Google HTTP Client with vertx non-blocking WebClient. We also tested compatibility up to OpenJDK version 20 and maven 3.9.2. Added When \"x-raw-xml\" HTTP request header is set to \"true\", the AsyncHttpClient will skip the built-in XML serialization so that your application can retrieve the original XML text. Removed Retire Google HTTP client Changed Upgrade maven plugin versions. Version 3.0.0, 4/18/2023 This is a major release with some breaking changes. Please refer to Chapter-10 (Migration guide) for details. This version brings the best of preemptive and cooperating multitasking to Java (version 1.8 to 19) before Java 19 virtual thread feature becomes officially available. Added Function execution engine supporting kernel thread pool, Kotlin coroutine and suspend function \"Event over HTTP\" service for inter-container communication Support for Spring Boot version 3 and WebFlux Sample code for a pre-configured Spring Boot 3 application Removed Remove blocking APIs from platform-core Retire PM2 process manager sample script due to compatibility issue Changed Refactor \"async.http.request\" to use vertx web client for non-blocking operation Update log4j2 version 2.20.0 and slf4j version 2.0.7 in platform-core Update JBoss RestEasy JAX_RS to version 3.15.6.Final in rest-spring Update vertx to 4.4.2 Update Spring Boot parent pom to 2.7.12 and 3.1.0 for spring boot 2 and 3 respectively Remove com.fasterxml.classmate dependency from rest-spring Version 2.8.0, 3/20/2023 Added N/A Removed N/A Changed Improved load balancing in cloud-connector Filter URI to avoid XSS attack Upgrade to SnakeYaml 2.0 and patch Spring Boot 2.6.8 for compatibility with it Upgrade to Vertx 4.4.0, classgraph 4.8.157, tomcat 9.0.73 Version 2.7.1, 12/22/2022 Added standalone benchmark report app client and server benchmark apps add timeout tag to RPC events Removed N/A Changed Updated open sources dependencies Netty 4.1.86.Final Tomcat 9.0.69 Vertx 4.3.6 classgraph 4.8.152 google-http-client 1.42.3 Improved unit tests to use assertThrows to evaluate exception Enhanced AsyncHttpRequest serialization Version 2.7.0, 11/11/2022 In this version, REST automation code is moved to platform-core such that REST and Websocket service can share the same port. Added AsyncObjectStreamReader is added for non-blocking read operation from an object stream. Support of LocalDateTime in SimpleMapper Add \"removeElement\" method to MultiLevelMap Automatically convert a map to a PoJo when the sender does not specify class in event body Removed N/A Changed REST automation becomes part of platform-core and it can co-exist with Spring Web in the rest-spring module Enforce Spring Boot lifecycle management such that user apps will start after Spring Boot has loaded all components Update netty to version 4.1.84.Final Version 2.6.0, 10/13/2022 In this version, websocket notification example code has been removed from the REST automation system. If your application uses this feature, please recover the code from version 2.5.0 and refactor it as a separate library. Added N/A Removed Simplify REST automation system by removing websocket notification example in REST automation. Changed Replace Tomcat websocket server with Vertx non-blocking websocket server library Update netty to version 4.1.79.Final Update kafka client to version 2.8.2 Update snake yaml to version 1.33 Update gson to version 2.9.1 Version 2.5.0, 9/10/2022 Added New Preload annotation class to automate pre-registration of LambdaFunction. Removed Removed Spring framework and Tomcat dependencies from platform-core so that the core library can be applied to legacy J2EE application without library conflict. Changed Bugfix for proper housekeeping of future events. Make Gson and MsgPack handling of integer/long consistent Updated open sources libraries. Eclipse vertx-core version 4.3.4 MsgPack version 0.9.3 Google httpclient version 1.42.2 SnakeYaml version 1.31 Version 2.3.6, 6/21/2022 Added Support more than one event stream cluster. User application can share the same event stream cluster for pub/sub or connect to an alternative cluster for pub/sub use cases. Removed N/A Changed Cloud connector libraries update to Hazelcast 5.1.2 Version 2.3.5, 5/30/2022 Added Add tagging feature to handle language connector's routing and exception handling Removed Remove language pack's pub/sub broadcast feature Changed Update Spring Boot parent to version 2.6.8 to fetch Netty 4.1.77 and Spring Framework 5.3.20 Streamlined language connector transport protocol for compatibility with both Python and Node.js Version 2.3.4, 5/14/2022 Added N/A Removed Remove swagger-ui distribution from api-playground such that developer can clone the latest version Changed Update application.properties (from spring.resources.static-locations to spring.web.resources.static-locations) Update log4j, Tomcat and netty library version using Spring parent 2.6.6 Version 2.3.3, 3/30/2022 Added Enhanced AsyncRequest to handle non-blocking fork-n-join Removed N/A Changed Upgrade Spring Boot from 2.6.3 to 2.6.6 Version 2.3.2, 2/21/2022 Added Add support of queue API in native pub/sub module for improved ESB compatibility Removed N/A Changed N/A Version 2.3.1, 2/19/2022 Added N/A Removed N/A Changed Update Vertx to version 4.2.4 Update Tomcat to version 5.0.58 Use Tomcat websocket server for presence monitors Bugfix - Simple Scheduler's leader election searches peers correctly Version 2.3.0, 1/28/2022 Added N/A Removed N/A Changed Update copyright notice Update Vertx to version 4.2.3 Bugfix - RSA key generator supporting key length from 1024 to 4096 bits CryptoAPI - support different AES algorithms and custom IV Update Spring Boot to version 2.6.3 Version 2.2.3, 12/29/2021 Added Transaction journaling Add parameter distributed.trace.aggregation in application.properties such that trace aggregation may be disabled. Removed N/A Changed Update JBoss RestEasy library to 3.15.3.Final Improved po.search(route) to scan local and remote service registries. Added \"remoteOnly\" selection. Fix bug in releasing presence monitor topic for specific closed user group Update Apache log4j to version 2.17.1 Update Spring Boot parent to version 2.6.1 Update Netty to version 4.1.72.Final Update Vertx to version 4.2.2 Convenient class \"UserNotification\" for backend service to publish events to the UI when REST automation is deployed Version 2.2.2, 11/12/2021 Added User defined API authentication functions can be selected using custom HTTP request header \"Exception chaining\" feature in EventEnvelope New \"deferred.commit.log\" parameter for backward compatibility with older PowerMock in unit tests Removed N/A Changed Improved and streamlined SimpleXmlParser to handle arrays Bugfix for file upload in Service Gateway (REST automation library) Update Tomcat library from 9.0.50 to 9.0.54 Update Spring Boot library to 2.5.6 Update GSON library to 2.8.9 Version 2.2.1, 10/1/2021 Added Callback function can implement ServiceExceptionHandler to catch exception. It adds the onError() method. Removed N/A Changed Open sources library update - Vert.x 4.1.3, Netty 4.1.68-Final Version 2.1.1, 9/10/2021 Added User defined PoJo and Generics mapping Standardized serializers for default case, snake_case and camelCase Support of EventEnvelope as input parameter in TypedLambdaFunction so application function can inspect event's metadata Application can subscribe to life cycle events of other application instances Removed N/A Changed Replace Tomcat websocket server engine with Vertx in presence monitor for higher performance Bugfix for MsgPack transport of integer, long, BigInteger and BigDecimal Version 2.1.0, 7/25/2021 Added Multicast - application can define a multicast.yaml config to relay events to more than one target service. StreamFunction - function that allows the application to control back-pressure Removed \"object.streams.io\" route is removed from platform-core Changed Elastic Queue - Refactored using Oracle Berkeley DB Object stream I/O - simplified design using the new StreamFunction feature Open sources library update - Spring Boot 2.5.2, Tomcat 9.0.50, Vert.x 4.1.1, Netty 4.1.66-Final Version 2.0.0, 5/5/2021 Vert.x is introduced as the in-memory event bus Added ActiveMQ and Tibco connectors Admin endpoints to stop, suspend and resume an application instance Handle edge case to detect stalled application instances Add \"isStreamingPubSub\" method to the PubSub interface Removed Event Node event stream emulator has been retired. You may use standalone Kafka server as a replacement for development and testing in your laptop. Multi-tenancy namespace configuration has been retired. It is replaced by the \"closed user group\" feature. Changed Refactored Kafka and Hazelcast connectors to support virtual topics and closed user groups. Updated ConfigReader to be consistent with Spring value substitution logic for application properties Replace Akka actor system with Vert.x event bus Common code for various cloud connectors consolidated into cloud core libraries Version 1.13.0, 1/15/2021 Version 1.13.0 is the last version that uses Akka as the in-memory event system. Version 1.12.66, 1/15/2021 Added A simple websocket notification service is integrated into the REST automation system Seamless migration feature is added to the REST automation system Removed Legacy websocket notification example application Changed N/A Version 1.12.65, 12/9/2020 Added \"kafka.pubsub\" is added as a cloud service File download example in the lambda-example project \"trace.log.header\" added to application.properties - when tracing is enabled, this inserts the trace-ID of the transaction in the log context. For more details, please refer to the Developer Guide Add API to pub/sub engine to support creation of topic with partitions TypedLambdaFunction is added so that developer can predefine input and output classes in a service without casting Removed N/A Changed Decouple Kafka pub/sub from kafka connector so that native pub/sub can be used when application is running in standalone mode Rename \"relay\" to \"targetHost\" in AsyncHttpRequest data model Enhanced routing table distribution by sending a complete list of route tables, thus reducing network admin traffic. Version 1.12.64, 9/28/2020 Added If predictable topic is set, application instances will report their predictable topics as \"instance ID\" to the presence monitor. This improves visibility when a developer tests their application in \"hybrid\" mode. i.e. running the app locally and connect to the cloud remotely for event streams and cloud resources. Removed N/A Changed N/A Version 1.12.63, 8/27/2020 Added N/A Removed N/A Changed Improved Kafka producer and consumer pairing Version 1.12.62, 8/12/2020 Added New presence monitor's admin endpoint for the operator to force routing table synchronization (\"/api/ping/now\") Removed N/A Changed Improved routing table integrity check Version 1.12.61, 8/8/2020 Added Event stream systems like Kafka assume topic to be used long term. This version adds support to reuse the same topic when an application instance restarts. You can create a predictable topic using unique application name and instance ID. For example, with Kubernetes, you can use the POD name as the unique application instance topic. Removed N/A Changed N/A Version 1.12.56, 8/4/2020 Added Automate trace for fork-n-join use case Removed N/A Changed N/A Version 1.12.55, 7/19/2020 Added N/A Removed N/A Changed Improved distributed trace - set the \"from\" address in EventEnvelope automatically. Version 1.12.54, 7/10/2020 Added N/A Removed N/A Changed Application life-cycle management - User provided main application(s) will be started after Spring Boot declares web application ready. This ensures correct Spring autowiring or dependencies are available. Bugfix for locale - String.format(float) returns comma as decimal point that breaks number parser. Replace with BigDecimal decimal point scaling. Bugfix for Tomcat 9.0.35 - Change Async servlet default timeout from 30 seconds to -1 so the system can handle the whole life-cycle directly. Version 1.12.52, 6/11/2020 Added new \"search\" method in Post Office to return a list of application instances for a service simple \"cron\" job scheduler as an extension project add \"sequence\" to MainApplication annotation for orderly execution when more than one MainApplication is available support \"Optional\" object in EventEnvelope so a LambdaFunction can read and return Optional Removed N/A Changed The rest-spring library has been updated to support both JAR and WAR deployment All pom.xml files updated accordingly PersistentWsClient will back off for 10 seconds when disconnected by remote host Version 1.12.50, 5/20/2020 Added Payload segmentation For large payload in an event, the payload is automatically segmented into 64 KB segments. When there are more than one target application instances, the system ensures that the segments of the same event is delivered to exactly the same target. PersistentWsClient added - generalized persistent websocket client for Event Node, Kafka reporter and Hazelcast reporter. Removed N/A Changed Code cleaning to improve consistency Upgraded to hibernate-validator to v6.1.5.Final and Hazelcast version 4.0.1 REST automation is provided as a library and an application to handle different use cases Version 1.12.40, 5/4/2020 Added N/A Removed N/A Changed For security reason, upgrade log4j to version 2.13.2 Version 1.12.39, 5/3/2020 Added Use RestEasy JAX-RS library Removed For security reason, removed Jersey JAX-RS library Changed Updated RestLoader to initialize RestEasy servlet dispatcher Support nested arrays in MultiLevelMap Version 1.12.36, 4/16/2020 Added N/A Removed For simplicity, retire route-substitution admin endpoint. Route substitution uses a simple static table in route-substitution.yaml. Changed N/A Version 1.12.35, 4/12/2020 Added N/A Removed SimpleRBAC class is retired Changed Improved ConfigReader and AppConfigReader with automatic key-value normalization for YAML and JSON files Improved pub/sub module in kafka-connector Version 1.12.34, 3/28/2020 Added N/A Removed Retired proprietary config manager since we can use the \"BeforeApplication\" approach to load config from Kubernetes configMap or other systems of config record. Changed Added \"isZero\" method to the SimpleMapper class Convert BigDecimal to string without scientific notation (i.e. toPlainString instead of toString) Corresponding unit tests added to verify behavior Version 1.12.32, 3/14/2020 Added N/A Removed N/A Changed Kafka-connector will shutdown application instance when the EventProducer cannot send event to Kafka. This would allow the infrastructure to restart application instance automatically. Version 1.12.31, 2/26/2020 Added N/A Removed N/A Changed Kafka-connector now supports external service provider for Kafka properties and credentials. If your application implements a function with route name \"kafka.properties.provider\" before connecting to cloud, the kafka-connector will retrieve kafka credentials on demand. This addresses case when kafka credentials change after application start-up. Interceptors are designed to forward requests and thus they do not generate replies. However, if you implement a function as an EventInterceptor, your function can throw exception just like a regular function and the exception will be returned to the calling function. This makes it easier to write interceptors. Version 1.12.30, 2/6/2020 Added Expose \"async.http.request\" as a PUBLIC function (\"HttpClient as a service\") Removed N/A Changed Improved Hazelcast client connection stability Improved Kafka native pub/sub Version 1.12.29, 1/10/2020 Added Rest-automation will transport X-Trace-Id from/to Http request/response, therefore extending distributed trace across systems that support the X-Trace-Id HTTP header. Added endpoint and service to shutdown application instance. Removed N/A Changed Updated SimpleXmlParser with XML External Entity (XXE) injection prevention. Bug fix for hazelcast recovery logic - when a hazelcast node is down, the app instance will restart the hazelcast client and reset routing table correctly. HSTS header insertion is optional so that we can disable it to avoid duplicated header when API gateway is doing it. Version 1.12.26, 1/4/2020 Added Feature to disable PoJo deserialization so that caller can decide if the result set should be in PoJo or a Map. Removed N/A Changed Simplified key management for Event Node AsyncHttpRequest case insensitivity for headers, cookies, path parameters and session key-values Make built-in configuration management optional Version 1.12.19, 12/28/2019 Added Added HTTP relay feature in rest-automation project Removed N/A Changed Improved hazelcast retry and peer discovery logic Refactored rest-automation's service gateway module to use AsyncHttpRequest Info endpoint to show routing table of a peer Version 1.12.17, 12/16/2019 Added Simple configuration management is added to event-node, hazelcast-presence and kafka-presence monitors Added BeforeApplication annotation - this allows user application to execute some setup logic before the main application starts. e.g. modifying parameters in application.properties Added API playground as a convenient standalone application to render OpenAPI 2.0 and 3.0 yaml and json files Added argument parser in rest-automation helper app to use a static HTML folder in the local file system if arguments -html file_path is given when starting the JAR file. Removed N/A Changed Kafka publisher timeout value changed from 10 to 20 seconds Log a warning when Kafka takes more than 5 seconds to send an event Version 1.12.14, 11/20/2019 Added getRoute() method is added to PostOffice to facilitate RBAC The route name of the current service is added to an outgoing event when the \"from\" field is not present Simple RBAC using YAML configuration instead of code Removed N/A Changed Updated Spring Boot to v2.2.1 Version 1.12.12, 10/26/2019 Added Multi-tenancy support for event streams (Hazelcast and Kafka). This allows the use of a single event stream cluster for multiple non-prod environments. For production, it must use a separate event stream cluster for security reason. Removed N/A Changed logging framework changed from logback to log4j2 (version 2.12.1) Use JSR-356 websocket annotated ClientEndpoint Improved websocket reconnection logic Version 1.12.9, 9/14/2019 Added Distributed tracing implemented in platform-core and rest-automation Improved HTTP header transformation for rest-automation Removed N/A Changed language pack API key obtained from environment variable Version 1.12.8, 8/15/2019 Added N/A Removed rest-core subproject has been merged with rest-spring Changed N/A Version 1.12.7, 7/15/2019 Added Periodic routing table integrity check (15 minutes) Set kafka read pointer to the beginning for new application instances except presence monitor REST automation helper application in the \"extensions\" project Support service discovery of multiple routes in the updated PostOffice's exists() method logback to set log level based on environment variable LOG_LEVEL (default is INFO) Removed N/A Changed Minor refactoring of kafka-connector and hazelcast-connector to ensure that they can coexist if you want to include both of these dependencies in your project. This is for convenience of dev and testing. In production, please select only one cloud connector library to reduce memory footprint. Version 1.12.4, 6/24/2019 Added Add inactivity expiry timer to ObjectStreamIO so that house-keeper can clean up resources that are idle Removed N/A Changed Disable HTML encape sequence for GSON serializer Bug fix for GSON serialization optimization Bug fix for Object Stream housekeeper By default, GSON serializer converts all numbers to double, resulting in unwanted decimal point for integer and long. To handle custom map serialization for correct representation of numbers, an unintended side effect was introduced in earlier releases. List of inner PoJo would be incorrectly serialized as map, resulting in casting exception. This release resolves this issue. Version 1.12.1, 6/10/2019 Added Store-n-forward pub/sub API will be automatically enabled if the underlying cloud connector supports it. e.g. kafka ObjectStreamIO, a convenient wrapper class, to provide event stream I/O API. Object stream feature is now a standard feature instead of optional. Deferred delivery added to language connector. Removed N/A Changed N/A Version 1.11.40, 5/25/2019 Added Route substitution for simple versioning use case Add \"Strict Transport Security\" header if HTTPS (https://tools.ietf.org/html/rfc6797) Event stream connector for Kafka Distributed housekeeper feature for Hazelcast connector Removed System log service Changed Refactoring of Hazelcast event stream connector library to sync up with the new Kafka connector. Version 1.11.39, 4/30/2019 Added Language-support service application for Python, Node.js and Go, etc. Python language pack project is available at https://github.com/Accenture/mercury-python Removed N/A Changed replace Jackson serialization engine with Gson ( platform-core project) replace Apache HttpClient with Google Http Client ( rest-spring ) remove Jackson dependencies from Spring Boot ( rest-spring ) interceptor improvement Version 1.11.33, 3/25/2019 Added N/A Removed N/A Changed Move safe.data.models validation rules from EventEnvelope to SimpleMapper Apache fluent HTTP client downgraded to version 4.5.6 because the pom file in 4.5.7 is invalid Version 1.11.30, 3/7/2019 Added Added retry logic in persistent queue when OS cannot update local file metadata in real-time for Windows based machine. Removed N/A Changed pom.xml changes - update with latest 3rd party open sources dependencies. Version 1.11.29, 1/25/2019 Added platform-core Support for long running functions so that any long queries will not block the rest of the system. \"safe.data.models\" is available as an option in the application.properties. This is an additional security measure to protect against Jackson deserialization vulnerability. See example below: # # additional security to protect against model injection # comma separated list of model packages that are considered safe to be used for object deserialization # #safe.data.models=com.accenture.models rest-spring \"/env\" endpoint is added. See sample application.properties below: # # environment and system properties to be exposed to the \"/env\" admin endpoint # show.env.variables=USER, TEST show.application.properties=server.port, cloud.connector Removed N/A Changed platform-core Use Java Future and an elastic cached thread pool for executing user functions. Fixed N/A Version 1.11.28, 12/20/2018 Added Hazelcast support is added. This includes two projects (hazelcast-connector and hazelcast-presence). Hazelcast-connector is a cloud connector library. Hazelcast-presence is the \"Presence Monitor\" for monitoring the presence status of each application instance. Removed platform-core The \"fixed resource manager\" feature is removed because the same outcome can be achieved at the application level. e.g. The application can broadcast requests to multiple application instances with the same route name and use a callback function to receive response asynchronously. The services can provide resource metrics so that the caller can decide which is the most available instance to contact. For simplicity, resources management is better left to the cloud platform or the application itself. Changed N/A Fixed N/A","title":"Release notes"},{"location":"CHANGELOG/#changelog","text":"","title":"Changelog"},{"location":"CHANGELOG/#release-notes","text":"All notable changes to this project will be documented in this file. The format is based on Keep a Changelog , and this project adheres to Semantic Versioning .","title":"Release notes"},{"location":"CHANGELOG/#version-411-12182024","text":"","title":"Version 4.1.1, 12/18/2024"},{"location":"CHANGELOG/#added","text":"Added \"map\" constant type in input data mapping.","title":"Added"},{"location":"CHANGELOG/#removed","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed","text":"Updated Chapter-4 for the new \"map\" constant feature.","title":"Changed"},{"location":"CHANGELOG/#version-410-12112024","text":"This milestone version achieves ideal event choreography by removing additional event routing to and from the Event Manager. This would boost internal event routing performance by 50 percent.","title":"Version 4.1.0, 12/11/2024"},{"location":"CHANGELOG/#added_1","text":"Performance optimization for Event Script","title":"Added"},{"location":"CHANGELOG/#removed_1","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_1","text":"The platform-core module uses virtual threads to execute event.script.manager and task.executor directly to eliminate additional serialization overheads since the two functions are event routers themselves.","title":"Changed"},{"location":"CHANGELOG/#version-4033-12112024","text":"","title":"Version 4.0.33, 12/11/2024"},{"location":"CHANGELOG/#added_2","text":"Support of custom content types in application.yml","title":"Added"},{"location":"CHANGELOG/#removed_2","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_2","text":"Improved websocket housekeeping logic Use bench.add to replace bench.offer API","title":"Changed"},{"location":"CHANGELOG/#version-4032-1292024","text":"","title":"Version 4.0.32, 12/9/2024"},{"location":"CHANGELOG/#added_3","text":"For completeness, added Boolean AND and OR operations for simple type mapping. Added traceId as metadata for a flow instance","title":"Added"},{"location":"CHANGELOG/#removed_3","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_3","text":"Update Chapter-4 for the new AND/OR type mapping feature Consistent custom HTTP headers for event over http protocol and streaming content","title":"Changed"},{"location":"CHANGELOG/#version-4031-1252024","text":"","title":"Version 4.0.31, 12/5/2024"},{"location":"CHANGELOG/#added_4","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_4","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_4","text":"The \"keep.original\" key is renamed as \"keep-original\" to comply with convention. Continue processing if some preload override config files are missing.","title":"Changed"},{"location":"CHANGELOG/#version-4030-1252024","text":"","title":"Version 4.0.30, 12/5/2024"},{"location":"CHANGELOG/#added_5","text":"Implemented unique task naming feature for event flow configuration.","title":"Added"},{"location":"CHANGELOG/#removed_5","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_5","text":"The \"keep_original\" key is renamed as \"keep.original\" in preload override Chapter-4 of developer guide updated with the new task alias feature","title":"Changed"},{"location":"CHANGELOG/#version-4029-1232024","text":"","title":"Version 4.0.29, 12/3/2024"},{"location":"CHANGELOG/#added_6","text":"Added integer, long, float, double and boolean type matching for state machine.","title":"Added"},{"location":"CHANGELOG/#removed_6","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_6","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-4028-11292024","text":"","title":"Version 4.0.28, 11/29/2024"},{"location":"CHANGELOG/#added_7","text":"Support for simple data type matching processing (text, substring, binary and b64) Optional external state machine","title":"Added"},{"location":"CHANGELOG/#removed_7","text":"Removed \"http.input.\" and \"http.output.\" aliases from event script. Instead, use the generic \"input.\" and \"output.\" namespaces.","title":"Removed"},{"location":"CHANGELOG/#changed_7","text":"Bugfix for AsyncHttpClient to allow missing HTTP request body in POST, PUT or PATCH request Mono reactive flow control","title":"Changed"},{"location":"CHANGELOG/#version-4027-11272024","text":"","title":"Version 4.0.27, 11/27/2024"},{"location":"CHANGELOG/#added_8","text":"Support for Mono/Flux return type for KotlinLambdaFunction Implemented Websocket handshake handler to adjust to API changes in vertx 4.5.11","title":"Added"},{"location":"CHANGELOG/#removed_8","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_8","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-4026-11262024","text":"","title":"Version 4.0.26, 11/26/2024"},{"location":"CHANGELOG/#added_9","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_9","text":"Remove pom.xml version override for netty and spring framework because Spring Boot 3.4.0 fetches the correct versions of netty and spring framework. Earlier override was done to avoid security vulnerabilities of older versions of netty and spring framework.","title":"Removed"},{"location":"CHANGELOG/#changed_9","text":"Handle the case that Mono will not return payload if the payload is null OSS update: Classgraph 4.8.179, Vertx 4.5.11, Spring Boot 3.4.0, Kafka Client 3.9.0","title":"Changed"},{"location":"CHANGELOG/#version-4025-11212024","text":"","title":"Version 4.0.25, 11/21/2024"},{"location":"CHANGELOG/#added_10","text":"Support more than one REST configuration files. When a duplicated REST entry is detected, the system will abort REST endpoint rendering and print out an error message in application log. If you have unit tests to cover the REST endpoints, the unit tests will fail accordingly.","title":"Added"},{"location":"CHANGELOG/#removed_10","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_10","text":"Improved environment variable parsing in config reader. System will skip entries with invalid environment variable reference syntax.","title":"Changed"},{"location":"CHANGELOG/#version-4024-11202024","text":"","title":"Version 4.0.24, 11/20/2024"},{"location":"CHANGELOG/#added_11","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_11","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_11","text":"Bugfix for an edge case in config reader to handle control character of brackets inside an environment variable reference. e.g. some.key=${ENV_VAR:something/{test1}/{test2}}","title":"Changed"},{"location":"CHANGELOG/#version-4023-11192024","text":"","title":"Version 4.0.23, 11/19/2024"},{"location":"CHANGELOG/#added_12","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_12","text":"ObjectStreamWriter and AsyncObjectStreamReader are removed","title":"Removed"},{"location":"CHANGELOG/#changed_12","text":"Replace ObjectStreamWriter with FluxPublisher Replace AsyncObjectStreamReader with FluxConsumer Bugfix for FluxConsumer expiry - change type from \"data\" to \"exception\".","title":"Changed"},{"location":"CHANGELOG/#version-4022-11182024","text":"","title":"Version 4.0.22, 11/18/2024"},{"location":"CHANGELOG/#added_13","text":"FluxPublisher and FluxConsumer for integration with Flux reactive response object","title":"Added"},{"location":"CHANGELOG/#removed_13","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_13","text":"Unit tests in event streaming and post office to support Flux integration Select reactor-core version 3.7.0 using dependency management (reactor-bom version 2024.0.0)","title":"Changed"},{"location":"CHANGELOG/#version-4021-11142024","text":"","title":"Version 4.0.21, 11/14/2024"},{"location":"CHANGELOG/#added_14","text":"Support for user function to return a Mono reactive response object","title":"Added"},{"location":"CHANGELOG/#removed_14","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_14","text":"Update netty to version 4.1.115.Final to address security vulnerability in 4.1.114 Move reactor-core library from rest-spring-3 to platform-core","title":"Changed"},{"location":"CHANGELOG/#version-4020-11132024","text":"","title":"Version 4.0.20, 11/13/2024"},{"location":"CHANGELOG/#added_15","text":"For ease of configuration, added \"com.accenture\" to the base packages so that user applications do not need to include it to use the event-script-engine module.","title":"Added"},{"location":"CHANGELOG/#removed_15","text":"if-then-else pipeline feature in event-script","title":"Removed"},{"location":"CHANGELOG/#changed_15","text":"Update Event Script syntax for consistency Fix error in counting number of compiled flows","title":"Changed"},{"location":"CHANGELOG/#version-4016-11102024","text":"","title":"Version 4.0.16, 11/10/2024"},{"location":"CHANGELOG/#added_16","text":"Generate unique flow instance ID as reference during flow execution.","title":"Added"},{"location":"CHANGELOG/#removed_16","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_16","text":"Save the original correlation-ID from the calling party in a flow instance and return this value to the calling party at the end of flow execution.","title":"Changed"},{"location":"CHANGELOG/#version-4015-1172024","text":"","title":"Version 4.0.15, 11/7/2024"},{"location":"CHANGELOG/#added_17","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_17","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_17","text":"renamed StartFlow to FlowExecutor","title":"Changed"},{"location":"CHANGELOG/#version-4014-1172024","text":"","title":"Version 4.0.14, 11/7/2024"},{"location":"CHANGELOG/#added_18","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_18","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_18","text":"Health check function can return either a text string or a Map StartFlow API updates","title":"Changed"},{"location":"CHANGELOG/#version-4013-1152024","text":"","title":"Version 4.0.13, 11/5/2024"},{"location":"CHANGELOG/#added_19","text":"Added helper class \"StartFlow\" to start a flow, including internal flows without HTTP or Kafka.","title":"Added"},{"location":"CHANGELOG/#removed_19","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_19","text":"Bugfix for empty YAML file to avoid null pointer exception Sort event scripts for orderly logging in the CompileFlows validation process","title":"Changed"},{"location":"CHANGELOG/#version-4012-10312024","text":"","title":"Version 4.0.12, 10/31/2024"},{"location":"CHANGELOG/#added_20","text":"New feature to support resolution of more than one environment variable for a parameter using the ConfigReader","title":"Added"},{"location":"CHANGELOG/#removed_20","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_20","text":"Update OSS modules 1. classgraph version 4.8.177 2. kotlin version 2.0.21 3. guava version 33.3.1-jre 4. jUnit version 5 jupiter Adjusted all unit tests to use jUnit 5","title":"Changed"},{"location":"CHANGELOG/#version-4011-10282024","text":"","title":"Version 4.0.11, 10/28/2024"},{"location":"CHANGELOG/#added_21","text":"New features to support: 1. multiple preload override config file 2. multiple flow list config files","title":"Added"},{"location":"CHANGELOG/#removed_21","text":"unused class \"UnauthorizedObj\" in platform-core commons-io dependency in Kafka-Standalone subproject","title":"Removed"},{"location":"CHANGELOG/#changed_21","text":"Unit test for the preload override feature JavaDoc for the MainApplication","title":"Changed"},{"location":"CHANGELOG/#version-4010-10242024","text":"","title":"Version 4.0.10, 10/24/2024"},{"location":"CHANGELOG/#added_22","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_22","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_22","text":"OSS update - Spring Boot 3.3.5 Security patch for CR/LF exploit for HTTP cookie","title":"Changed"},{"location":"CHANGELOG/#version-409-10182024","text":"","title":"Version 4.0.9, 10/18/2024"},{"location":"CHANGELOG/#added_23","text":"Added Kafka Raft for the Kafka-standalone app.","title":"Added"},{"location":"CHANGELOG/#removed_23","text":"Removed zookeeper from Kafka-standalone app.","title":"Removed"},{"location":"CHANGELOG/#changed_23","text":"Update spring framework verison 6.1.14 to avoid vulnerability in webflux","title":"Changed"},{"location":"CHANGELOG/#version-408-1092024","text":"","title":"Version 4.0.8, 10/9/2024"},{"location":"CHANGELOG/#added_24","text":"Partial support of Active Profile using the \"spring.profiles.active\" parameter Hierarchy of flows","title":"Added"},{"location":"CHANGELOG/#removed_24","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_24","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-407-1012024","text":"","title":"Version 4.0.7, 10/1/2024"},{"location":"CHANGELOG/#added_25","text":"A generic \"no-op\" function for use in event scripts.","title":"Added"},{"location":"CHANGELOG/#removed_25","text":"Feature to ping a function without payload and headers.","title":"Removed"},{"location":"CHANGELOG/#changed_25","text":"Simplified api-playground application","title":"Changed"},{"location":"CHANGELOG/#version-406-9272024","text":"","title":"Version 4.0.6, 9/27/2024"},{"location":"CHANGELOG/#added_26","text":"HTTP request Cookie value filtering using RFC-6265 strict syntax","title":"Added"},{"location":"CHANGELOG/#removed_26","text":"Automatic index page redirection filter for Spring Boot","title":"Removed"},{"location":"CHANGELOG/#changed_26","text":"Upgrade SHA-1 to SHA-512 algorithm in CryptoAPI utility Fix security vulnerability associated with HTTP request header and cookie manipulation","title":"Changed"},{"location":"CHANGELOG/#version-405-9242024","text":"","title":"Version 4.0.5, 9/24/2024"},{"location":"CHANGELOG/#added_27","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_27","text":"Feature for automatic PoJo transport in EventEnvelope and MsgPack Feature for safe.data.model deserialization Benchmark-server is no longer required","title":"Removed"},{"location":"CHANGELOG/#changed_27","text":"Update OSS versions - vertx 4.5.10, kotlin 2.0.20, spring boot 3.3.4","title":"Changed"},{"location":"CHANGELOG/#version-404-952024","text":"","title":"Version 4.0.4, 9/5/2024"},{"location":"CHANGELOG/#added_28","text":"New feature for AsyncHttpClient to render small streaming HTTP response (i.e. chunked binary data) as byte array. For details, Please refer to Appendix III, Developer Guide","title":"Added"},{"location":"CHANGELOG/#removed_28","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_28","text":"Bugfix for parsing default value of environment variable in ConfigReader. This resolves an issue when the special character colon (\":\") is used more than once in the default value.","title":"Changed"},{"location":"CHANGELOG/#version-403-942024","text":"","title":"Version 4.0.3, 9/4/2024"},{"location":"CHANGELOG/#added_29","text":"The \"preload override\" feature is added. This allows overriding a reusable composable library with a set of new route names that are unique for use in an event flow configuration script. For details, Please refer to Chapter 4, Developer Guide","title":"Added"},{"location":"CHANGELOG/#removed_29","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_29","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-402-8312024","text":"","title":"Version 4.0.2, 8/31/2024"},{"location":"CHANGELOG/#added_30","text":"New \"classpath\" namespace for input data mapping Support for input data mapping to handle subset of input request body as a Map or PoJo","title":"Added"},{"location":"CHANGELOG/#removed_30","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_30","text":"Remove the class \"type\" variable from AsyncHttpRequest Improve the \"removeElement\" method in MultiLevelMap Make HTTP input request header labels key-insensitive Update Spring Boot to version 3.3.3","title":"Changed"},{"location":"CHANGELOG/#version-401-8192024","text":"","title":"Version 4.0.1, 8/19/2024"},{"location":"CHANGELOG/#added_31","text":"new File read/write feature in Event Script's I/O data mapping","title":"Added"},{"location":"CHANGELOG/#removed_31","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_31","text":"Update Spring Boot to version 3.3.2 Update Guava to version 33.3.0-jre Update Vertx to version 4.5.9 Update Kotlin to version 2.0.10 Change \"upstream\" to \"dependency\" in the \"/health\" endpoint","title":"Changed"},{"location":"CHANGELOG/#version-400-6242024","text":"This version merges Event Script into the Mercury Composable repository.","title":"Version 4.0.0, 6/24/2024"},{"location":"CHANGELOG/#added_32","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_32","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_32","text":"Update Spring Boot to version 3.3.1 Update Guava to version 33.2.1-jre Update Vertx to version 4.5.8 Update Kotlin to version 2.0.0 Update classgraph to version 4.8.174 Optional reply event for a flow configuration Kafka-standalone is still using Spring Boot 3.2.5 due to compatibility issue","title":"Changed"},{"location":"CHANGELOG/#version-315-512024","text":"This version supercedes 3.1.4 due to updated data structure for static content handling.","title":"Version 3.1.5, 5/1/2024"},{"location":"CHANGELOG/#added_33","text":"Added optional static-content.no-cache-pages in rest.yaml AsyncHttpClientLoader","title":"Added"},{"location":"CHANGELOG/#removed_33","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_33","text":"Updated data structure for static-content section in rest.yaml Fixed bug for setting multiple HTTP cookies Unified configuration file prefix \"yaml.\"","title":"Changed"},{"location":"CHANGELOG/#version-314-4282024","text":"","title":"Version 3.1.4, 4/28/2024"},{"location":"CHANGELOG/#added_34","text":"Added optional static content HTTP-GET request filter in rest.yaml","title":"Added"},{"location":"CHANGELOG/#removed_34","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_34","text":"Updated syntax for static-content-filter","title":"Changed"},{"location":"CHANGELOG/#version-313-4242024","text":"","title":"Version 3.1.3, 4/24/2024"},{"location":"CHANGELOG/#added_35","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_35","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_35","text":"Enhanced OptionalService annotation.","title":"Changed"},{"location":"CHANGELOG/#version-312-4172024","text":"","title":"Version 3.1.2, 4/17/2024"},{"location":"CHANGELOG/#added_36","text":"Added \"app-config-reader.yml\" file in the resources folder so that you can override the default application configuration files.","title":"Added"},{"location":"CHANGELOG/#removed_36","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_36","text":"Open sources library update (Spring Boot 3.2.5, Vertx 4.5.7) Improve AppConfigReader and ConfigReader to use the app-config-reader.yml file. Enhanced OptionalService annotation.","title":"Changed"},{"location":"CHANGELOG/#version-311-282024","text":"","title":"Version 3.1.1, 2/8/2024"},{"location":"CHANGELOG/#added_37","text":"AutoStart to run application as Spring Boot if the rest-spring-3 library is packaged in app Configurable \"Event over HTTP\" - automatic forward events over HTTP using a configuration Support user defined serializer with PreLoad annotation and platform API","title":"Added"},{"location":"CHANGELOG/#removed_37","text":"Bugfix: removed websocket client connection timeout that causes the first connection to drop after one minute","title":"Removed"},{"location":"CHANGELOG/#changed_37","text":"Open sources library update (Spring Boot 3.2.2, Vertx 4.5.3 and MsgPack 0.9.8) Rename application parameter \"event.worker.pool\" to \"kernel.thread.pool\"","title":"Changed"},{"location":"CHANGELOG/#version-310-152024","text":"","title":"Version 3.1.0, 1/5/2024"},{"location":"CHANGELOG/#added_38","text":"Full integration with Java 21 Virtual Thread Default execution mode is set to \"virtual thread\" KernelThreadRunner annotation added to provide optional support of kernel threads","title":"Added"},{"location":"CHANGELOG/#removed_38","text":"Retired Spring Boot version 2 Hazelcast and ActiveMQ network connectors","title":"Removed"},{"location":"CHANGELOG/#changed_38","text":"platform-core engine updated with virtual thread","title":"Changed"},{"location":"CHANGELOG/#version-307-12232023","text":"","title":"Version 3.0.7, 12/23/2023"},{"location":"CHANGELOG/#added_39","text":"Print out basic JVM information before startup for verification of base container image.","title":"Added"},{"location":"CHANGELOG/#removed_39","text":"Removed Maven Shade packager","title":"Removed"},{"location":"CHANGELOG/#changed_39","text":"Updated open sources libraries to address security vulnerabilities Spring Boot 2/3 to version 2.7.18 and 3.2.1 respectively Tomcat 9.0.84 Vertx 4.5.1 Classgraph 4.8.165 Netty 4.1.104.Final slf4j API 2.0.9 log4j2 2.22.0 Kotlin 1.9.22 Artemis 2.31.2 Hazelcast 5.3.6 Guava 33.0.0-jre","title":"Changed"},{"location":"CHANGELOG/#version-306-10262023","text":"","title":"Version 3.0.6, 10/26/2023"},{"location":"CHANGELOG/#added_40","text":"Enhanced Benchmark tool to support \"Event over HTTP\" protocol to evaluate performance efficiency for commmunication between application containers using HTTP.","title":"Added"},{"location":"CHANGELOG/#removed_40","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_40","text":"Updated open sources libraries Spring Boot 2/3 to version 2.7.17 and 3.1.5 respectively Kafka-client 3.6.0","title":"Changed"},{"location":"CHANGELOG/#version-305-10212023","text":"","title":"Version 3.0.5, 10/21/2023"},{"location":"CHANGELOG/#added_41","text":"Support two executable JAR packaging system: 1. Maven Shade packager 2. Spring Boot packager Starting from version 3.0.5, we have replaced Spring Boot packager with Maven Shade. This avoids a classpath edge case for Spring Boot packager when running kafka-client under Java 11 or higher. Maven Shade also results in smaller executable JAR size.","title":"Added"},{"location":"CHANGELOG/#removed_41","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_41","text":"Updated open sources libraries Spring-Boot 2.7.16 / 3.1.4 classgraph 4.8.163 snakeyaml 2.2 kotlin 1.9.10 vertx 4.4.6 guava 32.1.3-jre msgpack 0.9.6 slj4j 2.0.9 zookeeper 3.7.2 The \"/info/lib\" admin endpoint has been enhanced to list library dependencies for executable JAR generated by either Maven Shade or Spring Boot Packager. Improved ConfigReader to recognize both \".yml\" and \".yaml\" extensions and their uses are interchangeable.","title":"Changed"},{"location":"CHANGELOG/#version-304-862023","text":"","title":"Version 3.0.4, 8/6/2023"},{"location":"CHANGELOG/#added_42","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_42","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_42","text":"Updated open sources libraries Spring-Boot 2.7.14 / 3.1.2 Kafka-client 3.5.1 classgraph 4.8.161 guava 32.1.2-jre msgpack 0.9.5","title":"Changed"},{"location":"CHANGELOG/#version-303-6272023","text":"","title":"Version 3.0.3, 6/27/2023"},{"location":"CHANGELOG/#added_43","text":"File extension to MIME type mapping for static HTML file handling","title":"Added"},{"location":"CHANGELOG/#removed_43","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_43","text":"Open sources library update - Kotlin version 1.9.0","title":"Changed"},{"location":"CHANGELOG/#version-302-692023","text":"","title":"Version 3.0.2, 6/9/2023"},{"location":"CHANGELOG/#added_44","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_44","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_44","text":"Consistent exception handling for Event API endpoint Open sources lib update - Vertx 4.4.4, Spring Boot 2.7.13, Spring Boot 3.1.1, classgraph 4.8.160, guava 32.0.1-jre","title":"Changed"},{"location":"CHANGELOG/#version-301-652023","text":"In this release, we have replace Google HTTP Client with vertx non-blocking WebClient. We also tested compatibility up to OpenJDK version 20 and maven 3.9.2.","title":"Version 3.0.1, 6/5/2023"},{"location":"CHANGELOG/#added_45","text":"When \"x-raw-xml\" HTTP request header is set to \"true\", the AsyncHttpClient will skip the built-in XML serialization so that your application can retrieve the original XML text.","title":"Added"},{"location":"CHANGELOG/#removed_45","text":"Retire Google HTTP client","title":"Removed"},{"location":"CHANGELOG/#changed_45","text":"Upgrade maven plugin versions.","title":"Changed"},{"location":"CHANGELOG/#version-300-4182023","text":"This is a major release with some breaking changes. Please refer to Chapter-10 (Migration guide) for details. This version brings the best of preemptive and cooperating multitasking to Java (version 1.8 to 19) before Java 19 virtual thread feature becomes officially available.","title":"Version 3.0.0, 4/18/2023"},{"location":"CHANGELOG/#added_46","text":"Function execution engine supporting kernel thread pool, Kotlin coroutine and suspend function \"Event over HTTP\" service for inter-container communication Support for Spring Boot version 3 and WebFlux Sample code for a pre-configured Spring Boot 3 application","title":"Added"},{"location":"CHANGELOG/#removed_46","text":"Remove blocking APIs from platform-core Retire PM2 process manager sample script due to compatibility issue","title":"Removed"},{"location":"CHANGELOG/#changed_46","text":"Refactor \"async.http.request\" to use vertx web client for non-blocking operation Update log4j2 version 2.20.0 and slf4j version 2.0.7 in platform-core Update JBoss RestEasy JAX_RS to version 3.15.6.Final in rest-spring Update vertx to 4.4.2 Update Spring Boot parent pom to 2.7.12 and 3.1.0 for spring boot 2 and 3 respectively Remove com.fasterxml.classmate dependency from rest-spring","title":"Changed"},{"location":"CHANGELOG/#version-280-3202023","text":"","title":"Version 2.8.0, 3/20/2023"},{"location":"CHANGELOG/#added_47","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_47","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_47","text":"Improved load balancing in cloud-connector Filter URI to avoid XSS attack Upgrade to SnakeYaml 2.0 and patch Spring Boot 2.6.8 for compatibility with it Upgrade to Vertx 4.4.0, classgraph 4.8.157, tomcat 9.0.73","title":"Changed"},{"location":"CHANGELOG/#version-271-12222022","text":"","title":"Version 2.7.1, 12/22/2022"},{"location":"CHANGELOG/#added_48","text":"standalone benchmark report app client and server benchmark apps add timeout tag to RPC events","title":"Added"},{"location":"CHANGELOG/#removed_48","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_48","text":"Updated open sources dependencies Netty 4.1.86.Final Tomcat 9.0.69 Vertx 4.3.6 classgraph 4.8.152 google-http-client 1.42.3 Improved unit tests to use assertThrows to evaluate exception Enhanced AsyncHttpRequest serialization","title":"Changed"},{"location":"CHANGELOG/#version-270-11112022","text":"In this version, REST automation code is moved to platform-core such that REST and Websocket service can share the same port.","title":"Version 2.7.0, 11/11/2022"},{"location":"CHANGELOG/#added_49","text":"AsyncObjectStreamReader is added for non-blocking read operation from an object stream. Support of LocalDateTime in SimpleMapper Add \"removeElement\" method to MultiLevelMap Automatically convert a map to a PoJo when the sender does not specify class in event body","title":"Added"},{"location":"CHANGELOG/#removed_49","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_49","text":"REST automation becomes part of platform-core and it can co-exist with Spring Web in the rest-spring module Enforce Spring Boot lifecycle management such that user apps will start after Spring Boot has loaded all components Update netty to version 4.1.84.Final","title":"Changed"},{"location":"CHANGELOG/#version-260-10132022","text":"In this version, websocket notification example code has been removed from the REST automation system. If your application uses this feature, please recover the code from version 2.5.0 and refactor it as a separate library.","title":"Version 2.6.0, 10/13/2022"},{"location":"CHANGELOG/#added_50","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_50","text":"Simplify REST automation system by removing websocket notification example in REST automation.","title":"Removed"},{"location":"CHANGELOG/#changed_50","text":"Replace Tomcat websocket server with Vertx non-blocking websocket server library Update netty to version 4.1.79.Final Update kafka client to version 2.8.2 Update snake yaml to version 1.33 Update gson to version 2.9.1","title":"Changed"},{"location":"CHANGELOG/#version-250-9102022","text":"","title":"Version 2.5.0, 9/10/2022"},{"location":"CHANGELOG/#added_51","text":"New Preload annotation class to automate pre-registration of LambdaFunction.","title":"Added"},{"location":"CHANGELOG/#removed_51","text":"Removed Spring framework and Tomcat dependencies from platform-core so that the core library can be applied to legacy J2EE application without library conflict.","title":"Removed"},{"location":"CHANGELOG/#changed_51","text":"Bugfix for proper housekeeping of future events. Make Gson and MsgPack handling of integer/long consistent Updated open sources libraries. Eclipse vertx-core version 4.3.4 MsgPack version 0.9.3 Google httpclient version 1.42.2 SnakeYaml version 1.31","title":"Changed"},{"location":"CHANGELOG/#version-236-6212022","text":"","title":"Version 2.3.6, 6/21/2022"},{"location":"CHANGELOG/#added_52","text":"Support more than one event stream cluster. User application can share the same event stream cluster for pub/sub or connect to an alternative cluster for pub/sub use cases.","title":"Added"},{"location":"CHANGELOG/#removed_52","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_52","text":"Cloud connector libraries update to Hazelcast 5.1.2","title":"Changed"},{"location":"CHANGELOG/#version-235-5302022","text":"","title":"Version 2.3.5, 5/30/2022"},{"location":"CHANGELOG/#added_53","text":"Add tagging feature to handle language connector's routing and exception handling","title":"Added"},{"location":"CHANGELOG/#removed_53","text":"Remove language pack's pub/sub broadcast feature","title":"Removed"},{"location":"CHANGELOG/#changed_53","text":"Update Spring Boot parent to version 2.6.8 to fetch Netty 4.1.77 and Spring Framework 5.3.20 Streamlined language connector transport protocol for compatibility with both Python and Node.js","title":"Changed"},{"location":"CHANGELOG/#version-234-5142022","text":"","title":"Version 2.3.4, 5/14/2022"},{"location":"CHANGELOG/#added_54","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_54","text":"Remove swagger-ui distribution from api-playground such that developer can clone the latest version","title":"Removed"},{"location":"CHANGELOG/#changed_54","text":"Update application.properties (from spring.resources.static-locations to spring.web.resources.static-locations) Update log4j, Tomcat and netty library version using Spring parent 2.6.6","title":"Changed"},{"location":"CHANGELOG/#version-233-3302022","text":"","title":"Version 2.3.3, 3/30/2022"},{"location":"CHANGELOG/#added_55","text":"Enhanced AsyncRequest to handle non-blocking fork-n-join","title":"Added"},{"location":"CHANGELOG/#removed_55","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_55","text":"Upgrade Spring Boot from 2.6.3 to 2.6.6","title":"Changed"},{"location":"CHANGELOG/#version-232-2212022","text":"","title":"Version 2.3.2, 2/21/2022"},{"location":"CHANGELOG/#added_56","text":"Add support of queue API in native pub/sub module for improved ESB compatibility","title":"Added"},{"location":"CHANGELOG/#removed_56","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_56","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-231-2192022","text":"","title":"Version 2.3.1, 2/19/2022"},{"location":"CHANGELOG/#added_57","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_57","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_57","text":"Update Vertx to version 4.2.4 Update Tomcat to version 5.0.58 Use Tomcat websocket server for presence monitors Bugfix - Simple Scheduler's leader election searches peers correctly","title":"Changed"},{"location":"CHANGELOG/#version-230-1282022","text":"","title":"Version 2.3.0, 1/28/2022"},{"location":"CHANGELOG/#added_58","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_58","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_58","text":"Update copyright notice Update Vertx to version 4.2.3 Bugfix - RSA key generator supporting key length from 1024 to 4096 bits CryptoAPI - support different AES algorithms and custom IV Update Spring Boot to version 2.6.3","title":"Changed"},{"location":"CHANGELOG/#version-223-12292021","text":"","title":"Version 2.2.3, 12/29/2021"},{"location":"CHANGELOG/#added_59","text":"Transaction journaling Add parameter distributed.trace.aggregation in application.properties such that trace aggregation may be disabled.","title":"Added"},{"location":"CHANGELOG/#removed_59","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_59","text":"Update JBoss RestEasy library to 3.15.3.Final Improved po.search(route) to scan local and remote service registries. Added \"remoteOnly\" selection. Fix bug in releasing presence monitor topic for specific closed user group Update Apache log4j to version 2.17.1 Update Spring Boot parent to version 2.6.1 Update Netty to version 4.1.72.Final Update Vertx to version 4.2.2 Convenient class \"UserNotification\" for backend service to publish events to the UI when REST automation is deployed","title":"Changed"},{"location":"CHANGELOG/#version-222-11122021","text":"","title":"Version 2.2.2, 11/12/2021"},{"location":"CHANGELOG/#added_60","text":"User defined API authentication functions can be selected using custom HTTP request header \"Exception chaining\" feature in EventEnvelope New \"deferred.commit.log\" parameter for backward compatibility with older PowerMock in unit tests","title":"Added"},{"location":"CHANGELOG/#removed_60","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_60","text":"Improved and streamlined SimpleXmlParser to handle arrays Bugfix for file upload in Service Gateway (REST automation library) Update Tomcat library from 9.0.50 to 9.0.54 Update Spring Boot library to 2.5.6 Update GSON library to 2.8.9","title":"Changed"},{"location":"CHANGELOG/#version-221-1012021","text":"","title":"Version 2.2.1, 10/1/2021"},{"location":"CHANGELOG/#added_61","text":"Callback function can implement ServiceExceptionHandler to catch exception. It adds the onError() method.","title":"Added"},{"location":"CHANGELOG/#removed_61","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_61","text":"Open sources library update - Vert.x 4.1.3, Netty 4.1.68-Final","title":"Changed"},{"location":"CHANGELOG/#version-211-9102021","text":"","title":"Version 2.1.1, 9/10/2021"},{"location":"CHANGELOG/#added_62","text":"User defined PoJo and Generics mapping Standardized serializers for default case, snake_case and camelCase Support of EventEnvelope as input parameter in TypedLambdaFunction so application function can inspect event's metadata Application can subscribe to life cycle events of other application instances","title":"Added"},{"location":"CHANGELOG/#removed_62","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_62","text":"Replace Tomcat websocket server engine with Vertx in presence monitor for higher performance Bugfix for MsgPack transport of integer, long, BigInteger and BigDecimal","title":"Changed"},{"location":"CHANGELOG/#version-210-7252021","text":"","title":"Version 2.1.0, 7/25/2021"},{"location":"CHANGELOG/#added_63","text":"Multicast - application can define a multicast.yaml config to relay events to more than one target service. StreamFunction - function that allows the application to control back-pressure","title":"Added"},{"location":"CHANGELOG/#removed_63","text":"\"object.streams.io\" route is removed from platform-core","title":"Removed"},{"location":"CHANGELOG/#changed_63","text":"Elastic Queue - Refactored using Oracle Berkeley DB Object stream I/O - simplified design using the new StreamFunction feature Open sources library update - Spring Boot 2.5.2, Tomcat 9.0.50, Vert.x 4.1.1, Netty 4.1.66-Final","title":"Changed"},{"location":"CHANGELOG/#version-200-552021","text":"Vert.x is introduced as the in-memory event bus","title":"Version 2.0.0, 5/5/2021"},{"location":"CHANGELOG/#added_64","text":"ActiveMQ and Tibco connectors Admin endpoints to stop, suspend and resume an application instance Handle edge case to detect stalled application instances Add \"isStreamingPubSub\" method to the PubSub interface","title":"Added"},{"location":"CHANGELOG/#removed_64","text":"Event Node event stream emulator has been retired. You may use standalone Kafka server as a replacement for development and testing in your laptop. Multi-tenancy namespace configuration has been retired. It is replaced by the \"closed user group\" feature.","title":"Removed"},{"location":"CHANGELOG/#changed_64","text":"Refactored Kafka and Hazelcast connectors to support virtual topics and closed user groups. Updated ConfigReader to be consistent with Spring value substitution logic for application properties Replace Akka actor system with Vert.x event bus Common code for various cloud connectors consolidated into cloud core libraries","title":"Changed"},{"location":"CHANGELOG/#version-1130-1152021","text":"Version 1.13.0 is the last version that uses Akka as the in-memory event system.","title":"Version 1.13.0, 1/15/2021"},{"location":"CHANGELOG/#version-11266-1152021","text":"","title":"Version 1.12.66, 1/15/2021"},{"location":"CHANGELOG/#added_65","text":"A simple websocket notification service is integrated into the REST automation system Seamless migration feature is added to the REST automation system","title":"Added"},{"location":"CHANGELOG/#removed_65","text":"Legacy websocket notification example application","title":"Removed"},{"location":"CHANGELOG/#changed_65","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-11265-1292020","text":"","title":"Version 1.12.65, 12/9/2020"},{"location":"CHANGELOG/#added_66","text":"\"kafka.pubsub\" is added as a cloud service File download example in the lambda-example project \"trace.log.header\" added to application.properties - when tracing is enabled, this inserts the trace-ID of the transaction in the log context. For more details, please refer to the Developer Guide Add API to pub/sub engine to support creation of topic with partitions TypedLambdaFunction is added so that developer can predefine input and output classes in a service without casting","title":"Added"},{"location":"CHANGELOG/#removed_66","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_66","text":"Decouple Kafka pub/sub from kafka connector so that native pub/sub can be used when application is running in standalone mode Rename \"relay\" to \"targetHost\" in AsyncHttpRequest data model Enhanced routing table distribution by sending a complete list of route tables, thus reducing network admin traffic.","title":"Changed"},{"location":"CHANGELOG/#version-11264-9282020","text":"","title":"Version 1.12.64, 9/28/2020"},{"location":"CHANGELOG/#added_67","text":"If predictable topic is set, application instances will report their predictable topics as \"instance ID\" to the presence monitor. This improves visibility when a developer tests their application in \"hybrid\" mode. i.e. running the app locally and connect to the cloud remotely for event streams and cloud resources.","title":"Added"},{"location":"CHANGELOG/#removed_67","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_67","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-11263-8272020","text":"","title":"Version 1.12.63, 8/27/2020"},{"location":"CHANGELOG/#added_68","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_68","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_68","text":"Improved Kafka producer and consumer pairing","title":"Changed"},{"location":"CHANGELOG/#version-11262-8122020","text":"","title":"Version 1.12.62, 8/12/2020"},{"location":"CHANGELOG/#added_69","text":"New presence monitor's admin endpoint for the operator to force routing table synchronization (\"/api/ping/now\")","title":"Added"},{"location":"CHANGELOG/#removed_69","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_69","text":"Improved routing table integrity check","title":"Changed"},{"location":"CHANGELOG/#version-11261-882020","text":"","title":"Version 1.12.61, 8/8/2020"},{"location":"CHANGELOG/#added_70","text":"Event stream systems like Kafka assume topic to be used long term. This version adds support to reuse the same topic when an application instance restarts. You can create a predictable topic using unique application name and instance ID. For example, with Kubernetes, you can use the POD name as the unique application instance topic.","title":"Added"},{"location":"CHANGELOG/#removed_70","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_70","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-11256-842020","text":"","title":"Version 1.12.56, 8/4/2020"},{"location":"CHANGELOG/#added_71","text":"Automate trace for fork-n-join use case","title":"Added"},{"location":"CHANGELOG/#removed_71","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_71","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-11255-7192020","text":"","title":"Version 1.12.55, 7/19/2020"},{"location":"CHANGELOG/#added_72","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_72","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_72","text":"Improved distributed trace - set the \"from\" address in EventEnvelope automatically.","title":"Changed"},{"location":"CHANGELOG/#version-11254-7102020","text":"","title":"Version 1.12.54, 7/10/2020"},{"location":"CHANGELOG/#added_73","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_73","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_73","text":"Application life-cycle management - User provided main application(s) will be started after Spring Boot declares web application ready. This ensures correct Spring autowiring or dependencies are available. Bugfix for locale - String.format(float) returns comma as decimal point that breaks number parser. Replace with BigDecimal decimal point scaling. Bugfix for Tomcat 9.0.35 - Change Async servlet default timeout from 30 seconds to -1 so the system can handle the whole life-cycle directly.","title":"Changed"},{"location":"CHANGELOG/#version-11252-6112020","text":"","title":"Version 1.12.52, 6/11/2020"},{"location":"CHANGELOG/#added_74","text":"new \"search\" method in Post Office to return a list of application instances for a service simple \"cron\" job scheduler as an extension project add \"sequence\" to MainApplication annotation for orderly execution when more than one MainApplication is available support \"Optional\" object in EventEnvelope so a LambdaFunction can read and return Optional","title":"Added"},{"location":"CHANGELOG/#removed_74","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_74","text":"The rest-spring library has been updated to support both JAR and WAR deployment All pom.xml files updated accordingly PersistentWsClient will back off for 10 seconds when disconnected by remote host","title":"Changed"},{"location":"CHANGELOG/#version-11250-5202020","text":"","title":"Version 1.12.50, 5/20/2020"},{"location":"CHANGELOG/#added_75","text":"Payload segmentation For large payload in an event, the payload is automatically segmented into 64 KB segments. When there are more than one target application instances, the system ensures that the segments of the same event is delivered to exactly the same target. PersistentWsClient added - generalized persistent websocket client for Event Node, Kafka reporter and Hazelcast reporter.","title":"Added"},{"location":"CHANGELOG/#removed_75","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_75","text":"Code cleaning to improve consistency Upgraded to hibernate-validator to v6.1.5.Final and Hazelcast version 4.0.1 REST automation is provided as a library and an application to handle different use cases","title":"Changed"},{"location":"CHANGELOG/#version-11240-542020","text":"","title":"Version 1.12.40, 5/4/2020"},{"location":"CHANGELOG/#added_76","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_76","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_76","text":"For security reason, upgrade log4j to version 2.13.2","title":"Changed"},{"location":"CHANGELOG/#version-11239-532020","text":"","title":"Version 1.12.39, 5/3/2020"},{"location":"CHANGELOG/#added_77","text":"Use RestEasy JAX-RS library","title":"Added"},{"location":"CHANGELOG/#removed_77","text":"For security reason, removed Jersey JAX-RS library","title":"Removed"},{"location":"CHANGELOG/#changed_77","text":"Updated RestLoader to initialize RestEasy servlet dispatcher Support nested arrays in MultiLevelMap","title":"Changed"},{"location":"CHANGELOG/#version-11236-4162020","text":"","title":"Version 1.12.36, 4/16/2020"},{"location":"CHANGELOG/#added_78","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_78","text":"For simplicity, retire route-substitution admin endpoint. Route substitution uses a simple static table in route-substitution.yaml.","title":"Removed"},{"location":"CHANGELOG/#changed_78","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-11235-4122020","text":"","title":"Version 1.12.35, 4/12/2020"},{"location":"CHANGELOG/#added_79","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_79","text":"SimpleRBAC class is retired","title":"Removed"},{"location":"CHANGELOG/#changed_79","text":"Improved ConfigReader and AppConfigReader with automatic key-value normalization for YAML and JSON files Improved pub/sub module in kafka-connector","title":"Changed"},{"location":"CHANGELOG/#version-11234-3282020","text":"","title":"Version 1.12.34, 3/28/2020"},{"location":"CHANGELOG/#added_80","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_80","text":"Retired proprietary config manager since we can use the \"BeforeApplication\" approach to load config from Kubernetes configMap or other systems of config record.","title":"Removed"},{"location":"CHANGELOG/#changed_80","text":"Added \"isZero\" method to the SimpleMapper class Convert BigDecimal to string without scientific notation (i.e. toPlainString instead of toString) Corresponding unit tests added to verify behavior","title":"Changed"},{"location":"CHANGELOG/#version-11232-3142020","text":"","title":"Version 1.12.32, 3/14/2020"},{"location":"CHANGELOG/#added_81","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_81","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_81","text":"Kafka-connector will shutdown application instance when the EventProducer cannot send event to Kafka. This would allow the infrastructure to restart application instance automatically.","title":"Changed"},{"location":"CHANGELOG/#version-11231-2262020","text":"","title":"Version 1.12.31, 2/26/2020"},{"location":"CHANGELOG/#added_82","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_82","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_82","text":"Kafka-connector now supports external service provider for Kafka properties and credentials. If your application implements a function with route name \"kafka.properties.provider\" before connecting to cloud, the kafka-connector will retrieve kafka credentials on demand. This addresses case when kafka credentials change after application start-up. Interceptors are designed to forward requests and thus they do not generate replies. However, if you implement a function as an EventInterceptor, your function can throw exception just like a regular function and the exception will be returned to the calling function. This makes it easier to write interceptors.","title":"Changed"},{"location":"CHANGELOG/#version-11230-262020","text":"","title":"Version 1.12.30, 2/6/2020"},{"location":"CHANGELOG/#added_83","text":"Expose \"async.http.request\" as a PUBLIC function (\"HttpClient as a service\")","title":"Added"},{"location":"CHANGELOG/#removed_83","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_83","text":"Improved Hazelcast client connection stability Improved Kafka native pub/sub","title":"Changed"},{"location":"CHANGELOG/#version-11229-1102020","text":"","title":"Version 1.12.29, 1/10/2020"},{"location":"CHANGELOG/#added_84","text":"Rest-automation will transport X-Trace-Id from/to Http request/response, therefore extending distributed trace across systems that support the X-Trace-Id HTTP header. Added endpoint and service to shutdown application instance.","title":"Added"},{"location":"CHANGELOG/#removed_84","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_84","text":"Updated SimpleXmlParser with XML External Entity (XXE) injection prevention. Bug fix for hazelcast recovery logic - when a hazelcast node is down, the app instance will restart the hazelcast client and reset routing table correctly. HSTS header insertion is optional so that we can disable it to avoid duplicated header when API gateway is doing it.","title":"Changed"},{"location":"CHANGELOG/#version-11226-142020","text":"","title":"Version 1.12.26, 1/4/2020"},{"location":"CHANGELOG/#added_85","text":"Feature to disable PoJo deserialization so that caller can decide if the result set should be in PoJo or a Map.","title":"Added"},{"location":"CHANGELOG/#removed_85","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_85","text":"Simplified key management for Event Node AsyncHttpRequest case insensitivity for headers, cookies, path parameters and session key-values Make built-in configuration management optional","title":"Changed"},{"location":"CHANGELOG/#version-11219-12282019","text":"","title":"Version 1.12.19, 12/28/2019"},{"location":"CHANGELOG/#added_86","text":"Added HTTP relay feature in rest-automation project","title":"Added"},{"location":"CHANGELOG/#removed_86","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_86","text":"Improved hazelcast retry and peer discovery logic Refactored rest-automation's service gateway module to use AsyncHttpRequest Info endpoint to show routing table of a peer","title":"Changed"},{"location":"CHANGELOG/#version-11217-12162019","text":"","title":"Version 1.12.17, 12/16/2019"},{"location":"CHANGELOG/#added_87","text":"Simple configuration management is added to event-node, hazelcast-presence and kafka-presence monitors Added BeforeApplication annotation - this allows user application to execute some setup logic before the main application starts. e.g. modifying parameters in application.properties Added API playground as a convenient standalone application to render OpenAPI 2.0 and 3.0 yaml and json files Added argument parser in rest-automation helper app to use a static HTML folder in the local file system if arguments -html file_path is given when starting the JAR file.","title":"Added"},{"location":"CHANGELOG/#removed_87","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_87","text":"Kafka publisher timeout value changed from 10 to 20 seconds Log a warning when Kafka takes more than 5 seconds to send an event","title":"Changed"},{"location":"CHANGELOG/#version-11214-11202019","text":"","title":"Version 1.12.14, 11/20/2019"},{"location":"CHANGELOG/#added_88","text":"getRoute() method is added to PostOffice to facilitate RBAC The route name of the current service is added to an outgoing event when the \"from\" field is not present Simple RBAC using YAML configuration instead of code","title":"Added"},{"location":"CHANGELOG/#removed_88","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_88","text":"Updated Spring Boot to v2.2.1","title":"Changed"},{"location":"CHANGELOG/#version-11212-10262019","text":"","title":"Version 1.12.12, 10/26/2019"},{"location":"CHANGELOG/#added_89","text":"Multi-tenancy support for event streams (Hazelcast and Kafka). This allows the use of a single event stream cluster for multiple non-prod environments. For production, it must use a separate event stream cluster for security reason.","title":"Added"},{"location":"CHANGELOG/#removed_89","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_89","text":"logging framework changed from logback to log4j2 (version 2.12.1) Use JSR-356 websocket annotated ClientEndpoint Improved websocket reconnection logic","title":"Changed"},{"location":"CHANGELOG/#version-1129-9142019","text":"","title":"Version 1.12.9, 9/14/2019"},{"location":"CHANGELOG/#added_90","text":"Distributed tracing implemented in platform-core and rest-automation Improved HTTP header transformation for rest-automation","title":"Added"},{"location":"CHANGELOG/#removed_90","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_90","text":"language pack API key obtained from environment variable","title":"Changed"},{"location":"CHANGELOG/#version-1128-8152019","text":"","title":"Version 1.12.8, 8/15/2019"},{"location":"CHANGELOG/#added_91","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_91","text":"rest-core subproject has been merged with rest-spring","title":"Removed"},{"location":"CHANGELOG/#changed_91","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-1127-7152019","text":"","title":"Version 1.12.7, 7/15/2019"},{"location":"CHANGELOG/#added_92","text":"Periodic routing table integrity check (15 minutes) Set kafka read pointer to the beginning for new application instances except presence monitor REST automation helper application in the \"extensions\" project Support service discovery of multiple routes in the updated PostOffice's exists() method logback to set log level based on environment variable LOG_LEVEL (default is INFO)","title":"Added"},{"location":"CHANGELOG/#removed_92","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_92","text":"Minor refactoring of kafka-connector and hazelcast-connector to ensure that they can coexist if you want to include both of these dependencies in your project. This is for convenience of dev and testing. In production, please select only one cloud connector library to reduce memory footprint.","title":"Changed"},{"location":"CHANGELOG/#version-1124-6242019","text":"","title":"Version 1.12.4, 6/24/2019"},{"location":"CHANGELOG/#added_93","text":"Add inactivity expiry timer to ObjectStreamIO so that house-keeper can clean up resources that are idle","title":"Added"},{"location":"CHANGELOG/#removed_93","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_93","text":"Disable HTML encape sequence for GSON serializer Bug fix for GSON serialization optimization Bug fix for Object Stream housekeeper By default, GSON serializer converts all numbers to double, resulting in unwanted decimal point for integer and long. To handle custom map serialization for correct representation of numbers, an unintended side effect was introduced in earlier releases. List of inner PoJo would be incorrectly serialized as map, resulting in casting exception. This release resolves this issue.","title":"Changed"},{"location":"CHANGELOG/#version-1121-6102019","text":"","title":"Version 1.12.1, 6/10/2019"},{"location":"CHANGELOG/#added_94","text":"Store-n-forward pub/sub API will be automatically enabled if the underlying cloud connector supports it. e.g. kafka ObjectStreamIO, a convenient wrapper class, to provide event stream I/O API. Object stream feature is now a standard feature instead of optional. Deferred delivery added to language connector.","title":"Added"},{"location":"CHANGELOG/#removed_94","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_94","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#version-11140-5252019","text":"","title":"Version 1.11.40, 5/25/2019"},{"location":"CHANGELOG/#added_95","text":"Route substitution for simple versioning use case Add \"Strict Transport Security\" header if HTTPS (https://tools.ietf.org/html/rfc6797) Event stream connector for Kafka Distributed housekeeper feature for Hazelcast connector","title":"Added"},{"location":"CHANGELOG/#removed_95","text":"System log service","title":"Removed"},{"location":"CHANGELOG/#changed_95","text":"Refactoring of Hazelcast event stream connector library to sync up with the new Kafka connector.","title":"Changed"},{"location":"CHANGELOG/#version-11139-4302019","text":"","title":"Version 1.11.39, 4/30/2019"},{"location":"CHANGELOG/#added_96","text":"Language-support service application for Python, Node.js and Go, etc. Python language pack project is available at https://github.com/Accenture/mercury-python","title":"Added"},{"location":"CHANGELOG/#removed_96","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_96","text":"replace Jackson serialization engine with Gson ( platform-core project) replace Apache HttpClient with Google Http Client ( rest-spring ) remove Jackson dependencies from Spring Boot ( rest-spring ) interceptor improvement","title":"Changed"},{"location":"CHANGELOG/#version-11133-3252019","text":"","title":"Version 1.11.33, 3/25/2019"},{"location":"CHANGELOG/#added_97","text":"N/A","title":"Added"},{"location":"CHANGELOG/#removed_97","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_97","text":"Move safe.data.models validation rules from EventEnvelope to SimpleMapper Apache fluent HTTP client downgraded to version 4.5.6 because the pom file in 4.5.7 is invalid","title":"Changed"},{"location":"CHANGELOG/#version-11130-372019","text":"","title":"Version 1.11.30, 3/7/2019"},{"location":"CHANGELOG/#added_98","text":"Added retry logic in persistent queue when OS cannot update local file metadata in real-time for Windows based machine.","title":"Added"},{"location":"CHANGELOG/#removed_98","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_98","text":"pom.xml changes - update with latest 3rd party open sources dependencies.","title":"Changed"},{"location":"CHANGELOG/#version-11129-1252019","text":"","title":"Version 1.11.29, 1/25/2019"},{"location":"CHANGELOG/#added_99","text":"platform-core Support for long running functions so that any long queries will not block the rest of the system. \"safe.data.models\" is available as an option in the application.properties. This is an additional security measure to protect against Jackson deserialization vulnerability. See example below: # # additional security to protect against model injection # comma separated list of model packages that are considered safe to be used for object deserialization # #safe.data.models=com.accenture.models rest-spring \"/env\" endpoint is added. See sample application.properties below: # # environment and system properties to be exposed to the \"/env\" admin endpoint # show.env.variables=USER, TEST show.application.properties=server.port, cloud.connector","title":"Added"},{"location":"CHANGELOG/#removed_99","text":"N/A","title":"Removed"},{"location":"CHANGELOG/#changed_99","text":"platform-core Use Java Future and an elastic cached thread pool for executing user functions.","title":"Changed"},{"location":"CHANGELOG/#fixed","text":"N/A","title":"Fixed"},{"location":"CHANGELOG/#version-11128-12202018","text":"","title":"Version 1.11.28, 12/20/2018"},{"location":"CHANGELOG/#added_100","text":"Hazelcast support is added. This includes two projects (hazelcast-connector and hazelcast-presence). Hazelcast-connector is a cloud connector library. Hazelcast-presence is the \"Presence Monitor\" for monitoring the presence status of each application instance.","title":"Added"},{"location":"CHANGELOG/#removed_100","text":"platform-core The \"fixed resource manager\" feature is removed because the same outcome can be achieved at the application level. e.g. The application can broadcast requests to multiple application instances with the same route name and use a callback function to receive response asynchronously. The services can provide resource metrics so that the caller can decide which is the most available instance to contact. For simplicity, resources management is better left to the cloud platform or the application itself.","title":"Removed"},{"location":"CHANGELOG/#changed_100","text":"N/A","title":"Changed"},{"location":"CHANGELOG/#fixed_1","text":"N/A","title":"Fixed"},{"location":"CODE_OF_CONDUCT/","text":"Contributor Covenant Code of Conduct Our Pledge In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. Our Standards Examples of behavior that contributes to creating a positive environment include: Using welcoming and inclusive language Being respectful of differing viewpoints and experiences Gracefully accepting constructive criticism Focusing on what is best for the community Showing empathy towards other community members Examples of unacceptable behavior by participants include: The use of sexualized language or imagery and unwelcome sexual attention or advances Trolling, insulting/derogatory comments, and personal or political attacks Public or private harassment Publishing others' private information, such as a physical or electronic address, without explicit permission Other conduct which could reasonably be considered inappropriate in a professional setting Our Responsibilities Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. Scope This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting Kevin Bader (the current project maintainer). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. Attribution This Code of Conduct is adapted from the Contributor Covenant , version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html","title":"Code of Conduct"},{"location":"CODE_OF_CONDUCT/#contributor-covenant-code-of-conduct","text":"","title":"Contributor Covenant Code of Conduct"},{"location":"CODE_OF_CONDUCT/#our-pledge","text":"In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.","title":"Our Pledge"},{"location":"CODE_OF_CONDUCT/#our-standards","text":"Examples of behavior that contributes to creating a positive environment include: Using welcoming and inclusive language Being respectful of differing viewpoints and experiences Gracefully accepting constructive criticism Focusing on what is best for the community Showing empathy towards other community members Examples of unacceptable behavior by participants include: The use of sexualized language or imagery and unwelcome sexual attention or advances Trolling, insulting/derogatory comments, and personal or political attacks Public or private harassment Publishing others' private information, such as a physical or electronic address, without explicit permission Other conduct which could reasonably be considered inappropriate in a professional setting","title":"Our Standards"},{"location":"CODE_OF_CONDUCT/#our-responsibilities","text":"Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.","title":"Our Responsibilities"},{"location":"CODE_OF_CONDUCT/#scope","text":"This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.","title":"Scope"},{"location":"CODE_OF_CONDUCT/#enforcement","text":"Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting Kevin Bader (the current project maintainer). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.","title":"Enforcement"},{"location":"CODE_OF_CONDUCT/#attribution","text":"This Code of Conduct is adapted from the Contributor Covenant , version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html","title":"Attribution"},{"location":"CONTRIBUTING/","text":"Contributing to the Mercury framework Thanks for taking the time to contribute! The following is a set of guidelines for contributing to Mercury and its packages, which are hosted in the Accenture Organization on GitHub. These are mostly guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request. Code of Conduct This project and everyone participating in it is governed by our Code of Conduct . By participating, you are expected to uphold this code. Please report unacceptable behavior to Kevin Bader, who is the current project maintainer. What should I know before I get started? We follow the standard GitHub workflow . Before submitting a Pull Request: Please write tests. Make sure you run all tests and check for warnings. Think about whether it makes sense to document the change in some way. For smaller, internal changes, inline documentation might be sufficient, while more visible ones might warrant a change to the developer's guide or the README . Update CHANGELOG.md file with your current change in form of [Type of change e.g. Config, Kafka, .etc] with a short description of what it is all about and a link to issue or pull request, and choose a suitable section (i.e., changed, added, fixed, removed, deprecated). Design Decisions When we make a significant decision in how to write code, or how to maintain the project and what we can or cannot support, we will document it using Architecture Decision Records (ADR) . Take a look at the design notes for existing ADRs. If you have a question around how we do things, check to see if it is documented there. If it is not documented there, please ask us - chances are you're not the only one wondering. Of course, also feel free to challenge the decisions by starting a discussion on the mailing list.","title":"Contribution"},{"location":"CONTRIBUTING/#contributing-to-the-mercury-framework","text":"Thanks for taking the time to contribute! The following is a set of guidelines for contributing to Mercury and its packages, which are hosted in the Accenture Organization on GitHub. These are mostly guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request.","title":"Contributing to the Mercury framework"},{"location":"CONTRIBUTING/#code-of-conduct","text":"This project and everyone participating in it is governed by our Code of Conduct . By participating, you are expected to uphold this code. Please report unacceptable behavior to Kevin Bader, who is the current project maintainer.","title":"Code of Conduct"},{"location":"CONTRIBUTING/#what-should-i-know-before-i-get-started","text":"We follow the standard GitHub workflow . Before submitting a Pull Request: Please write tests. Make sure you run all tests and check for warnings. Think about whether it makes sense to document the change in some way. For smaller, internal changes, inline documentation might be sufficient, while more visible ones might warrant a change to the developer's guide or the README . Update CHANGELOG.md file with your current change in form of [Type of change e.g. Config, Kafka, .etc] with a short description of what it is all about and a link to issue or pull request, and choose a suitable section (i.e., changed, added, fixed, removed, deprecated).","title":"What should I know before I get started?"},{"location":"CONTRIBUTING/#design-decisions","text":"When we make a significant decision in how to write code, or how to maintain the project and what we can or cannot support, we will document it using Architecture Decision Records (ADR) . Take a look at the design notes for existing ADRs. If you have a question around how we do things, check to see if it is documented there. If it is not documented there, please ask us - chances are you're not the only one wondering. Of course, also feel free to challenge the decisions by starting a discussion on the mailing list.","title":"Design Decisions"},{"location":"INCLUSIVITY/","text":"TECHNOLOGY INCLUSIVE LANGUAGE GUIDEBOOK As an organization, Accenture believes in building an inclusive workplace and contributing to a world where equality thrives. Certain terms or expressions can unintentionally harm, perpetuate damaging stereotypes, and insult people. Inclusive language avoids bias, slang terms, and word choices which express derision of groups of people based on race, gender, sexuality, or socioeconomic status. The Accenture North America Technology team created this guidebook to provide Accenture employees with a view into inclusive language and guidance for working to avoid its use\u2014helping to ensure that we communicate with respect, dignity and fairness. How to use this guide? As of 8/2023, Accenture has over 730,000 employees from diverse backgrounds, who perform consulting and delivery work for an equally diverse set of clients and partners. When communicating with your colleagues and representing Accenture, consider the connotation, however unintended, of certain terms in your written and verbal communication. The guidelines are intended to help you recognize non-inclusive words and understand potential meanings that these words might convey. Our goal with these recommendations is not to require you to use specific words, but to ask you to take a moment to consider how your audience may be affected by the language you choose. Inclusive Categories Non-inclusive term Replacement Explanation Race, Ethnicity & National Origin master primary client source leader Using the terms \u201cmaster/slave\u201d in this context inappropriately normalizes and minimizes the very large magnitude that slavery and its effects have had in our history. slave secondary replica follower blacklist deny list block list The term \u201cblacklist\u201d was first used in the early 1600s to describe a list of those who were under suspicion and thus not to be trusted, whereas \u201cwhitelist\u201d referred to those considered acceptable. Accenture does not want to promote the association of \u201cblack\u201d and negative, nor the connotation of \u201cwhite\u201d being the inverse, or positive. whitelist allow list approved list native original core feature Referring to \u201cnative\u201d vs \u201cnon-native\u201d to describe technology platforms carries overtones of minimizing the impact of colonialism on native people, and thus minimizes the negative associations the terminology has in the latter context. non-native non-original non-core feature Gender & Sexuality man-hours work-hours business-hours When people read the words \u2018man\u2019 or \u2018he,\u2019 people often picture males only. Usage of the male terminology subtly suggests that only males can perform certain work or hold certain jobs. Gender-neutral terms include the whole audience, and thus using terms such as \u201cbusiness executive\u201d instead of \u201cbusinessman,\u201d or informally, \u201cfolks\u201d instead of \u201cguys\u201d is preferable because it is inclusive. man-days work-days business-days Ability Status & (Dis)abilities sanity check insanity check confidence check quality check rationality check Using the \u201cHuman Engagement, People First\u2019 approach, putting people - all people - at the center is important. Denoting ability status in the context of inferior or problematic work implies that people with mental illnesses are inferior, wrong, or incorrect. dummy variables indicator variables Violence STONITH, kill, hit conclude cease discontinue Using the \u201cHuman Engagement, People First\u2019 approach, putting people - all people - at the center is important. Denoting ability status in the context of inferior or problematic work implies that people with mental illnesses are inferior, wrong, or incorrect. one throat to choke single point of contact primary contact This guidebook is a living document and will be updated as terminology evolves. We encourage our users to provide feedback on the effectiveness of this document and we welcome additional suggestions. Contact us at Technology_ProjectElevate@accenture.com .","title":"Inclusivity"},{"location":"INCLUSIVITY/#technology-inclusive-language-guidebook","text":"As an organization, Accenture believes in building an inclusive workplace and contributing to a world where equality thrives. Certain terms or expressions can unintentionally harm, perpetuate damaging stereotypes, and insult people. Inclusive language avoids bias, slang terms, and word choices which express derision of groups of people based on race, gender, sexuality, or socioeconomic status. The Accenture North America Technology team created this guidebook to provide Accenture employees with a view into inclusive language and guidance for working to avoid its use\u2014helping to ensure that we communicate with respect, dignity and fairness. How to use this guide? As of 8/2023, Accenture has over 730,000 employees from diverse backgrounds, who perform consulting and delivery work for an equally diverse set of clients and partners. When communicating with your colleagues and representing Accenture, consider the connotation, however unintended, of certain terms in your written and verbal communication. The guidelines are intended to help you recognize non-inclusive words and understand potential meanings that these words might convey. Our goal with these recommendations is not to require you to use specific words, but to ask you to take a moment to consider how your audience may be affected by the language you choose. Inclusive Categories Non-inclusive term Replacement Explanation Race, Ethnicity & National Origin master primary client source leader Using the terms \u201cmaster/slave\u201d in this context inappropriately normalizes and minimizes the very large magnitude that slavery and its effects have had in our history. slave secondary replica follower blacklist deny list block list The term \u201cblacklist\u201d was first used in the early 1600s to describe a list of those who were under suspicion and thus not to be trusted, whereas \u201cwhitelist\u201d referred to those considered acceptable. Accenture does not want to promote the association of \u201cblack\u201d and negative, nor the connotation of \u201cwhite\u201d being the inverse, or positive. whitelist allow list approved list native original core feature Referring to \u201cnative\u201d vs \u201cnon-native\u201d to describe technology platforms carries overtones of minimizing the impact of colonialism on native people, and thus minimizes the negative associations the terminology has in the latter context. non-native non-original non-core feature Gender & Sexuality man-hours work-hours business-hours When people read the words \u2018man\u2019 or \u2018he,\u2019 people often picture males only. Usage of the male terminology subtly suggests that only males can perform certain work or hold certain jobs. Gender-neutral terms include the whole audience, and thus using terms such as \u201cbusiness executive\u201d instead of \u201cbusinessman,\u201d or informally, \u201cfolks\u201d instead of \u201cguys\u201d is preferable because it is inclusive. man-days work-days business-days Ability Status & (Dis)abilities sanity check insanity check confidence check quality check rationality check Using the \u201cHuman Engagement, People First\u2019 approach, putting people - all people - at the center is important. Denoting ability status in the context of inferior or problematic work implies that people with mental illnesses are inferior, wrong, or incorrect. dummy variables indicator variables Violence STONITH, kill, hit conclude cease discontinue Using the \u201cHuman Engagement, People First\u2019 approach, putting people - all people - at the center is important. Denoting ability status in the context of inferior or problematic work implies that people with mental illnesses are inferior, wrong, or incorrect. one throat to choke single point of contact primary contact This guidebook is a living document and will be updated as terminology evolves. We encourage our users to provide feedback on the effectiveness of this document and we welcome additional suggestions. Contact us at Technology_ProjectElevate@accenture.com .","title":"TECHNOLOGY INCLUSIVE LANGUAGE GUIDEBOOK"},{"location":"arch-decisions/DESIGN-NOTES/","text":"Design notes Event choreography by configuration The recommended way to write a composable application is event choreography by configuration using \"Event Script\". This would potentially reduce code size by half. Support sequential synchronous RPC in a non-blocking fashion The foundation library (platform-core) has been integrated with Java 21 virtual thread and Kotlin suspend function features. When a user function makes a RPC call using virtual thread or suspend function, the user function appears to be \"blocked\" so that the code can execute sequentially. Behind the curtain, the function is actually \"suspended\". This makes sequential code with RPC performs as good as reactive code. More importantly, the sequential code represents the intent of the application clearly, thus making code easier to read and maintain. Low level control of function execution strategies You can precisely control how your functions execute, using virtual threads, suspend functions or kernel thread pools to yield the highest performance and throughput. Serialization Gson We are using Gson for its minimalist design. We have customized the serialization behavior to be similar to Jackson and other serializers. i.e. Integer and long values are kept without decimal points. For API functional compatibility with Jackson, we have added the writeValueAsString, writeValueAsBytes and readValue methods. The convertValue method has been consolidated into the readValue method. MsgPack For efficient and serialization performance, we use MsgPack as schemaless binary transport for EventEnvelope that contains event metadata, headers and payload. User provided serializers This provides more flexibility for user function to take full control of their PoJo serialization needs. Custom JSON and XML serializers For consistency, we have customized Spring Boot and Servlet serialization and exception handlers. Reactive design Mercury uses the temporary local file system ( /tmp ) as an overflow area for events when the consumer is slower than the producer. This event buffering design means that user application does not have to handle back-pressure logic directly. However, it does not restrict you from implementing your flow-control logic. In-memory event system In Mercury version 1, the Akka actor system is used as the in-memory event bus. Since Mercury version 2, we have migrated from Akka to Eclipse Vertx. In Mercury version 3, we extend the engine to be fully non-blocking with low-level control of application performance and throughput. In Mercury version 3.1, the platform core engine is fully integrated with Java 21 virtual thread. Spring Boot 3 The platform-core includes a non-blocking HTTP and websocket server for standalone operation without Spring Boot. The rest-spring-3 library is designed to turn your code to be a Spring Boot application. You may also use the platform-core library with a regular Spring Boot application without the rest-spring-3 library if you prefer. Support of Mono and Flux results A user function may return a regular result that can be a PoJo, HashMap or Java primitive. It can also return a Mono or Flux reactive response object for a future result or a future series of results. Other reactive response objects must be converted to a Mono or Flux object.","title":"Design notes"},{"location":"arch-decisions/DESIGN-NOTES/#design-notes","text":"","title":"Design notes"},{"location":"arch-decisions/DESIGN-NOTES/#event-choreography-by-configuration","text":"The recommended way to write a composable application is event choreography by configuration using \"Event Script\". This would potentially reduce code size by half.","title":"Event choreography by configuration"},{"location":"arch-decisions/DESIGN-NOTES/#support-sequential-synchronous-rpc-in-a-non-blocking-fashion","text":"The foundation library (platform-core) has been integrated with Java 21 virtual thread and Kotlin suspend function features. When a user function makes a RPC call using virtual thread or suspend function, the user function appears to be \"blocked\" so that the code can execute sequentially. Behind the curtain, the function is actually \"suspended\". This makes sequential code with RPC performs as good as reactive code. More importantly, the sequential code represents the intent of the application clearly, thus making code easier to read and maintain.","title":"Support sequential synchronous RPC in a non-blocking fashion"},{"location":"arch-decisions/DESIGN-NOTES/#low-level-control-of-function-execution-strategies","text":"You can precisely control how your functions execute, using virtual threads, suspend functions or kernel thread pools to yield the highest performance and throughput.","title":"Low level control of function execution strategies"},{"location":"arch-decisions/DESIGN-NOTES/#serialization","text":"","title":"Serialization"},{"location":"arch-decisions/DESIGN-NOTES/#gson","text":"We are using Gson for its minimalist design. We have customized the serialization behavior to be similar to Jackson and other serializers. i.e. Integer and long values are kept without decimal points. For API functional compatibility with Jackson, we have added the writeValueAsString, writeValueAsBytes and readValue methods. The convertValue method has been consolidated into the readValue method.","title":"Gson"},{"location":"arch-decisions/DESIGN-NOTES/#msgpack","text":"For efficient and serialization performance, we use MsgPack as schemaless binary transport for EventEnvelope that contains event metadata, headers and payload.","title":"MsgPack"},{"location":"arch-decisions/DESIGN-NOTES/#user-provided-serializers","text":"This provides more flexibility for user function to take full control of their PoJo serialization needs.","title":"User provided serializers"},{"location":"arch-decisions/DESIGN-NOTES/#custom-json-and-xml-serializers","text":"For consistency, we have customized Spring Boot and Servlet serialization and exception handlers.","title":"Custom JSON and XML serializers"},{"location":"arch-decisions/DESIGN-NOTES/#reactive-design","text":"Mercury uses the temporary local file system ( /tmp ) as an overflow area for events when the consumer is slower than the producer. This event buffering design means that user application does not have to handle back-pressure logic directly. However, it does not restrict you from implementing your flow-control logic.","title":"Reactive design"},{"location":"arch-decisions/DESIGN-NOTES/#in-memory-event-system","text":"In Mercury version 1, the Akka actor system is used as the in-memory event bus. Since Mercury version 2, we have migrated from Akka to Eclipse Vertx. In Mercury version 3, we extend the engine to be fully non-blocking with low-level control of application performance and throughput. In Mercury version 3.1, the platform core engine is fully integrated with Java 21 virtual thread.","title":"In-memory event system"},{"location":"arch-decisions/DESIGN-NOTES/#spring-boot-3","text":"The platform-core includes a non-blocking HTTP and websocket server for standalone operation without Spring Boot. The rest-spring-3 library is designed to turn your code to be a Spring Boot application. You may also use the platform-core library with a regular Spring Boot application without the rest-spring-3 library if you prefer.","title":"Spring Boot 3"},{"location":"arch-decisions/DESIGN-NOTES/#support-of-mono-and-flux-results","text":"A user function may return a regular result that can be a PoJo, HashMap or Java primitive. It can also return a Mono or Flux reactive response object for a future result or a future series of results. Other reactive response objects must be converted to a Mono or Flux object.","title":"Support of Mono and Flux results"},{"location":"guides/APPENDIX-I/","text":"Application Configuration The following parameters are used by the system. You can define them in either the application.properties or application.yml file. When you use both application.properties and application.yml, the parameters in application.properties will take precedence. Key Value (example) Required application.name Application name Yes spring.application.name Alias for application name Yes*1 info.app.version major.minor.build (e.g. 1.0.0) Yes info.app.description Something about your application Yes web.component.scan your own package path or parent path Yes server.port e.g. 8083 Yes*1 rest.server.port e.g. 8085 Optional websocket.server.port Alias for rest.server.port Optional rest.automation true if you want to enable automation Optional yaml.rest.automation Config location e.g. classpath:/rest.yaml Optional yaml.event.over.http Config location classpath:/event-over-http.yaml Optional yaml.multicast Config location classpath:/multicast.yaml Optional yaml.journal Config location classpath:/journal.yaml Optional yaml.route.substitution Config location Optional yaml.topic.substitution Config location Optional yaml.cron Config location Optional yaml.flow.automation Config location. e.g. classpath:/flows.yaml EventScript static.html.folder classpath:/public/ Yes spring.web.resources.static-locations (alias for static.html.folder) Yes*1 mime.types Map of file extensions to MIME types (application.yml only) Optional spring.mvc.static-path-pattern /** Yes*1 show.env.variables comma separated list of variable names Optional show.application.properties comma separated list of property names Optional cloud.connector kafka, none, etc. Optional cloud.services e.g. some.interesting.service Optional snake.case.serialization true (recommended) Optional protect.info.endpoints true to disable actuators. Default: true Optional trace.http.header comma separated list. Default \"X-Trace-Id\" Optional hsts.feature default is true Optional* application.feature.route.substitution default is false Optional application.feature.topic.substitution default is false Optional kafka.replication.factor 3 Kafka cloud.client.properties e.g. classpath:/kafka.properties Connector user.cloud.client.properties e.g. classpath:/second-kafka.properties Connector default.app.group.id groupId for the app instance. Default: appGroup Connector default.monitor.group.id groupId for the presence-monitor. Default: monitorGroup Connector monitor.topic topic for the presence-monitor. Default: service.monitor Connector app.topic.prefix Default: multiplex (DO NOT change) Connector app.partitions.per.topic Max Kafka partitions per topic. Default: 32 Connector max.virtual.topics Max virtual topics = partitions * topics. Default: 288 Connector max.closed.user.groups Number of closed user groups. Default: 10, range: 3 - 30 Connector closed.user.group Closed user group. Default: 1 Connector transient.data.store Default is \"/tmp/reactive\" Optional running.in.cloud Default is false (set to true if containerized) Optional deferred.commit.log Default is false (for unit tests only) Optional kernel.thread.pool Default 100. Not more than 200. Optional * - when using the \"rest-spring\" library Base configuration files By default, the system assumes the following application configuration files: application.properties application.yml You can change this behavior by adding the app-config-reader.yml in your project's resources folder. resources: - application.properties - application.yml You can tell the system to load application configuration from different set of files. You can use either PROPERTIES or YAML files. YAML files can use \"yml\" or \"yaml\" extension. For example, you may use only \"application.yml\" file without scanning application.properties. Partial support of Spring Active Profiles When the parameter \"spring.profiles.active\" is available in application.properties or application.yml, the AppConfigReader will try to load the additional configuration files. For example, if \"spring.profiles.active=dev\", the system will load \"application-dev.properties\" and \"application-dev.yml\" accordingly. When more than one active profile is needed, you can use a comma separated list of profiles in \"spring.profiles.active\". For Spring Boot compatibility, the filename prefix \"application-\" is fixed. Special handling for PROPERTIES file Since application.properties and application.yml can be used together, the system must enforce keyspace uniqueness because YAML keyspaces are hierarchical. For example, if you have x.y and x.y.z, x.y is the parent of x.y.z. Therefore, you cannot set a value for the parent key since the parent is a key-value container. This hierarchical rule is enforced for PROPERTIES files. If you have x.y=3 and x.y.z=2 in the same PROPERTIES file, x.y will become a parent of x.y.z and its intended value of 3 will be lost. Optional Service The OptionalService annotation may be used with the following class annotations: BeforeApplication MainApplication PreLoad WebSocketService When the OptionalService annotation is available, the system will evaluate the annotation value as a conditional statement where it supports one or more simple condition using a key-value in the application configuration. For examples: OptionalService(\"rest.automation\") - the class will be loaded when rest.automation=true OptionalService(\"!rest.automation\") - the class will be loaded when rest.automation is false or non-exist OptionalService(\"interesting.key=100\") - the system will load the class when \"interesting.key\" is set to 100 in application configuration. To specify more than one condition, use a comma separated list as the value like this: OptionalService(\"web.socket.enabled, rest.automation\") - this tells the system to load the class when either web.socket.enabled or rest.automation is true. Static HTML contents You can place static HTML files (e.g. the HTML bundle for a UI program) in the \"resources/public\" folder or in the local file system using the \"static.html.folder\" parameter. The system supports a bare minimal list of file extensions to MIME types. If your use case requires additional MIME type mapping, you may define them in the application.yml configuration file under the mime.types section like this: mime.types: pdf: 'application/pdf' doc: 'application/msword' Note that application.properties file cannot be used for the \"mime.types\" section because it only supports text key-values. HTTP and websocket port assignment If rest.automation=true and rest.server.port or server.port are configured, the system will start a lightweight non-blocking HTTP server. If rest.server.port is not available, it will fall back to server.port . If rest.automation=false and you have a websocket server endpoint annotated as WebsocketService , the system will start a non-blocking Websocket server with a minimalist HTTP server that provides actuator services. If websocket.server.port is not available, it will fall back to rest.server.port or server.port . If you add Spring Boot dependency, Spring Boot will use server.port to start Tomcat or similar HTTP server. The built-in lightweight non-blocking HTTP server and Spring Boot can co-exist when you configure rest.server.port and server.port to use different ports. Note that the websocket.server.port parameter is an alias of rest.server.port . Transient data store The system handles back-pressure automatically by overflowing events from memory to a transient data store. As a cloud native best practice, the folder must be under \"/tmp\". The default is \"/tmp/reactive\". The \"running.in.cloud\" parameter must be set to false when your apps are running in IDE or in your laptop. When running in kubernetes, it can be set to true. Snake or Camel case serializers Serialization and de-serialization of events are performed automatically. If there is a genuine need to programmatically perform serialization, you may use the pre-configured serializer so that the serialization behavior is consistent. You can get an instance of the serializer with SimpleMapper.getInstance().getMapper() . The serializer may perform snake case or camel serialization depending on the parameter snake.case.serialization . If you want to ensure snake case or camel, you can select the serializer like this: SimpleObjectMapper snakeCaseMapper = SimpleMapper.getInstance().getSnakeCaseMapper(); SimpleObjectMapper camelCaseMapper = SimpleMapper.getInstance().getCamelCaseMapper(); The trace.http.header parameter The trace.http.header parameter sets the HTTP header for trace ID. When configured with more than one label, the system will retrieve trace ID from the corresponding HTTP header and propagate it through the transaction that may be served by multiple services. If trace ID is presented in an HTTP request, the system will use the same label to set HTTP response traceId header. X-Trace-Id: a9a4e1ec-1663-4c52-b4c3-7b34b3e33697 or X-Correlation-Id: a9a4e1ec-1663-4c52-b4c3-7b34b3e33697 Kafka specific configuration If you use the kafka-connector (cloud connector) and kafka-presence (presence monitor), you may want to externalize kafka.properties like this: cloud.client.properties=file:/tmp/config/kafka.properties Note that \"classpath\" refers to embedded config file in the \"resources\" folder in your source code and \"file\" refers to an external config file. You want also use the embedded config file as a backup like this: cloud.client.properties=file:/tmp/config/kafka.properties, classpath:/kafka.properties Distributed trace To enable distributed trace logging, please set this in log4j2.xml:HTTP-${status}
${warning}
| Type | error |
| Status | ${status} |
| Message | ${message} |
| Path | ${path} |
HTTP-${status}
${warning}
| Type | error |
| Status | ${status} |
| Message | ${message} |
| Path | ${path} |