Skip to content

Commit

Permalink
Merge branch 'conductor-oss:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
sukruthiarjula authored Jul 10, 2024
2 parents 7b41298 + 3a43cc6 commit d81b0ca
Show file tree
Hide file tree
Showing 82 changed files with 1,772 additions and 678 deletions.
72 changes: 33 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,11 @@
Conductor is a platform _originally_ created at Netflix to orchestrate microservices and events.
Conductor OSS is maintained by the team of developers at [Orkes](https://orkes.io/) along with the members of the open source community.

The latest version is [![Github release](https://img.shields.io/github/v/release/conductor-oss/conductor.svg)](https://GitHub.com/conductor-oss/conductor/releases)
## Conductor OSS
This is the new home for the Conductor open source going forward (previously hosted at Netflix/Conductor).
> [!IMPORTANT]
> Going forward, all the bug fixes, feature requests and security patches will be applied and released from this repository.

The last published version of Netflix Conductor will be **3.15.0** which we will continue to support.
_The last published version of Netflix Conductor will be **3.15.0** which we will continue to support._

If you would like to participate in the roadmap and development, [please reach out](https://forms.gle/P2i1xHrxPQLrjzTB7).

Expand All @@ -22,23 +20,43 @@ Show support for the Conductor OSS. Please help spread the awareness by starrin

[![GitHub stars](https://img.shields.io/github/stars/conductor-oss/conductor.svg?style=social&label=Star&maxAge=)](https://GitHub.com/conductor-oss/conductor/)

## Update your local forks/clones
Please update your forks to point to this repo. This will ensure your commits and PRs can be send against this repository
## Getting Started

### Using Docker (Recommended)
Follow the steps below to launch the docker container:

```shell
git remote set-url origin https://github.com/conductor-oss/conductor
docker compose -f docker/docker-compose.yaml up
```
> [!IMPORTANT]
> **Follow the steps below if you have an active PR against the Netflix/Conductor repository**
> 1. Fork **this** repository
> 2. Update your local repository to change the remote to this repository
> 3. Send a PR against the `main` branch
* Navigate to http://localhost:5000 once the container starts to launch UI.
* APIs are accessible at http://localhost:8080
* Swagger Docs:http://localhost:8080/swagger-ui/index.html?configUrl=/api-docs/swagger-config#/

## Database Requirements

* The default persistence used is Redis
* The indexing backend is [Elasticsearch](https://www.elastic.co/) (7.x)

## Configuration for various database backends

| Backend | Configuration |
|----------------|---------------------------------------------------------------------------------------|
| Redis + ES7 | [config-redis.properties](docker/server/config/config-redis.properties) |
| Postgres | [config-postgres.properties](docker/server/config/config-postgres.properties) |
| Postgres + ES7 | [config-postgres-es7.properties](docker/server/config/config-postgres-es7.properties) |
| MySQL + ES7 | [config-mysql.properties](docker/server/config/config-mysql.properties) |

## Other Requirements
* JDK 17+
* UI requires Node 14 to build. Earlier Node versions may work but are untested.

### Building From Source
If you wish to build your own distribution, you can run ```./gradlew build``` from this project that products the runtime artifacts.
The runnable server is in server/ module.

## Conductor OSS Roadmap
[See the roadmap for the Conductor](ROADMAP.md)

## Releases
The latest version is [![Github release](https://img.shields.io/github/v/release/conductor-oss/conductor.svg)](https://GitHub.com/conductor-oss/conductor/releases)

## Resources
#### [Slack Community](https://join.slack.com/t/orkes-conductor/shared_invite/zt-2hmxn0i3n-_W~a9rWMbvMoYmlJo3Y15g)
We have an active [community](https://join.slack.com/t/orkes-conductor/shared_invite/zt-2hmxn0i3n-_W~a9rWMbvMoYmlJo3Y15g) of Conductor users and contributors on the channel.
Expand All @@ -52,31 +70,7 @@ Conductor supports creating workflows using JSON and Code.
SDK support for creating workflows using code is available in multiple languages and can be found at https://github.com/conductor-sdk


## Getting Started - Building & Running Conductor

### From Source:
If you wish to build your own distribution, you can run ```./gradlew build``` from this project that products the runtime artifacts.
The runnable server is in server/ module.

### Using Docker (Recommended)
Follow the steps below to launch the docker container:

```shell
docker compose -f docker/docker-compose.yaml up
```
* Navigate to http://localhost:5000 once the container starts to launch UI.
* APIs are accessible at http://localhost:8080
* Swagger Docs:http://localhost:8080/swagger-ui/index.html?configUrl=/api-docs/swagger-config#/


## Database Requirements

* The default persistence used is Redis
* The indexing backend is [Elasticsearch](https://www.elastic.co/) (7.x)

## Other Requirements
* JDK 17+
* UI requires Node 14 to build. Earlier Node versions may work but are untested.

## Get Support
There are several ways to get in touch with us:
Expand Down
3 changes: 0 additions & 3 deletions amqp/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,4 @@ dependencies {

compileOnly 'org.springframework.boot:spring-boot-starter'
compileOnly 'org.springframework.boot:spring-boot-starter-web'



}
2 changes: 1 addition & 1 deletion annotations-processor/README.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
[Annotations Processor](docs/documentation/advanced/annotation-processor.md)
Annotation processor is used to generate protobuf files from the annotations.
1 change: 1 addition & 0 deletions annotations/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Annotations
Used for Conductor to convert Java POJs to protobuf files.

- `protogen` Annotations
- Original Author: Vicent Martí - https://github.com/vmg
Expand Down
4 changes: 4 additions & 0 deletions awss3-storage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# S3 external storage support
Used by Conductor to support external payload into S3 blob.

See [https://docs.conductor-oss.org/documentation/advanced/externalpayloadstorage.html](https://docs.conductor-oss.org/documentation/advanced/externalpayloadstorage.html) for more details
3 changes: 2 additions & 1 deletion azureblob-storage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ It has only been tested with **v12.2.0**.

### Usage

Cf. Documentation [External Payload Storage](https://netflix.github.io/conductor/externalpayloadstorage/#azure-blob-storage)
Documentation [External Payload Storage]([https://netflix.github.io/conductor/externalpayloadstorage/#azure-blob-storage](https://docs.conductor-oss.org/documentation/advanced/externalpayloadstorage.html))

See [https://docs.conductor-oss.org/documentation/advanced/externalpayloadstorage.html]() for more details
### Example

```properties
Expand Down
44 changes: 36 additions & 8 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ buildscript {
}
}
dependencies {
classpath 'org.springframework.boot:spring-boot-gradle-plugin:3.2.1'
classpath 'org.springframework.boot:spring-boot-gradle-plugin:3.1.4'
classpath 'com.diffplug.spotless:spotless-plugin-gradle:6.+'
}
}
Expand All @@ -21,6 +21,7 @@ plugins {
id 'signing'
id 'java-library'
id "com.diffplug.spotless" version "6.25.0"
id 'org.springframework.boot' version '3.3.0'
}

// Establish version and status
Expand Down Expand Up @@ -50,11 +51,19 @@ allprojects {

group = 'org.conductoross'

configurations.all {
exclude group: 'ch.qos.logback', module: 'logback-classic'
exclude group: 'ch.qos.logback', module: 'logback-core'
exclude group: 'org.apache.logging.log4j', module: 'log4j-to-slf4j'
exclude group: 'org.slf4j', module: 'slf4j-log4j12'
configurations {
all {
exclude group: 'ch.qos.logback', module: 'logback-classic'
exclude group: 'ch.qos.logback', module: 'logback-core'
exclude group: 'org.apache.logging.log4j', module: 'log4j-to-slf4j'
exclude group: 'org.slf4j', module: 'slf4j-log4j12'

resolutionStrategy.eachDependency { details ->
if (details.requested.group.startsWith('com.fasterxml.jackson.') ) {
details.useVersion "2.15.2"
}
}
}
}

repositories {
Expand All @@ -73,15 +82,31 @@ allprojects {
implementation('org.apache.logging.log4j:log4j-api')
implementation('org.apache.logging.log4j:log4j-slf4j-impl')
implementation('org.apache.logging.log4j:log4j-jul')
implementation('org.apache.logging.log4j:log4j-web')
implementation('org.apache.logging.log4j:log4j-web')

compileOnly 'org.projectlombok:lombok:1.18.34'

annotationProcessor 'org.projectlombok:lombok:1.18.34'
annotationProcessor 'org.springframework.boot:spring-boot-configuration-processor'

testImplementation('org.springframework.boot:spring-boot-starter-test')
testImplementation('org.springframework.boot:spring-boot-starter-log4j2')
testImplementation 'junit:junit'
testImplementation "org.junit.vintage:junit-vintage-engine"
}
testAnnotationProcessor 'org.projectlombok:lombok:1.18.34'

//Locks for the dependecies
implementation('org.codehaus.jettison:jettison') {
version {
strictly '1.5.4'
}
}
implementation('org.apache.tomcat.embed:tomcat-embed-core') {
version {
strictly '10.1.25'
}
}
}
// processes additional configuration metadata json file as described here
// https://docs.spring.io/spring-boot/docs/2.3.1.RELEASE/reference/html/appendix-configuration-metadata.html#configuration-metadata-additional-metadata
compileJava.inputs.files(processResources)
Expand All @@ -95,6 +120,9 @@ allprojects {
showStandardStreams = false
}
}
bootJar {
enabled = false
}
}

// all client and their related modules are published with Java 17 compatibility
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -403,44 +403,6 @@ class CassandraExecutionDAOSpec extends CassandraSpec {
eventExecutionList != null && eventExecutionList.empty
}

def "verify workflow serialization"() {
given: 'define a workflow'
String workflowId = new IDGenerator().generate()
WorkflowTask workflowTask = new WorkflowTask(taskDefinition: new TaskDef(concurrentExecLimit: 2))
WorkflowDef workflowDef = new WorkflowDef(name: UUID.randomUUID().toString(), version: 1, tasks: [workflowTask])
WorkflowModel workflow = new WorkflowModel(workflowDefinition: workflowDef, workflowId: workflowId, status: WorkflowModel.Status.RUNNING, createTime: System.currentTimeMillis())

when: 'serialize workflow'
def workflowJson = objectMapper.writeValueAsString(workflow)

then:
!workflowJson.contains('failedReferenceTaskNames')
// workflowTask
!workflowJson.contains('decisionCases')
!workflowJson.contains('defaultCase')
!workflowJson.contains('forkTasks')
!workflowJson.contains('joinOn')
!workflowJson.contains('defaultExclusiveJoinTask')
!workflowJson.contains('loopOver')
}

def "verify task serialization"() {
given: 'define a workflow and tasks for this workflow'
String workflowId = new IDGenerator().generate()
WorkflowTask workflowTask = new WorkflowTask(taskDefinition: new TaskDef(concurrentExecLimit: 2))
TaskModel task = new TaskModel(workflowInstanceId: workflowId, taskType: UUID.randomUUID().toString(), referenceTaskName: UUID.randomUUID().toString(), status: TaskModel.Status.SCHEDULED, taskId: new IDGenerator().generate(), workflowTask: workflowTask)

when: 'serialize task'
def taskJson = objectMapper.writeValueAsString(task)

then:
!taskJson.contains('decisionCases')
!taskJson.contains('defaultCase')
!taskJson.contains('forkTasks')
!taskJson.contains('joinOn')
!taskJson.contains('defaultExclusiveJoinTask')
}

def "serde of workflow with large number of tasks"() {
given: 'create a workflow and tasks for this workflow'
String workflowId = new IDGenerator().generate()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
/*
* Copyright 2022 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.netflix.conductor.annotations.protogen;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
* ProtoEnum annotates an enum type that will be exposed via the GRPC API as a native Protocol
* Buffers enum.
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface ProtoEnum {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Copyright 2022 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.netflix.conductor.annotations.protogen;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
* ProtoField annotates a field inside an struct with metadata on how to expose it on its
* corresponding Protocol Buffers struct. For a field to be exposed in a ProtoBuf struct, the
* containing struct must also be annotated with a {@link ProtoMessage} or {@link ProtoEnum} tag.
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface ProtoField {
/**
* Mandatory. Sets the Protocol Buffer ID for this specific field. Once a field has been
* annotated with a given ID, the ID can never change to a different value or the resulting
* Protocol Buffer struct will not be backwards compatible.
*
* @return the numeric ID for the field
*/
int id();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* Copyright 2022 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.netflix.conductor.annotations.protogen;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
* ProtoMessage annotates a given Java class so it becomes exposed via the GRPC API as a native
* Protocol Buffers struct. The annotated class must be a POJO.
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface ProtoMessage {
/**
* Sets whether the generated mapping code will contain a helper to translate the POJO for this
* class into the equivalent ProtoBuf object.
*
* @return whether this class will generate a mapper to ProtoBuf objects
*/
boolean toProto() default true;

/**
* Sets whether the generated mapping code will contain a helper to translate the ProtoBuf
* object for this class into the equivalent POJO.
*
* @return whether this class will generate a mapper from ProtoBuf objects
*/
boolean fromProto() default true;

/**
* Sets whether this is a wrapper class that will be used to encapsulate complex nested type
* interfaces. Wrapper classes are not directly exposed by the ProtoBuf API and must be mapped
* manually.
*
* @return whether this is a wrapper class
*/
boolean wrapper() default false;
}
Loading

0 comments on commit d81b0ca

Please sign in to comment.