Skip to content

Commit

Permalink
Merge pull request #27 from conductor-oss/community-docs-cleanup-2
Browse files Browse the repository at this point in the history
cleanup docs
  • Loading branch information
v1r3n authored Dec 22, 2023
2 parents 00c0673 + 5c06b37 commit c016206
Show file tree
Hide file tree
Showing 69 changed files with 124 additions and 289 deletions.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ assignees: ''

---

Please read our [contributor guide](https://github.com/Netflix/conductor/blob/main/CONTRIBUTING.md) before creating an issue.
Also consider discussing your idea on the [discussion forum](https://github.com/Netflix/conductor/discussions) first.
Please read our [contributor guide](https://github.com/conductor-oss/conductor/blob/main/CONTRIBUTING.md) before creating an issue.
Also consider discussing your idea on the [discussion forum](https://github.com/conductor-oss/conductor/discussions) first.

## Describe the Feature Request
_A clear and concise description of what the feature request is._
Expand Down
2 changes: 1 addition & 1 deletion RELATED.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
[Related Projects](docs/docs/resources/related.md)
[Related Projects](docs/resources/related.md)
2 changes: 1 addition & 1 deletion annotations-processor/README.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
[Annotations Processor](docs/docs/reference-docs/annotations-processor.md)
[Annotations Processor](docs/documentation/advanced/annotation-processor.md)
2 changes: 1 addition & 1 deletion awss3-storage/build.gradle
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2022 Netflix, Inc.
* Copyright 2023 Conductor authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
Expand Down
32 changes: 1 addition & 31 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ buildscript {
}
}
dependencies {
classpath 'com.netflix.nebula:gradle-extra-configurations-plugin:7.0.0'
classpath 'org.springframework.boot:spring-boot-gradle-plugin:3.1.4'
classpath 'com.diffplug.spotless:spotless-plugin-gradle:6.+'
}
Expand All @@ -20,11 +19,10 @@ plugins {
id 'application'
id 'jacoco'
id 'nebula.netflixoss' version '10.6.0'
id 'org.sonarqube' version '3.4.0.2513'
}

/*
* Copyright 2021 Netflix, Inc.
* Copyright 2023 Conductor authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
Expand Down Expand Up @@ -66,26 +64,6 @@ allprojects {

repositories {
mavenCentral()

// oss-candidate for -rc.* verions:
maven {
url "https://artifactory-oss.prod.netflix.net/artifactory/maven-oss-candidates"
}

/**
* This repository locates artifacts that don't exist in maven central but we had to backup from jcenter
* The exclusiveContent
*/
exclusiveContent {
forRepository {
maven {
url "https://artifactory-oss.prod.netflix.net/artifactory/required-jcenter-modules-backup"
}
}
filter {
includeGroupByRegex "com\\.github\\.vmg.*"
}
}
}

dependencyManagement {
Expand Down Expand Up @@ -145,14 +123,6 @@ task server {
dependsOn ':conductor-server:bootRun'
}

sonarqube {
properties {
property "sonar.projectKey", "com.netflix.conductor:conductor"
property "sonar.organization", "netflix"
property "sonar.host.url", "https://sonarcloud.io"
}
}

configure(allprojects - project(':conductor-grpc')) {
apply plugin: 'com.diffplug.spotless'

Expand Down
2 changes: 1 addition & 1 deletion cassandra-persistence/build.gradle
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2021 Netflix, Inc.
* Copyright 2023 Conductor authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
Expand Down
2 changes: 1 addition & 1 deletion common/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ dependencies {
}

/*
* Copyright 2021 Netflix, Inc.
* Copyright 2023 Conductor authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
Expand Down
2 changes: 1 addition & 1 deletion core/build.gradle
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2021 Netflix, Inc.
* Copyright 2023 Conductor authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
Expand Down
2 changes: 1 addition & 1 deletion core/src/main/resources/META-INF/validation.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<!--
Copyright 2020 Netflix, Inc.
Copyright 2023 Conductor authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright 2020 Netflix, Inc.
Copyright 2023 Conductor authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion dependencies.gradle
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2022 Netflix, Inc.
* Copyright 2023 Conductor authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
Expand Down
2 changes: 1 addition & 1 deletion docker/server/bin/startup.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/sh
#
# Copyright 2021 Netflix, Inc.
# Copyright 2023 Conductor authors
# <p>
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
Expand Down
2 changes: 1 addition & 1 deletion docker/server/config/log4j-file-appender.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2020 Netflix, Inc.
# Copyright 2023 Conductor authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion docker/server/config/log4j.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017 Netflix, Inc.
# Copyright 2023 Conductor authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion docker/ui/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# conductor:ui - Netflix Conductor UI
# conductor:ui - Conductor UI
#
FROM node:20-alpine
LABEL maintainer="Orkes OSS <[email protected]>"
Expand Down
9 changes: 0 additions & 9 deletions docs/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,6 @@ body {
background-size: cover;
}

/* Footer */
.md-copyright__highlight {
background-image: url('/img/netflix-oss.png');
background-size: contain;
background-repeat: no-repeat;
color: rgba(0,0,0,0);
height: 60px;
}

/* Comparison block */
.compare {
background-color: var(--brand-lt-blue);
Expand Down
2 changes: 1 addition & 1 deletion docs/devguide/architecture/technicaldetails.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ The goal is to allow only one decide to run on a workflow at any given time acro
**Preventing stale data updates with fencing tokens:**
While the locking service helps to run one decide at a time, it might still be possible for nodes with timed out locks to reactivate and continue execution from where it left off (usually with stale data). This can be avoided with fencing tokens, which basically is an incrementing counter on workflow state with read-before-write support in a transaction or similar construct.

*At Netflix, we use Cassandra. Considering the tradeoffs of Cassandra’s Lightweight Transactions (LWT) and the probability of this stale updates happening, and our testing results, we’ve decided to first only rollout distributed locking with Zookeeper. We'll monitor our system and add C* LWT if needed.
*Netflix uses Cassandra. Considering the tradeoffs of Cassandra’s Lightweight Transactions (LWT) and the probability of this stale updates happening, and our testing results, we’ve decided to first only rollout distributed locking with Zookeeper. We'll monitor our system and add C* LWT if needed.

#### Setting up desired level of consistency

Expand Down
2 changes: 1 addition & 1 deletion docs/devguide/concepts/why.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Why Conductor?
Conductor was built to help Netflix orchestrate microservices based process flows.
Conductor was built to help orchestrate microservices based process flows.

## Features

Expand Down
2 changes: 1 addition & 1 deletion docs/devguide/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Conductor, however will run [system tasks](../documentation/configuration/workfl

## How can I schedule workflows to run at a specific time?

Netflix Conductor itself does not provide any scheduling mechanism. But there is a community project [_Schedule Conductor Workflows_](https://github.com/jas34/scheduledwf) which provides workflow scheduling capability as a pluggable module as well as workflow server.
Conductor itself does not provide any scheduling mechanism. But there is a community project [_Schedule Conductor Workflows_](https://github.com/jas34/scheduledwf) which provides workflow scheduling capability as a pluggable module as well as workflow server.
Other way is you can use any of the available scheduling systems to make REST calls to Conductor to start a workflow. Alternatively, publish a message to a supported eventing system like SQS to trigger a workflow.
More details about [eventing](../documentation/configuration/eventhandlers.md).

Expand Down
55 changes: 1 addition & 54 deletions docs/devguide/how-tos/Workers/build-a-golang-task-worker.md
Original file line number Diff line number Diff line change
@@ -1,56 +1,3 @@
# Build a Go Task Worker

## Install
```shell
go get github.com/netflix/conductor/client/go
```
This will create a Go project under $GOPATH/src and download any dependencies.

## Implementing a Task a Worker
`task`package provies the types used to implement the worker. Here is a reference worker implementation:

```go
package task

import (
"fmt"
)

// Implementation for "task_1"
func Task_1_Execution_Function(t *task.Task) (taskResult *task.TaskResult, err error) {
log.Println("Executing Task_1_Execution_Function for", t.TaskType)

//Do some logic
taskResult = task.NewTaskResult(t)

output := map[string]interface{}{"task":"task_1", "key2":"value2", "key3":3, "key4":false}
taskResult.OutputData = output
taskResult.Status = "COMPLETED"
err = nil

return taskResult, err
}
```

## Worker Polling
Here is an example that shows how to start polling for tasks after defining the tasks.

```go
package main

import (
"github.com/netflix/conductor/client/go"
"github.com/netflix/conductor/client/go/task/sample"
)

func main() {
c := conductor.NewConductorWorker("{{ server_host }}", 1, 10000)

c.Start("task_1", "", sample.Task_1_Execution_Function, false)
c.Start("task_2", "mydomain", sample.Task_2_Execution_Function, true)
}
```
### `NewConductorWoker` parameters
1. baseUrl: Server address.
2. threadCount: No. of threads. Number of threads should be at-least same as the number of workers
3. pollingInterval: Time in millisecond between subsequent polls
See [conductor-sdk/conductor-go](https://github.com/conductor-sdk/conductor-go/blob/main/README.md)
6 changes: 3 additions & 3 deletions docs/devguide/how-tos/Workers/build-a-java-task-worker.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,14 +70,14 @@ Worker's core implementation logic goes in the `execute` method. Upon completion

The `getTaskDefName()` method returns the name of the task for which this worker provides the execution logic.

See [SampleWorker.java](https://github.com/Netflix/conductor/blob/main/client/src/test/java/com/netflix/conductor/client/sample/SampleWorker.java) for the complete example.
See [SampleWorker.java](https://github.com/conductor-oss/conductor/blob/main/client/src/test/java/com/netflix/conductor/client/sample/SampleWorker.java) for the complete example.

## Configuring polling using TaskRunnerConfigurer

The `TaskRunnerConfigurer` can be used to register the worker(s) and initialize the polling loop.
It manages the task workers thread pool and server communication (poll and task update).

Use the [Builder](https://github.com/Netflix/conductor/blob/main/client/src/main/java/com/netflix/conductor/client/automator/TaskRunnerConfigurer.java#L64) to create an instance of the `TaskRunnerConfigurer`. The builder accepts the following parameters:
Use the [Builder](https://github.com/conductor-oss/conductor/blob/main/client/src/main/java/com/netflix/conductor/client/automator/TaskRunnerConfigurer.java#L64) to create an instance of the `TaskRunnerConfigurer`. The builder accepts the following parameters:

```java
TaskClient taskClient = new TaskClient();
Expand All @@ -97,7 +97,7 @@ Use the [Builder](https://github.com/Netflix/conductor/blob/main/client/src/main
configurer.init();
```

See [Sample](https://github.com/Netflix/conductor/blob/main/client/src/test/java/com/netflix/conductor/client/sample/Main.java) for full example.
See [Sample](https://github.com/conductor-oss/conductor/blob/main/client/src/test/java/com/netflix/conductor/client/sample/Main.java) for full example.

### Configuration Details

Expand Down
80 changes: 1 addition & 79 deletions docs/devguide/how-tos/Workers/build-a-python-task-worker.md
Original file line number Diff line number Diff line change
@@ -1,82 +1,4 @@
# Build a Python Task Worker
## Install the python client
```shell
virtualenv conductorclient
source conductorclient/bin/activate
cd ../conductor/client/python
python setup.py install
```

## Implement a Task Worker
[ConductorWorker](https://github.com/Netflix/conductor/blob/main/polyglot-clients/python/conductor/ConductorWorker.py#L36)
class is used to implement task workers.
The following script shows how to bring up two task workers named `book_flight` and `book_car`:

```python
from __future__ import print_function
from conductor.ConductorWorker import ConductorWorker

def book_flight_task(task):
return {'status': 'COMPLETED', 'output': {'booking_ref': 2341111, 'airline': 'delta'}, 'logs': ['trying delta', 'skipping aa']}

def book_car_task(task):
return {'status': 'COMPLETED', 'output': {'booking_ref': "84545fdfd", 'agency': 'hertz'}, 'logs': ['trying hertz']}

def main():
print('Starting Travel Booking workflows')
cc = ConductorWorker('{{ server_host }}{{ api_prefix }}', 1, 0.1)
cc.start('book_flight', book_flight_task, False)
cc.start('book_car', book_car_task, True)

if __name__ == '__main__':
main()
```
### `ConductorWorker` parameters
```python
server_url: str
The url to the server hosting the conductor api.
Ex: '{{ server_host }}{{ api_prefix }}'

thread_count: int
The number of threads that will be polling for and
executing tasks in case of using the start method.

polling_interval: float
The number of seconds that each worker thread will wait
between polls to the conductor server.

worker_id: str, optional
The worker_id of the worker that is going to execute the
task. For further details, refer to the documentation
By default, it is set to hostname of the machine
```
### `start` method parameters
```pythhon
taskType: str
The name of the task that the worker is looking to execute
exec_function: function
The function that the worker will execute. The function
must return a dict with the `status`, `output` and `logs`
keys present. If this is not present, an Exception will be
raised
wait: bool
Whether the worker will block execution of further code.
Since the workers are being run in daemon threads, when the
program completes execution, all the threads are destroyed.
Setting wait to True prevents the program from ending.
If multiple workers are being called from the same program,
all but the last start call but have wait set to False.
The last start call must always set wait to True. If a
single worker is being called, set wait to True.
domain: str, optional
The domain of the task under which the worker will run. For
further details refer to the conductor server documentation
By default, it is set to None
```

See
[https://github.com/Netflix/conductor/tree/main/polyglot-clients/python](https://github.com/Netflix/conductor/tree/main/polyglot-clients/python)
for the source code.
[conductor-sdk/conductor-python](https://github.com/conductor-sdk/conductor-python/blob/main/README.md)
4 changes: 2 additions & 2 deletions docs/devguide/running/docker.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

# Running Conductor Using Docker

In this article we will explore how you can set up Netflix Conductor on your local machine using Docker compose.
In this article we will explore how you can set up Conductor on your local machine using Docker compose.
The docker compose will bring up the following:

1. Conductor API Server
Expand All @@ -17,7 +17,7 @@ The docker compose will bring up the following:
### 1. Clone the Conductor Code

```shell
$ git clone https://github.com/Netflix/conductor.git
$ git clone https://github.com/conductor-oss/conductor.git
```

### 2. Build the Docker Compose
Expand Down
Loading

0 comments on commit c016206

Please sign in to comment.