-
Notifications
You must be signed in to change notification settings - Fork 515
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
188 changed files
with
2,303 additions
and
3,353 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,205 @@ | ||
| :root { | ||
| /*--main-text-color: #212121;*/ | ||
| --md-primary-fg-color: #1976d2; | ||
| --brand-blue: #1976d2; | ||
| --brand-dark-blue: #242A36; | ||
| --caption-color: #4f4f4f; | ||
| --brand-lt-blue: #f0f5fb; | ||
| --brand-gray: rgb(118, 118, 118); | ||
| --brand-lt-gray: rgb(203,204,207); | ||
| --brand-red: #e50914; | ||
| } | ||
|
|
||
| /* Grid */ | ||
| .row { | ||
| display: flex; | ||
| flex-direction: row; | ||
| } | ||
| .col-4 { | ||
| flex: 0 0 33.3333333333%; | ||
| max-width: 33.3333333333%; | ||
| } | ||
| .col-6 { | ||
| flex: 0 0 50%; | ||
| max-width: 50%; | ||
| } | ||
|
|
||
|
|
||
|
|
||
| /* Navbar */ | ||
| .md-header { | ||
| background-color: white !important; | ||
| color: var(--brand-dark-blue); | ||
| } | ||
| .md-header__title { | ||
| visibility: hidden; | ||
| } | ||
| .md-logo img{ | ||
| height: 38px !important; | ||
| } | ||
| .home { | ||
| margin-bottom: -1.2rem !important; | ||
| } | ||
| .md-search__form { | ||
| transition: none !important; | ||
| } | ||
| .md-search__input:hover { | ||
| background-color: #00000042 !important; | ||
| } | ||
| .md-search__input.focus-visible:hover { | ||
| background-color: #fff !important; | ||
| } | ||
|
|
||
| /* Fonts */ | ||
| body { | ||
| color: var(--brand-dark-blue); | ||
| font-family: "Roboto", sans-serif !important; | ||
| font-weight: 400 !important; | ||
| } | ||
|
|
||
| .md-content h1 { | ||
| font-family: "Inter", sans-serif !important; | ||
| color: var(--brand-dark-blue) !important; | ||
| font-size: 32px !important; | ||
| font-weight: 700 !important; | ||
| } | ||
|
|
||
| .md-content h2 { | ||
| font-family: "Inter", sans-serif !important; | ||
| color: var(--brand-dark-blue) !important; | ||
| font-size: 24px !important; | ||
| font-weight: 700 !important; | ||
| } | ||
|
|
||
| .md-content h3 { | ||
| font-family: "Roboto", sans-serif !important; | ||
| color: var(--brand-dark-blue) !important; | ||
| font-size: 20px !important; | ||
| font-weight: 500 !important; | ||
| } | ||
|
|
||
| .md-content h4 { | ||
| font-family: "Roboto", sans-serif !important; | ||
| color: var(--brand-dark-blue) !important; | ||
| font-size: 18px !important; | ||
| font-weight: 400 !important; | ||
| } | ||
|
|
||
| .btn { | ||
| font-family: "Roboto", sans-serif; | ||
| font-size: 14px; | ||
| border-radius: 0.25rem; | ||
| } | ||
| .btn-primary { | ||
| background: #1976D2; | ||
| border: none; | ||
| color: white !important; | ||
| } | ||
|
|
||
| .hero { | ||
| padding-top: 100px; | ||
| padding-bottom: 100px; | ||
| } | ||
|
|
||
| .hero .heading { | ||
| font-size: 56px; | ||
| font-weight: 900; | ||
| line-height: 68px; | ||
| } | ||
|
|
||
| .hero .btn { | ||
| font-size: 16px; | ||
| padding: 10px 20px; | ||
| } | ||
|
|
||
| .hero .illustration { | ||
| margin-left: 35px; | ||
| } | ||
|
|
||
|
|
||
| .bullets .heading, .module .heading { | ||
| font-family: "Inter", sans-serif; | ||
| font-size: 26px; | ||
| font-weight: 700; | ||
| } | ||
| .bullets .row { | ||
| margin-bottom: 60px; | ||
| } | ||
| .bullets .caption { | ||
| padding-top: 10px; | ||
| padding-right: 30px; | ||
| } | ||
| .icon { | ||
| height: 25px !important; | ||
| margin-right: 5px; | ||
| vertical-align: -3px; | ||
| } | ||
|
|
||
| .caption { | ||
| font-weight: 400; | ||
| font-size: 17px; | ||
| line-height: 24px; | ||
| color: var(--caption-color); | ||
| } | ||
|
|
||
| .module { | ||
| margin-top: 80px; | ||
| margin-bottom: 80px; | ||
| padding-top: 50px; | ||
| padding-bottom: 50px; | ||
| } | ||
|
|
||
| .module .caption { | ||
| padding-top: 10px; | ||
| padding-right: 80px; | ||
| } | ||
| .module .screenshot { | ||
| width: 600px; | ||
| height: 337px; | ||
| box-shadow:inset 0 1px 0 rgba(255,255,255,.6), 0 22px 70px 4px rgba(0,0,0,0.56), 0 0 0 1px rgba(0, 0, 0, 0.0); | ||
| border-radius: 5px; | ||
| background-size: cover; | ||
| } | ||
|
|
||
| /* Footer */ | ||
| .md-copyright__highlight { | ||
| background-image: url('/img/netflix-oss.png'); | ||
| background-size: contain; | ||
| background-repeat: no-repeat; | ||
| color: rgba(0,0,0,0); | ||
| height: 60px; | ||
| } | ||
|
|
||
| /* Comparison block */ | ||
| .compare { | ||
| background-color: var(--brand-lt-blue); | ||
| padding-top: 80px; | ||
| padding-bottom: 80px; | ||
| margin: 0px -1000px; | ||
| text-align: center; | ||
| } | ||
| .compare .container { | ||
| max-width: 61rem; | ||
| margin-left: auto; | ||
| margin-right: auto; | ||
| } | ||
|
|
||
| .compare .heading { | ||
| margin-bottom: 30px; | ||
| margin-top: 0px; | ||
| } | ||
| .compare .bubble { | ||
| background: #fff; | ||
| border-radius: 10px; | ||
| padding: 30px; | ||
| height: 100%; | ||
| } | ||
|
|
||
| .compare .caption { | ||
| font-size: 15px; | ||
| line-height: 22px; | ||
| } | ||
|
|
||
| .compare .row { | ||
| margin: 0 0.8rem; | ||
| } |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes
8 changes: 4 additions & 4 deletions
8
docs/docs/architecture/overview.md → docs/devguide/architecture/index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
38 changes: 25 additions & 13 deletions
38
docs/docs/architecture/tasklifecycle.md → docs/devguide/architecture/tasklifecycle.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,47 +1,59 @@ | ||
| # Task Lifecycle | ||
|
|
||
| ## Task state transitions | ||
|
|
||
| The figure below depicts the state transitions that a task can go through within a workflow execution. | ||
|
|
||
|  | ||
|  | ||
|
|
||
| ## Retries and Failure Scenarios | ||
|
|
||
| ### Task failure and retries | ||
| Retries for failed task executions of each task can be configured independently. retryCount, retryDelaySeconds and retryLogic can be used to configure the retry mechanism. | ||
|
|
||
|  | ||
| Retries for failed task executions of each task can be configured independently. `retryCount`, `retryDelaySeconds` and `retryLogic` can be used to configure the retry mechanism. | ||
|
|
||
|  | ||
|
|
||
| 1. Worker (W1) polls for task T1 from the Conductor server and receives the task. | ||
| 2. Upon processing this task, the worker determines that the task execution is a failure and reports this to the server with FAILED status after 10 seconds. | ||
| 3. The server will persist this FAILED execution of T1. A new execution of task T1 will be created and scheduled to be polled. This task will be available to be polled after 5 (retryDelaySeconds) seconds. | ||
|
|
||
| ### Poll Timeout Seconds | ||
|
|
||
| Poll timeout is the maximum amount of time by which a worker needs to poll a task, else the task will be marked as `TIMED_OUT`. | ||
|
|
||
|  | ||
|
|
||
| In the figure above, task T1 does not get polled by the worker within 60 seconds, so Conductor marks it as `TIMED_OUT`. | ||
|
|
||
| ### Timeout seconds | ||
| Timeout is the maximum amount of time that the task must reach a terminal state in, else the task will be marked as TIMED_OUT. | ||
|
|
||
|  | ||
| Timeout is the maximum amount of time that the task must reach a terminal state in, else it will be marked as `TIMED_OUT`. | ||
|
|
||
|  | ||
|
|
||
| **0 seconds** -> Worker polls for task T1 from the Conductor server and receives the task. T1 is put into IN_PROGRESS status by the server. | ||
| Worker starts processing the task but is unable to process the task at this time. Worker updates the server with T1 set to IN_PROGRESS status and a callback of 9 seconds. | ||
| **0 seconds** -> Worker polls for task T1 from the Conductor server and receives the task. T1 is put into `IN_PROGRESS` status by the server. | ||
| Worker starts processing the task but is unable to process the task at this time. Worker updates the server with T1 set to `IN_PROGRESS` status and a callback of 9 seconds. | ||
| Server puts T1 back in the queue but makes it invisible and the worker continues to poll for the task but does not receive T1 for 9 seconds. | ||
|
|
||
| **9,18 seconds** -> Worker receives T1 from the server and is still unable to process the task and updates the server with a callback of 9 seconds. | ||
|
|
||
| **27 seconds** -> Worker polls and receives task T1 from the server and is now able to process this task. | ||
|
|
||
| **30 seconds** (T1 timeout) -> Server marks T1 as TIMED_OUT because it is not in a terminal state after first being moved to IN_PROGRESS status. Server schedules a new task based on the retry count. | ||
|
|
||
| **32 seconds** -> Worker completes processing of T1 and updates the server with COMPLETED status. Server will ignore this update since T1 has already been moved to a terminal status (TIMED_OUT). | ||
| **30 seconds** (T1 timeout) -> Server marks T1 as `TIMED_OUT` because it is not in a terminal state after first being moved to `IN_PROGRESS` status. Server schedules a new task based on the retry count. | ||
|
|
||
| **32 seconds** -> Worker completes processing of T1 and updates the server with `COMPLETED` status. Server will ignore this update since T1 has already been moved to a terminal status (`TIMED_OUT`). | ||
|
|
||
| ### Response timeout seconds | ||
|
|
||
| Response timeout is the time within which the worker must respond to the server with an update for the task, else the task will be marked as TIMED_OUT. | ||
|
|
||
|  | ||
|  | ||
|
|
||
| **0 seconds** -> Worker polls for the task T1 from the Conductor server and receives the task. T1 is put into IN_PROGRESS status by the server. | ||
| **0 seconds** -> Worker polls for the task T1 from the Conductor server and receives the task. T1 is put into `IN_PROGRESS` status by the server. | ||
|
|
||
| Worker starts processing the task but the worker instance dies during this execution. | ||
|
|
||
| **20 seconds** (T1 responseTimeout) -> Server marks T1 as TIMED_OUT since the task has not been updated by the worker within the configured responseTimeoutSeconds (20). A new instance of task T1 is scheduled as per the retry configuration. | ||
| **20 seconds** (T1 responseTimeout) -> Server marks T1 as `TIMED_OUT` since the task has not been updated by the worker within the configured responseTimeoutSeconds (20). A new instance of task T1 is scheduled as per the retry configuration. | ||
|
|
||
| **25 seconds** -> The retried instance of T1 is available to be polled by the worker, after the retryDelaySeconds (5) has elapsed. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Basic Concepts | ||
| Conductor allows you to build a complex application using simple and granular tasks that do not | ||
| need to be aware of or keep track of the state of your application's execution flow. Conductor keeps track of the state, | ||
| calls tasks in the right order (sequentially or in parallel, as defined by you), retry calls if needed, handle failure | ||
| scenarios gracefully, and outputs the final result. | ||
|
|
||
|
|
||
|  | ||
|
|
||
| Leveraging workflows in Conductor enables developers to truly focus on their core mission - building their application | ||
| code in the languages of their choice. Conductor does the heavy lifting associated with ensuring high | ||
| reliability, transactional consistency, and long durability of their workflows. Simply put, wherever your application's | ||
| component lives and whichever languages they were written in, you can build a workflow in Conductor to orchestrate their | ||
| execution in a reliable & scalable manner. | ||
|
|
||
| [Workflows](workflows.md) and [Tasks](tasks.md) are the two key concepts that underlie the Conductor system. | ||
|
|
Oops, something went wrong.