Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Workflow] Add examples for reuse ID policy #4237

Closed
wants to merge 7 commits into from
Closed

[Workflow] Add examples for reuse ID policy #4237

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
fb3c1e0
add examples
hhunter-ms Jun 26, 2024
2ca982b
attempting to clarify
hhunter-ms Jun 27, 2024
f1ebce8
tweak first example
hhunter-ms Jun 27, 2024
1761f30
Merge branch 'v1.13' into issue_3909
hhunter-ms Jun 27, 2024
8272c18
Merge branch 'v1.13' into issue_3909
hhunter-ms Jul 8, 2024
02087f2
small update
hhunter-ms Jul 8, 2024
ba2c043
Merge branch 'issue_3909' of https://github.com/hhunter-ms/docs into …
hhunter-ms Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
attempting to clarify 
Signed-off-by: Hannah Hunter <hannahhunter@microsoft.com>
  • Loading branch information
@hhunter-ms
hhunter-ms committed Jun 27, 2024
commit 2ca982bcbb1ee5b9decbf4a33f8e588db743408a
72 changes: 67 additions & 5 deletions .../developing-applications/building-blocks/workflow/workflow-features-concepts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,17 @@ Each workflow you define has a type name, and individual executions of a workflo

If a workflow instance completes or fails, its ID can be reused by a new workflow instance. The new workflow instance effectively replaces the old one in the configured state store.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear here, you can set a policy called "api.REUSE_ID_ACTION_TERMINATE" that does force an existing workflow to terminate so you can start a new workflow with the same id.

Is this correct? Little unclear here in the wording. If so, then it would also be good to have a list of the reuse ID policies somewhere in the docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's correct

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaibocai - Can you then provide a list of the api.REUSE_ID_ACTION_XX values and what each one does in a table?


You can use the following policies to reuse workflow IDs:

| Policies | Description |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So there are only two Workflow reuse ID policies? What is the default if not set?

Why would use REUSE_ID_ACTION_IGNORE? This just means that a new workflow can never be started if there is an existing one already?

| -------- | ----------- |
| `api.REUSE_ID_ACTION_IGNORE` | Ignores the a new workflow being created with the same ID as an existing workflow if the existing workflow is `RUNNING`, `COMPLETED`, or `PENDING`. |
| `api.REUSE_ID_ACTION_TERMINATE` | Terminates a new workflow created with the same ID as an existing workflow if the existing workflow is `RUNNING`, `COMPLETED`, or `PENDING`. |

**Example 1**

The following example demonstrates `main` executing a workflow twice. In the example:
The following example demonstrates the default behavior, erroring out the workflow when reusing the workflow ID. In the example:

1. The workflow calls a single activity with orchestration ID reuse policy.
1. The reuse ID policy specifies the action `IGNORE_IF_RUNNING_OR_COMPLETED` and the target statuses of `RUNNING`, `COMPLETED`, `PENDING`.
1. The second call to create a workflow with the same instance ID is expected to be ignored if the first workflow instance is one of the target statuses.
Expand Down Expand Up @@ -70,15 +78,15 @@ func main() {
OperationStatus: []api.OrchestrationStatus{api.RUNTIME_STATUS_RUNNING, api.RUNTIME_STATUS_COMPLETED, api.RUNTIME_STATUS_PENDING},
}

// Run the orchestration
// Run the orchestration.
id, err := client.ScheduleNewOrchestration(ctx, "SingleActivity", api.WithInput("World"), api.WithInstanceID(instanceID))
if err != nil {
fmt.Println(err)
return
}
// Wait for orchestration to start
// Wait for orchestration to start...
client.WaitForOrchestrationStart(ctx, id)
// Schedule again, it should ignore creating the new orchestration
// Schedule the workflow again using the same id. However it will ignore creating the new orchestration, since the id is already in use.
id, err = client.ScheduleNewOrchestration(ctx, "SingleActivity", api.WithInput("World"), api.WithInstanceID(id), api.WithOrchestrationIdReusePolicy(reuseIDPolicy))
if err != nil {
fmt.Println(err)
Expand All @@ -87,8 +95,62 @@ func main() {
}
```


**Example 2**

The following example demonstrates `main` executing a workflow twice. In the example:
1. The workflow calls a single activity with orchestration ID reuse policy.
1. The reuse ID policy specifies the action `IGNORE_IF_RUNNING_OR_COMPLETED` and the target statuses of `RUNNING`, `COMPLETED`, `PENDING`.
1. The second call to create a workflow with the same instance ID is expected to be ignored if the first workflow instance is one of the target statuses.

```go
func main() {
r := task.NewTaskRegistry()
r.AddOrchestratorN("SingleActivity", func(ctx *task.OrchestrationContext) (any, error) {
var input string
if err := ctx.GetInput(&input); err != nil {
return nil, err
}
var output string
err := ctx.CallActivity("SayHello", task.WithActivityInput(input)).Await(&output)
return output, err
})
r.AddActivityN("SayHello", func(ctx task.ActivityContext) (any, error) {
var name string
if err := ctx.GetInput(&name); err != nil {
return nil, err
}
return fmt.Sprintf("Hello, %s!", name), nil
})

ctx := context.Background()
client, engine := startEngine(ctx, r)

instanceID := api.InstanceID("IGNORE_IF_RUNNING_OR_COMPLETED")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The string value here is intended to be the workflow ID, which is typically some business identity or is random (in which case we don't use it at all). It looks like we're trying to specify a policy as a workflow ID, which is a bit confusing. I think it would be better to not specify an explicit workflow ID at all to avoid confusion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this code all need to change to workflow example, this can be changed at the same time.

reuseIDPolicy := &api.OrchestrationIdReusePolicy{
Action: api.REUSE_ID_ACTION_IGNORE,
OperationStatus: []api.OrchestrationStatus{api.RUNTIME_STATUS_RUNNING, api.RUNTIME_STATUS_COMPLETED, api.RUNTIME_STATUS_PENDING},
}

// Run the orchestration.
id, err := client.ScheduleNewOrchestration(ctx, "SingleActivity", api.WithInput("World"), api.WithInstanceID(instanceID))
if err != nil {
fmt.Println(err)
return
}
// Wait for orchestration to start...
client.WaitForOrchestrationStart(ctx, id)
// Schedule the workflow again using the same id. However it will ignore creating the new orchestration, since the id is already in use.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean "ignore". You mean nothing happens (which is strange), or it will eventually get rescheduled. A silent failure event would be really hard to determine. We need to be clearer here on what happens IMO. Can you explain this more and why I would choose this option?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaibocai - See my questions. It would help to clarify clearly when these actions are used.
Action: api.REUSE_ID_ACTION_IGNORE

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean "ignore". You mean nothing happens (which is strange), or it will eventually get rescheduled. A silent failure event would be really hard to determine. We need to be clearer here on what happens IMO. Can you explain this more and why I would choose this option?

@kaibocai an explanation of this would help me as well!

Copy link
Contributor

@kaibocai kaibocai Jun 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so when you want to start an orchestration but only when it's not in running, then you can use this option. More discussion details can be found microsoft/durabletask-go#42

Copy link
Member

@msfussell msfussell Jul 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaibocai - Ok, but this is a discussion and there is no final conclusion here of what was actually implemented. We need a table for the reuse ID options and what each one does. Can you add this?

id, err = client.ScheduleNewOrchestration(ctx, "SingleActivity", api.WithInput("World"), api.WithInstanceID(id), api.WithOrchestrationIdReusePolicy(reuseIDPolicy))
if err != nil {
fmt.Println(err)
return
}
}
```

**Example 3**

In the following example:
1. The workflow calls a single activity with the orchestration ID reuse policy.
1. The reuse ID policy contains the action to `TERMINATE` and target statuses `RUNNING`, `COMPLETED`, and `PENDING`.
Expand Down Expand Up @@ -131,7 +193,7 @@ func main() {
}
// Wait for orchestration to start
client.WaitForOrchestrationStart(ctx, id)
// Schedule again, it should ignore creating the new orchestration
// Schedule again. This time, the workflow is successfully started (not ignored), since the policy terminates the existing workflow with the id and starts a new one
id, err = client.ScheduleNewOrchestration(ctx, "SingleActivity", api.WithInput("World"), api.WithInstanceID(id), api.WithOrchestrationIdReusePolicy(reuseIDPolicy))
if err != nil {
fmt.Println(err)
Expand Down
Loading