From 010b6e56a0c34ce4e4eddff5279fd026cd34da14 Mon Sep 17 00:00:00 2001 From: Nick Larew Date: Mon, 6 Nov 2023 16:45:09 -0500 Subject: [PATCH 1/2] (DOCSP-34109) Trigger Throughput Wording --- source/triggers.txt | 36 ++++++++++++++++----------- source/triggers/database-triggers.txt | 17 +++++++------ 2 files changed, 31 insertions(+), 22 deletions(-) diff --git a/source/triggers.txt b/source/triggers.txt index 5af2b3409..90662935e 100644 --- a/source/triggers.txt +++ b/source/triggers.txt @@ -29,7 +29,7 @@ can respond to events or use pre-defined schedules. Triggers listen for events of a configured type. Each Trigger links to a specific :doc:`Atlas Function `. When a Trigger observes an event that matches your -configuration, it *"fires."* The Trigger passes this event object as the +configuration, it *"fires"*. The Trigger passes this event object as the argument to its linked Function. A Trigger might fire on: @@ -39,7 +39,7 @@ A Trigger might fire on: - A scheduled time. App Services keeps track of the latest execution time for each -Trigger. +Trigger and guarantees that each event is processed at least once. .. _trigger-types: @@ -76,26 +76,34 @@ constraints as all Atlas Functions. Event Processing Throughput ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Trigger events are added to a queue and processed when capacity becomes -available. The rate at which a Trigger processes events depends on the -Trigger's configuration as well as the logic in the Trigger function. +Triggers process events when capacity becomes available. A Trigger's +capacity is determined by its event ordering configuration: -Ordered triggers process events from the queue one at a time. An -unordered trigger can process up to 10,000 events concurrently. +- Ordered triggers process events from the change stream one at a time + in sequence. The next event begins processing only after the previous + event finishes processing. -.. note:: +- Unordered triggers can process multiple events concurrently, up to + 10,000 at once by default. If your Trigger data source is an M10+ + Atlas cluster, you can configure individual unordered triggers to + exceed the 10,000 concurrent event threshold. To learn more, see + :ref:`Maximum Throughput Triggers `. - If the linked data source is a dedicated server (M10+ Tier), - you can configure individual triggers to process up to 50,000 events concurrently. +Trigger capacity is not a direct measure of throughput or a guaranteed +execution rate. Instead, it is a threshold for the maximum number of +events that a Trigger can process at one time. In practice, the rate at +which a Trigger can process events depends on the Trigger function's run +time logic and the number of events that it receives in a given +timeframe. To increase the throughput of a Trigger, you can try to: -- Optimize the Trigger function's run time. For example, you might - reduce the number of network calls that you make. +- Optimize the Trigger function's run time behavior. For example, you + might reduce the number of network calls that you make. - Reduce the size of each event object with the Trigger's projection - filter. For the best performance, we recommend limiting the size of - each change event to 2KB or less. + filter. For the best performance, limit the size of each change event + to 2KB or less. - Use a match filter to reduce the number of events that the Trigger processes. For example, you might want to do something only if a diff --git a/source/triggers/database-triggers.txt b/source/triggers/database-triggers.txt index 9312edab0..1147e3e1a 100644 --- a/source/triggers/database-triggers.txt +++ b/source/triggers/database-triggers.txt @@ -360,9 +360,9 @@ options are available: - .. _triggers-maximum-throughput: - If the linked data source is a dedicated server (M10+ Tier), + If the linked data source is a dedicated server (M10+ Tier), you can increase the :ref:`maximum throughput ` - from the default 10,000 concurrent processes to 50,000. + beyond the default 10,000 concurrent processes. .. important:: @@ -763,12 +763,13 @@ Consider disabling event ordering if your trigger fires on a collection that receives short bursts of events (e.g. inserting data as part of a daily batch job). -Ordered Triggers wait to execute a Function for a particular event until the -Functions of previous events have finished executing. As a consequence, -ordered Triggers are effectively rate-limited by the run time of each -sequential Trigger function. This may cause a significant delay between the -database event and the Trigger firing if a sufficiently large number of -Trigger executions are currently in the queue. +Ordered Triggers wait to execute a Function for a particular event until +the Functions of previous events have finished executing. As a +consequence, ordered Triggers are effectively rate-limited by the run +time of each sequential Trigger function. This may cause a significant +delay between the database event appearing on the change stream and the +Trigger firing. In certain extreme cases, database events might fall off +the oplog before a long-running ordered trigger processes them. Unordered Triggers execute functions in parallel if possible, which can be significantly faster (depending on your use case) but does not guarantee that From 5d0b65108ca4824878d82782ffd62356c0f285a5 Mon Sep 17 00:00:00 2001 From: Nick Larew Date: Tue, 7 Nov 2023 17:00:55 -0500 Subject: [PATCH 2/2] add link to trigger projections --- source/triggers.txt | 6 +++--- source/triggers/database-triggers.txt | 4 +++- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/source/triggers.txt b/source/triggers.txt index 90662935e..10c8f6d4c 100644 --- a/source/triggers.txt +++ b/source/triggers.txt @@ -101,9 +101,9 @@ To increase the throughput of a Trigger, you can try to: - Optimize the Trigger function's run time behavior. For example, you might reduce the number of network calls that you make. -- Reduce the size of each event object with the Trigger's projection - filter. For the best performance, limit the size of each change event - to 2KB or less. +- Reduce the size of each event object with the Trigger's + :ref:`projection filter `. For the best + performance, limit the size of each change event to 2KB or less. - Use a match filter to reduce the number of events that the Trigger processes. For example, you might want to do something only if a diff --git a/source/triggers/database-triggers.txt b/source/triggers/database-triggers.txt index 1147e3e1a..4391762c8 100644 --- a/source/triggers/database-triggers.txt +++ b/source/triggers/database-triggers.txt @@ -350,7 +350,9 @@ options are available: * - :guilabel:`Project Expression` - - .. include:: /includes/trigger-project-expression.rst + - .. _trigger-project-expression: + + .. include:: /includes/trigger-project-expression.rst * - :guilabel:`Auto-Resume Triggers`