From d856b262a33cd79b055c11d5c655647ef00212e1 Mon Sep 17 00:00:00 2001 From: MongoCaleb <32645888+MongoCaleb@users.noreply.github.com> Date: Wed, 1 Nov 2023 14:39:03 -0700 Subject: [PATCH] New Database Trigger Options (#611) --- source/reference/config/triggers.txt | 31 ++- source/triggers/database-triggers.txt | 307 +++++++++++++++++++------- 2 files changed, 253 insertions(+), 85 deletions(-) diff --git a/source/reference/config/triggers.txt b/source/reference/config/triggers.txt index 3e926022f..04fe68e39 100644 --- a/source/reference/config/triggers.txt +++ b/source/reference/config/triggers.txt @@ -96,8 +96,8 @@ Database Triggers Database trigger configurations conform to the base trigger schema with additional configuration options that specify which collection to watch and when -to fire the trigger. The following fields exist in *database* trigger -configuration files: +to fire the trigger. The following fields exist in *database* and *deployment* +trigger configuration files: .. code-block:: json :caption: triggers/.json @@ -149,12 +149,28 @@ configuration files: - A list of one or more :ref:`database operation types ` that cause the trigger to fire. - Valid operations types: + Valid operations types for all triggers: - ``"INSERT"`` - ``"UPDATE"`` - ``"REPLACE"`` - ``"DELETE"`` + + Valid operations types for database and deployment triggers: + + - ``"CREATE_COLLECTION"`` + - ``"MODIFY_COLLECTION"`` + - ``"RENAME_COLLECTION"`` + - ``"SHARD_COLLECTION"`` + - ``"DROP_COLLECTION"`` + - ``"RESHARD_COLLECTION"`` + - ``"REFINE_COLLECTION_SHARD_KEY"`` + - ``"CREATE_INDEXES"`` + - ``"DROP_INDEXES"`` + + Valid operations types for deployment triggers only: + + - ``"DROP_DATABASE"`` .. tip:: @@ -172,12 +188,9 @@ configuration files: .. note:: - Regardless of this setting: - - - ``INSERT`` and ``REPLACE`` events always include the - ``fullDocument`` field. - - - ``DELETE`` events never include the ``fullDocument`` field. + Regardless of this setting, ``INSERT`` and ``REPLACE`` events always + include the``fullDocument`` field. ``DELETE`` events never include the + ``fullDocument`` field. * - | ``config.full_document_before_change`` | ``boolean`` diff --git a/source/triggers/database-triggers.txt b/source/triggers/database-triggers.txt index 718caf11a..9312edab0 100644 --- a/source/triggers/database-triggers.txt +++ b/source/triggers/database-triggers.txt @@ -10,8 +10,10 @@ Database Triggers :depth: 2 :class: singlecol -Database Triggers allow you to execute server-side logic whenever a -document is added, updated, or removed in a linked MongoDB Atlas cluster. +Database Triggers allow you to execute server-side logic whenever a database +change occurs on a linked MongoDB Atlas cluster. You can configure triggers on +individual collections, entire databases, and on an entire cluster. + Unlike SQL data triggers, which run on the database server, triggers run on a serverless compute layer that scales independently of the database server. Triggers automatically call :ref:`Atlas Functions ` @@ -30,18 +32,6 @@ single change stream for each collection with at least one enabled trigger. If multiple triggers are enabled for a collection they all share the same change stream. -You control which operations cause a trigger to fire as well as what -happens when it does. For example, you can run a function whenever a -specific field of a document is updated. The function can access the -entire change event, so you always know what changed. You can also pass -the change event to :ref:`AWS EventBridge ` to handle -the event outside of Atlas. - -Triggers support :manual:`$match -` expressions to filter change -events and :manual:`$project ` -expressions to limit the data included in each event. - .. important:: Change Stream Limitations There are limits on the total number of change streams you can open @@ -53,6 +43,27 @@ expressions to limit the data included in each event. ` or :ref:`{+adf-instance+} ` because they do not support change streams. +You control which operations cause a trigger to fire as well as what +happens when it does. For example, you can run a function whenever a +specific field of a document is updated. The function can access the +entire change event, so you always know what changed. You can also pass +the change event to :ref:`AWS EventBridge ` to handle +the event outside of Atlas. + +Triggers support :manual:`$match ` +expressions to filter change events and :manual:`$project ` +expressions to limit the data included in each event. + +.. _trigger-recursion: + +.. warning:: + + In deployment and database level triggers, it is possible to configure triggers + in a way that causes other triggers to fire, resulting in recursion. + Examples include a database-level trigger writing to a collection within the + same database, or a cluster-level logger or log forwarder writing logs to + another database in the same cluster. + .. _create-a-database-trigger: Create a Database Trigger @@ -103,6 +114,9 @@ Configuration Database Triggers have the following configuration options: +Trigger Details +~~~~~~~~~~~~~~~ + .. list-table:: :header-rows: 1 :widths: 15 30 @@ -136,84 +150,190 @@ Database Triggers have the following configuration options: .. include:: /includes/trigger-event-ordering.rst -Within the :guilabel:`Trigger Source Details` section, the following configuration -options are available: +Trigger Source Details +~~~~~~~~~~~~~~~~~~~~~~ -.. list-table:: - :header-rows: 1 - :widths: 15 30 +Within the :guilabel:`Trigger Source Details` section, you first select the +:guilabel:`Watch Against`, based on the level of granularity you want. Your +options are: - * - Field - - Description +- :guilabel:`Collection`, when a change occurs on a specified collection +- :guilabel:`Database`, when a change occurs on any collection in a + specified database +- :guilabel:`Deployment`, when deployment changes occur on a specified + cluster. If you select the Deployment source type, the following + databases are **not watched** for changes: - * - :guilabel:`Cluster Name` + - The admin databases ``admin``, ``local``, and ``config`` + - The sync databases ``__realm_sync`` and ``__realm_sync_`` - - The name of the MongoDB Service that the Trigger is - associated with. + .. important:: - * - :guilabel:`Database Name` + The deployment-level source type is only available on dedicated tiers. + +Depending on which source type you are using, the additional options differ. The +following table describes these options. + +.. list-table:: + :header-rows: 1 + :widths: 15 30 - - The MongoDB database that contains the watched - collection. + * - Source Type + - Options - * - :guilabel:`Collection Name` + * - :guilabel:`Collection` + - | - - The name of the collection that the Trigger watches for - change events. + - :guilabel:`Cluster Name`. The name of the MongoDB cluster that the + Trigger is associated with. + + - :guilabel:`Database Name`. The MongoDB database that contains the watched + collection. - * - :guilabel:`Operation Type` + - :guilabel:`Collection Name`. The MongoDB collection to watch. Optional. + If you leave this option blank, the Source Type changes to "Database." - - Checkboxes of :ref:`database operation types ` - that cause the Trigger to fire. Select the operation types you want the - trigger to respond to. + - :guilabel:`Operation Type`. The The :ref:`operation types + ` that cause the Trigger to fire. + Select the operation types you want the trigger to respond to. Options + include: + + - Insert + - Update + - Replace + - Delete - .. warning:: + .. note:: + + Update operations executed from MongoDB Compass or the MongoDB Atlas + Data Explorer fully replace the previous document. As a result, + update operations from these clients will generate **Replace** + change events rather than **Update** events. + + - :guilabel:`Full Document`. If enabled, **Update** change events include + the latest :manual:`majority-committed ` + version of the modified document *after* the change was applied in + the ``fullDocument`` field. + + .. note:: + + Regardless of this setting, **Insert** and **Replace** events always + include the ``fullDocument`` field. **Delete** events never include + the ``fullDocument`` field. + + - :guilabel:`Document Preimage`. When enabled, change events include a + copy of the modified document from immediately *before* the change was + applied in the ``fullDocumentBeforeChange`` field. This has + :ref:`performance considerations `. All change events + except for **Insert** events include the document preimage. - Update operations executed from MongoDB Compass or the MongoDB Atlas - Data Explorer fully replace the previous document. As a result, - update operations from these clients will generate **Replace** - change events rather than **Update** events. + * - :guilabel:`Database` + - | - * - :guilabel:`Full Document` + - :guilabel:`Cluster Name`. The name of the MongoDB cluster that the + Trigger is associated with. + + - :guilabel:`Database Name`. The MongoDB database to watch. Optional. + If you leave this option blank, the Source Type changes to "Deployment," + unless you are on a shared tier, in which case App Services will not + let you save the trigger. + + - :guilabel:`Operation Type`. The :ref:`operation types + ` that cause the Trigger to fire. + Select the operation types you want the trigger to respond to. + Options include: + + - Create Collection + - Modify Collection + - Rename Collection + - Drop Collection + - Shard Collection + - Reshard Collection + - Refine Collection Shard Key + + .. note:: + + Update operations executed from MongoDB Compass or the MongoDB Atlas + Data Explorer fully replace the previous document. As a result, + update operations from these clients will generate **Replace** + change events rather than **Update** events. + + - :guilabel:`Full Document`. If enabled, **Update** change events include + the latest :manual:`majority-committed ` + version of the modified document *after* the change was applied in + the ``fullDocument`` field. + + .. note:: + + Regardless of this setting, **Insert** and **Replace** events always + include the ``fullDocument`` field. **Delete** events never include + the ``fullDocument`` field. + + - :guilabel:`Document Preimage`. When enabled, change events include a + copy of the modified document from immediately *before* the change was + applied in the ``fullDocumentBeforeChange`` field. This has + :ref:`performance considerations `. All change events + except for **Insert** events include the document preimage. Disabled + for Database and Deployment sources to limit unnecessary watches on the + cluster for a new collection being created. + + * - :guilabel:`Deployment` + - | + + - :guilabel:`Cluster Name`. The name of the MongoDB cluster that the + Trigger is associated with. + + - :guilabel:`Operation Type`. The The :ref:`operation types + ` that occur in the cluster that cause + the Trigger to fire. Select the operation types you want the trigger + to respond to. Options include: - - If enabled, **Update** change events include the latest - :manual:`majority-committed ` - version of the modified document *after* the change was applied in - the ``fullDocument`` field. + - Drop Database - .. note:: + - :guilabel:`Full Document`. If enabled, **Update** change events include + the latest :manual:`majority-committed ` + version of the modified document *after* the change was applied in + the ``fullDocument`` field. + + .. note:: + + Regardless of this setting, **Insert** and **Replace** events always + include the ``fullDocument`` field. **Delete** events never include + the ``fullDocument`` field. - Regardless of this setting: - - - **Insert** and **Replace** events always include the - ``fullDocument`` field. - - - **Delete** events never include the ``fullDocument`` field. + - :guilabel:`Document Preimage`. When enabled, change events include a + copy of the modified document from immediately *before* the change was + applied in the ``fullDocumentBeforeChange`` field. This has + :ref:`performance considerations `. All change events + except for **Insert** events include the document preimage. Disabled + for Database and Deployment sources to limit unnecessary watches on the + cluster for a new collection being created. - * - :guilabel:`Document Preimage` +.. _pre-image-perf: - - If enabled, change events include a copy of the modified document - from immediately *before* the change was applied in the - ``fullDocumentBeforeChange`` field. All change events except for - **Insert** events include the document preimage. +.. tip:: Preimages and Performance Optimization - .. tip:: Performance Optimization + Preimages require additional storage overhead that may affect + performance. If you're not using preimages on a collection, + you should disable preimages. To learn more, see :ref:`Disable + Collection-Level Preimages + `. - Preimages require additional storage overhead that may affect - performance. If you're not using preimages on a collection, - you should disable preimages. To learn more, see :ref:`Disable - Collection-Level Preimages - `. + Document preimages are supported on non-sharded Atlas clusters running + MongoDB 4.4+, and on sharded Atlas clusters running MongoDB 5.3 and later. + You can upgrade a non-sharded cluster (with preimages) to a + sharded cluster, as long as the cluster is running 5.3 or later. - Document preimages are supported on non-sharded Atlas clusters running - MongoDB 4.4+, and on sharded Atlas clusters running MongoDB 5.3 and later. - You can upgrade a non-sharded cluster (with preimages) to a - sharded cluster, as long as the cluster is running 5.3 or later. +Function +~~~~~~~~ Within the :guilabel:`Function` section, you choose what action is taken when the trigger fires. You can choose to run a :ref:`function ` or use :ref:`AWS EventBridge `. +Advanced +~~~~~~~~ + Within the :guilabel:`Advanced` section, the following **optional** configuration options are available: @@ -257,8 +377,8 @@ options are available: .. _database-events: -Database Change Events ----------------------- +Change Event Types +------------------ .. _database-event-operation-types: @@ -272,23 +392,52 @@ following operation types: .. list-table:: :header-rows: 1 - :widths: 10 30 + :widths: 30 15 * - Operation Type - Description - * - **Insert** + * - **Insert Document** (All trigger types) - Represents a new document added to the collection. - * - **Update** + * - **Update Document** (All trigger types) - Represents a change to an existing document in the collection. - * - **Delete** + * - **Delete Document** (All trigger types) - Represents a document deleted from the collection. - * - **Replace** + * - **Replace Document** (All trigger types) - Represents a new document that replaced a document in the collection. + * - **Create Collection** (Database and Deployment trigger types only) + - Represents the creation of a new collection. + + * - **Modify Collection** (Database and Deployment trigger types only) + - Represents the modification collection. + + * - **Rename Collection** (Database and Deployment trigger types only) + - Represents collection being renamed. + + * - **Drop Collection** (Database and Deployment trigger types only) + - Represents a collection being dropped. + + * - **Shard Collection** (Database and Deployment trigger types only) + - Represents a collection changing from unsharded to sharded. + + * - **Reshard Collection** (Database and Deployment trigger types only) + - Represents a change to a collection's sharding. + + * - **Refine Collection Shard Key** (Database and Deployment trigger types only) + - Represents a change in the shard key of a collection. + + * - **Create Indexes** (Database and Deployment trigger types only) + - Represents the creation of a new index. + + * - **Drop Indexes** (Database and Deployment trigger types only) + - Represents an index being dropped. + + * - **Drop Database** (Deployment trigger type only) + - Represents a database being dropped. Database change event objects have the following general form: @@ -334,7 +483,7 @@ collection as a document that resembles the following: To automate this process, the store creates a Database Trigger that listens for **Update** change events in the ``store.orders`` collection. -When the trigger observes an **Update** event, it passes the +When the trigger observes an **Update** event, it passes the :ref:`change event object ` to its associated Function, ``textShippingUpdate``. The Function checks the change event for any changes to the ``shippingLocation`` field and, if it was updated, sends @@ -675,8 +824,14 @@ the trigger to fire. For details, see the reference for each event type: - :manual:`update ` - :manual:`replace ` - :manual:`delete ` - - +- :manual:`create ` +- :manual:`modify ` +- :manual:`rename ` +- :manual:`drop ` +- :manual:`shardCollection ` +- :manual:`reshardCollection ` +- :manual:`refineCollectionShardKey ` +- :manual:`dropDatabase ` .. example::