Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): adds events based history #7922

Merged
merged 53 commits into from
Dec 9, 2024
Merged

feat(core): adds events based history #7922

merged 53 commits into from
Dec 9, 2024

Conversation

pedrobonamin
Copy link
Contributor

@pedrobonamin pedrobonamin commented Nov 29, 2024

Description

Introduces the document group events and adds a feature flag to decide wether to use the new events or the old timeline.
It also refactors the DocumentPaneProvider to receive the store by props, and support switching between events and timeline

Events.API.studio.changes.mov

How events connect with each other and through variants?

From a published document you can visit drafts and versions, because a draft or a version when it's published it impacts the published document.
But drafts and versions are not related between them.

Screenshot 2024-12-02 at 16 26 19

Document group Events

Events API

The Events API introduces a new mechanism for accessing DocumentGroupEvent entries derived from the translog. These events are accessible through the endpoint:

This events are accessible through:

/data/events/<dataset/documents/<documentId>

This API is designed to be lightweight, providing only the essential data required to render the UI. Additionally, it offers insights into the document variants impacted by each event.

Pre read

With the introduction of releases, users now select the DocumentVariantType (draft | published | version) they wish to work with in the form. This represents a significant shift from the previous behavior, where users interacted only with two types of documents published or `draft —without the ability to focus on one or the other independently.

How useEventsStore works:

When a user visits a document, the useEventsStore retrieves events relevant to the selected document variant (draft, version, or published). It also sets up a listener using getRemoteTransactions to monitor real-time transactions for the document.

When a new transaction is received, the system determines whether to reload the events or append the transaction to the existing event list. This decision depends on the transaction type and the specific document variant.

The retrieved events are displayed in the History Inspector. To render a diff, the system uses the getChangesList function, which fetches the transactions between two selected document revisions (since and revision). This process involves:

  1. Retrieving the full documents for the specified revisions from the history endpoint.
  2. Comparing the two documents to identify differences.
  3. Fetching transactions to enable attribution rendering, showing detailed information about who made the changes, what was modified, and when.

How it worked before:

Previously, we used the Timeline class, which always fetched transactions from the translog. It also set up listeners for incoming transactions, using both the draft and published IDs. The system then mapped these transactions into human-readable events like edit, publish, or live edit.

Why this is changing

The previous approach was tailored to a simpler model with only two variants (draft and published). It displayed the changes for both document types in a unified view. However, this model does not align with the new paradigm of multiple document variants.

Now, each document variant has its own dedicated view, showing only the changes relevant to that specific variant.

Key updates:

  • Draft document: Shows only changes made to the draft.
  • Version document: Displays changes specific to the selected version.
  • Published document: Includes changes made to the published document, specifying whether the change originated from publishing a draft or version, or from a live edit.

Types of Events and Document Lifecycle

Each event represents a significant state change in the lifecycle of a document, its versions, and its published state. These events help to track the creation, modification, scheduling, publication and deletion of documents.

Version Lifecycle

A version is a short-lived document that undergoes creation, editing, and publication or deletion. Once published, the same ID will not be reused for a new version. The events affecting a version include:

  • CreateDocumentVersionEvent: Triggered when a version is created.
  • DeleteDocumentVersionEvent: Occurs if a version is deleted, though this is uncommon.
  • PublishDocumentVersionEvent: Marks the publication of the version as part of a release.
  • ScheduleDocumentVersionEvent (not implemented yet): Indicates that the version's release has been scheduled.
  • UnscheduleDocumentVersionEvent (not implemented yet): Indicates that the version's release has been unscheduled.
  • EditDocumentVersionEvent: Triggered when the version is edited.

Draft Document

Draft documents are long-lived documents, which could be published and recreated multiple times. Events specific to drafts include:

  • CreateDocumentVersionEvent: Occurs when a new draft is created, after publishing or deleting a document.
  • DeleteDocumentVersionEvent: Triggered when a draft is discarded.
  • PublishDocumentVersionEvent: Indicates that the draft has been published, either through manual action or a scheduled release.
  • ScheduleDocumentVersionEvent (not implemented yet): Indicates that the draft has been scheduled for publication.
  • UnscheduleDocumentVersionEvent (not implemented yet): Indicates that a scheduled draft has been unscheduled.
  • EditDocumentVersionEvent: Triggered when edits are made to the draft.

Published Document

Published documents represent the live, publicly available version of a document. Events specific to published documents include:

  • UnpublishDocumentEvent: Triggered when the document is unpublished. This deletes the live document and creates a draft with the published document's data if no draft exists.
  • CreateLiveDocumentEvent: Occurs when a live document is created outside the typical publish action, such as through live edits or API-created documents.
  • UpdateLiveDocumentEvent: Triggered when a live document is edited directly, typically in live edit workflows or through API actions bypassing the publish process.
  • PublishDocumentVersionEvent: Captures when a draft or version document is published. This event includes details such as the ID and revision of the document that initiated the modification.

Edit Event

The Edit Event is a special case. It is not returned directly by the API but is generated within the Studio by:

  1. Fetching transactions from the translog.
  2. Processing new transactions as they are received in real-time through change listeners.

What to review

  • DocumentPaneProvider shows more changes than what it really has. I needed to refactor what we are exporting from it because we need to decide wether to use the legacy store or the new one. This is for backwards compatibility and to ensure that disabling the events api works. So it:
    • Update the export of DocumentPaneProvider to a new component that decides wether to use the events store or the timeline store.
    • Rename DocumentPaneProvider to DocumentPaneProviderInner and receive the necessary store data as props in the HistoryStoreProps param
  • Usage and definition of the events store .

Testing

Will be added in a follow up PR, ticket corel-235

Notes for release

Copy link

vercel bot commented Nov 29, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 9, 2024 4:05pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 9, 2024 4:05pm
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 9, 2024 4:05pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 9, 2024 4:05pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Dec 9, 2024 4:05pm

@@ -262,6 +262,7 @@
"tar-fs": "^2.1.1",
"tar-stream": "^3.1.7",
"use-device-pixel-ratio": "^1.1.0",
"use-effect-event": "^1.0.2",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already part of other packages installed with sanity, like @sanity/ui , portable-text etc.

➜  sanity git:(corel-history) ✗ pnpm why use-effect-event
Legend: production dependency, optional only, dev only

[email protected] /Users/pedrobonamin/code/sanity

devDependencies:
@sanity/tsdoc 1.0.134
└─┬ @sanity/ui 2.10.0-corel.0
  └── use-effect-event 1.0.2
@sanity/ui 2.10.0-corel.0
└── use-effect-event 1.0.2
sanity link:packages/sanity
├─┬ @portabletext/editor 1.11.3
│ └── use-effect-event 1.0.2
├─┬ @sanity/insert-menu 1.0.11-release.4
│ └─┬ @sanity/ui 2.10.0-corel.0
│   └── use-effect-event 1.0.2
├─┬ @sanity/presentation 1.17.8-release.4
│ ├─┬ @sanity/ui 2.10.0-corel.0
│ │ └── use-effect-event 1.0.2
│ └── use-effect-event 1.0.2
├─┬ @sanity/ui 2.10.0-corel.0
│ └── use-effect-event 1.0.2
├─┬ react-rx 4.1.7
│ └── use-effect-event 1.0.2
└── use-effect-event 1.0.2

Copy link
Contributor

No changes to documentation

Copy link
Contributor

github-actions bot commented Nov 29, 2024

Component Testing Report Updated Dec 9, 2024 3:56 PM (UTC)

✅ All Tests Passed -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 11s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 14s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 41s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 54s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 29s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 16s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 3m 0s 0 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 12s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 2m 51s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 48s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 14s 3 9 0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx ✅ Passed (Inspect) 43s 9 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 29s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 58s 12 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Nov 29, 2024

⚡️ Editor Performance Report

Updated Mon, 09 Dec 2024 15:58:02 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 21.7 efps (46ms) 14.8 efps (68ms) +22ms (+46.7%) 🔴
article (body) 51.8 efps (19ms) 43.9 efps (23ms) +3ms (+18.1%)
article (string inside object) 22.2 efps (45ms) 16.3 efps (62ms) +17ms (+36.7%) 🔴
article (string inside array) 18.9 efps (53ms) 14.3 efps (70ms) +17ms (+32.1%) 🔴
recipe (name) 46.5 efps (22ms) 28.6 efps (35ms) +14ms (+62.8%) 🔴
recipe (description) 55.6 efps (18ms) 29.9 efps (34ms) +16ms (+86.1%) 🔴
recipe (instructions) 99.9+ efps (6ms) 99.9+ efps (7ms) +1ms (-/-%)
synthetic (title) 17.2 efps (58ms) 6.8 efps (146ms) +88ms (+151.7%) 🔴
synthetic (string inside object) 17.5 efps (57ms) 7.1 efps (140ms) +83ms (+145.6%) 🔴

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 46ms 54ms 63ms 361ms 591ms 11.8s
article (body) 19ms 24ms 34ms 99ms 273ms 6.0s
article (string inside object) 45ms 48ms 55ms 245ms 264ms 7.7s
article (string inside array) 53ms 56ms 66ms 104ms 193ms 8.3s
recipe (name) 22ms 24ms 28ms 41ms 0ms 7.1s
recipe (description) 18ms 20ms 23ms 42ms 0ms 4.6s
recipe (instructions) 6ms 9ms 10ms 13ms 0ms 3.3s
synthetic (title) 58ms 60ms 62ms 124ms 417ms 14.9s
synthetic (string inside object) 57ms 63ms 75ms 734ms 1472ms 9.8s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 68ms 75ms 91ms 265ms 1563ms 14.6s
article (body) 23ms 28ms 49ms 249ms 426ms 7.3s
article (string inside object) 62ms 65ms 75ms 285ms 1359ms 9.9s
article (string inside array) 70ms 75ms 82ms 358ms 1868ms 10.1s
recipe (name) 35ms 38ms 49ms 80ms 81ms 8.7s
recipe (description) 34ms 35ms 41ms 94ms 24ms 6.2s
recipe (instructions) 7ms 9ms 10ms 17ms 5ms 3.3s
synthetic (title) 146ms 158ms 183ms 709ms 7744ms 24.5s
synthetic (string inside object) 140ms 148ms 166ms 611ms 6225ms 16.4s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

const documentTransactionsCache: Record<string, TransactionLogEventWithEffects[]> =
Object.create(null)

// Transactions could be cached, given they are not gonna change EVER.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a ticket to do this as a follow up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably a good idea to use an LRU, but you probably also already thought of that :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, what makes it a bit challenging and why I created another ticket to do this change is that we could reuse transactions that we have already fetched.
Given transactions are not changing and we know the order.

For example:

  1. User inspect the diff from rev1 to rev10 , we load all the transactions between this two revisions.

  2. User inspect the diff from rev4 to rev5, we are loading again the transactions between this two, but we could reuse the ones loaded in the first diff, given they are contained by this.

  3. User inspects rev8 to rev15 , we are loading from rev8, we could avoid this and load only from rev10 because we already have from 8 to 10 loaded from the action 1

Copy link
Member

@bjoerge bjoerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid work, @pedrobonamin, lets get this merged! 👏🏼

@pedrobonamin pedrobonamin merged commit 7b28097 into corel Dec 9, 2024
54 of 56 checks passed
@pedrobonamin pedrobonamin deleted the corel-history branch December 9, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants