Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evented architecture for moderation system #1617

Merged
merged 98 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
5ea9064
:construction: WIP with proposed lexicons for event based mod archite…
foysalit Sep 16, 2023
5445a6b
:construction: Remove unnecessary moderation action lexicon
foysalit Sep 16, 2023
2b5801c
Merge branch 'main' of github.com:bluesky-social/atproto into moderat…
foysalit Sep 20, 2023
b62a397
:construction: Working on event based actions
foysalit Sep 26, 2023
8781c87
:rightwards_twisted_arrows: Merge with upstream
foysalit Sep 27, 2023
39a81fd
:sparkles: Add escalated subject status
foysalit Sep 28, 2023
1637548
Merge branch 'main' of github.com:bluesky-social/atproto into moderat…
foysalit Sep 28, 2023
e1e09b3
:bug: Alright, fixed the error in lexicon
foysalit Sep 28, 2023
619eed5
:construction: Working through reversal
foysalit Sep 28, 2023
c8c1a55
:sparkles: Cleanup build errors
foysalit Sep 29, 2023
dfc4fec
:sparkles: Add subject status endpoint
foysalit Sep 29, 2023
75625a7
:sparkles: Add handler
foysalit Oct 1, 2023
a82d3d6
:sparkles: get reports from mod actions table
foysalit Oct 3, 2023
f734703
:rightwards_twisted_arrows: Merge with upstream
foysalit Oct 3, 2023
2cc6f1d
:rightwards_twisted_arrows: Merge with upstream
foysalit Oct 3, 2023
f8b3b78
:construction: Builds but test network doesnt start
foysalit Oct 5, 2023
84d79a8
:sparkles: Tests passing on event based status change
foysalit Oct 6, 2023
6b652ef
:sparkles: Rename index
foysalit Oct 9, 2023
cc87dcb
:recycle: Rename takeModerationAction->emitModerationEvent
foysalit Oct 9, 2023
10ee0e5
:sparkles: Implement label reversal
foysalit Oct 9, 2023
dacbb3f
:white_check_mark: Auto-revert test working
foysalit Oct 10, 2023
5a9bdd9
:recycle: :white_check_mark: Refactored to event types and tests are …
foysalit Oct 11, 2023
ddce910
:sparkles: Add takedown event sequence validation
foysalit Oct 11, 2023
164f391
:sparkles: Adds support for blobCid status
foysalit Oct 13, 2023
82870ec
:broom: Cleanup unnecessary method:
foysalit Oct 13, 2023
87364a8
:sparkles: Hydrate handles with status and events
foysalit Oct 20, 2023
79f4c97
:sparkles: Re-implement auto reversal
foysalit Oct 23, 2023
a07f64f
:sparkles: Add takendown and mute filters
foysalit Oct 24, 2023
d71b910
:sparkles: Allow filtering events by type
foysalit Oct 26, 2023
aea0b55
:sparkles: Allow filtering events by creator did
foysalit Oct 26, 2023
afb9a8b
:sparkles: Add subjectStatus to record and repoview
foysalit Oct 30, 2023
cc756ad
:sparkles: Add persistent note feature
foysalit Oct 31, 2023
f5204b0
:sparkles: Log send email event
foysalit Nov 1, 2023
23cea8e
:bug: Fix logging send email event
foysalit Nov 2, 2023
85e8322
:sparkles: Better type
foysalit Nov 2, 2023
3b0e41c
:sparkles: Adjust migration to create separate moderation_event table
foysalit Nov 2, 2023
3b3a1a9
:broom: Cleanup types
foysalit Nov 2, 2023
1ca8427
:white_check_mark: Adjust tests with mod event emitter
foysalit Nov 2, 2023
c442738
:rightwards_twisted_arrows: Merge with upstream
foysalit Nov 3, 2023
9973b8e
:sparkles: Fix more tests around takedowns
foysalit Nov 3, 2023
62ea6df
:white_check_mark: Get test suite to pass
foysalit Nov 3, 2023
efc4eaa
:white_check_mark: Get test suite to pass for pds
foysalit Nov 3, 2023
eb766bd
:white_check_mark: Get test suite to pass for pds
foysalit Nov 3, 2023
bda203d
:white_check_mark: Update snapshot for feedgen
foysalit Nov 3, 2023
aa9ac28
:white_check_mark: Why are more snapshots updating?
foysalit Nov 3, 2023
226b87e
:recycle: Rename getModerationEvents -> queryModerationEvents
foysalit Nov 3, 2023
8c1f026
:recycle: Rename getModerationStatuses -> queryModerationStatuses
foysalit Nov 3, 2023
1ca527b
:recycle: Rename persistNote->sticky
foysalit Nov 3, 2023
be425b9
:bug: Rename subject
foysalit Nov 3, 2023
677e2e8
:recycle: Cleanup expiresAt for scheduled actions
foysalit Nov 3, 2023
902e6b4
:sparkles: Add more tests, allow fetching mod history for all content…
foysalit Nov 8, 2023
96395e6
:white_check_mark: Fix repo and record tests
foysalit Nov 8, 2023
49f381f
:sparkles: Migrate reports and actions to events
foysalit Nov 9, 2023
39cf166
:bug: Fix escalated status overwrite
foysalit Nov 10, 2023
1bf5acc
:sparkles: Implement direct sql query to create events from actions a…
foysalit Nov 11, 2023
143e569
:construction: Adding keyset pagination for subject statuses
foysalit Nov 11, 2023
d220257
:sparkles: Add migration for lastReportedAt
foysalit Nov 12, 2023
6e76210
:sparkles: Migrate blob cids
foysalit Nov 12, 2023
dc57dc8
:twisted_rightwards_arrows: Merge with upstream
foysalit Nov 13, 2023
006e0dc
:sparkles: Fix pagination on mod subject list endpoint
foysalit Nov 14, 2023
2792f9b
:bug: Fix blob actions
foysalit Nov 15, 2023
d44b683
:white_check_mark: All tests passing on bsky package
foysalit Nov 15, 2023
6c034bd
:white_check_mark: Bring back snapshots
foysalit Nov 15, 2023
741aef4
:white_check_mark: Skipping timeline test temporarily
foysalit Nov 15, 2023
f8a9bbe
:white_check_mark: Skipping some more tests to isolate failing ones
foysalit Nov 15, 2023
554af17
Merge branch 'main' of github.com:bluesky-social/atproto into moderat…
foysalit Nov 15, 2023
56f3a61
:white_check_mark: Bring back list-feed test
foysalit Nov 15, 2023
90c8241
:white_check_mark: Bring back timeline test
foysalit Nov 15, 2023
e4bd6d6
:white_check_mark: Fix label action in seeding
foysalit Nov 15, 2023
9f11c6c
:white_check_mark: Enable timeline proxied test
foysalit Nov 15, 2023
2617b4d
:white_check_mark: Enable search actor proxied test
foysalit Nov 15, 2023
c25168f
:white_check_mark: Enable feedgen tests
foysalit Nov 16, 2023
bcb475e
:white_check_mark: Fix test for admin/get-record
foysalit Nov 16, 2023
79b0473
:sparkles: Move note to comment for subject status
foysalit Nov 17, 2023
954b523
:sparkles: Accept comments in mute event
foysalit Nov 17, 2023
e6e8416
:sparkles: Remap flag event to ack event
foysalit Nov 20, 2023
a608eb6
:bug: Add legacyRef in report union selection
foysalit Nov 20, 2023
82fb94f
Merge remote-tracking branch 'origin/main' into moderation-api-v2
devinivy Nov 20, 2023
31d1bb2
Merge branch 'moderation-api-v2' of github.com:bluesky-social/atproto…
devinivy Nov 20, 2023
a286b84
@atproto/api 0.6.24-next.0
devinivy Nov 21, 2023
a93a162
@atproto/api 0.6.24-next.1
devinivy Nov 21, 2023
22bd5f0
:sparkles: Adjust migration export and add index for blobCids column
foysalit Nov 21, 2023
c3d0093
Merge branch 'moderation-api-v2' of github.com:bluesky-social/atproto…
foysalit Nov 21, 2023
0d1eb3f
:sparkles: Maintin action ids when migrating
foysalit Nov 21, 2023
404a42f
:sparkles: Paginate events using createdAt timestamp
foysalit Nov 21, 2023
2d1f542
:white_check_mark: Update snapshot for pds test with events cursor up…
foysalit Nov 21, 2023
91d32d4
:white_check_mark: Use only events for snapshot testing
foysalit Nov 21, 2023
006372f
:white_check_mark: Use only events for snapshot in the remaining test
foysalit Nov 21, 2023
a2e6ea6
relative paths to lexicons for build
devinivy Nov 22, 2023
d21b3d9
fix bsky periodic event reversal in service entrypoint
devinivy Nov 22, 2023
0697ffc
:sparkles: Allow comments in takedown and label
foysalit Nov 22, 2023
99aef0a
:sparkles: Only import reports on consecutive run of the migration sc…
foysalit Nov 28, 2023
655fe09
:sparkles: Adjust moderation property of blob entries
foysalit Nov 28, 2023
d69bb10
determine latest reports to migrate
devinivy Nov 29, 2023
48253a8
:sparkles: Process new reports for subject status
foysalit Nov 29, 2023
a8b95bf
:sparkles: Process unresolved reports on first migration run
foysalit Nov 29, 2023
c75131f
fix transaction error, process just unresolved reports, make reported…
devinivy Nov 30, 2023
9eebb90
tidy
devinivy Nov 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 110 additions & 22 deletions lexicons/com/atproto/admin/defs.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,8 @@
"action",
"subject",
"subjectBlobCids",
"reason",
"createdBy",
"createdAt",
"resolvedReportIds"
"createdAt"
],
"properties": {
"id": { "type": "integer" },
Expand All @@ -28,10 +26,10 @@
"subjectBlobCids": { "type": "array", "items": { "type": "string" } },
"createLabelVals": { "type": "array", "items": { "type": "string" } },
"negateLabelVals": { "type": "array", "items": { "type": "string" } },
"reason": { "type": "string" },
"comment": { "type": "string" },
"createdBy": { "type": "string", "format": "did" },
"createdAt": { "type": "string", "format": "datetime" },
"reversal": { "type": "ref", "ref": "#actionReversal" },
"meta": { "type": "ref", "ref": "#actionMeta" },
"resolvedReportIds": { "type": "array", "items": { "type": "integer" } }
}
},
Expand All @@ -42,7 +40,6 @@
"action",
"subject",
"subjectBlobs",
"reason",
"createdBy",
"createdAt",
"resolvedReports"
Expand All @@ -69,10 +66,9 @@
},
"createLabelVals": { "type": "array", "items": { "type": "string" } },
"negateLabelVals": { "type": "array", "items": { "type": "string" } },
"reason": { "type": "string" },
"comment": { "type": "string" },
"createdBy": { "type": "string", "format": "did" },
"createdAt": { "type": "string", "format": "datetime" },
"reversal": { "type": "ref", "ref": "#actionReversal" },
"resolvedReports": {
"type": "array",
"items": { "type": "ref", "ref": "#reportView" }
Expand All @@ -93,16 +89,35 @@
},
"actionReversal": {
"type": "object",
"required": ["reason", "createdBy", "createdAt"],
"required": ["createdBy", "createdAt"],
"properties": {
"reason": { "type": "string" },
"comment": { "type": "string" },
"createdBy": { "type": "string", "format": "did" },
"createdAt": { "type": "string", "format": "datetime" }
}
},
"actionMeta": {
"type": "object",
"properties": {
"resolveReportIds": { "type": "array", "items": { "type": "integer" } },
"reportType": {
"type": "ref",
"ref": "com.atproto.moderation.defs#reasonType"
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am interested to hear a little more about how to think about these two fields as a part of action metadata.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveReportIds will go away since we don't need to keep track of individual reports being resolved.
reportType will stay and store the metadata for #report events only where the user specifies the reportType on the client.

}
},
"actionType": {
"type": "string",
"knownValues": ["#takedown", "#flag", "#acknowledge", "#escalate"]
"knownValues": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some of the codegen I see a #report type—should it be listed here too?

"#takedown",
"#flag",
"#acknowledge",
"#escalate",
"#comment",
"#label",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an example, if a moderator adds a comment and a label at the same time then clicks "save", what event(s) are generated, and are those two events bundled together in any way?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Referencing our previous conversations on this: we decided that this not blocking at the moment and we are settling for 1 event at a time model for now. Soon after release, if we see this being a common case, we could either end up adjusting the client to dispatch multiple requests through unified UI/UX or allow this endpoint to receive multiple events as array and process all in one go.

"#revert",
"#mute"
]
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the action type imply what content is allowed in the action? E.g. can you include labels in a #mute event even though there's also a #label event? If so, we might consider turning actions into an open union.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to keep this flexible for now and not apply too much validation based on action types just yet. Some of these props are kinda composable. for instance, I could #acknowledge for a certain period of time and the action will be reverted after that period or #mute AND apply a label. some are not so composable for instance a #revert with a expiry probably doesn't make too much sense.
so overall, there should be some specific combinations that are not allowed but I haven't thought too much into it yet.

"takedown": {
"type": "token",
Expand All @@ -120,6 +135,26 @@
"type": "token",
"description": "Moderation action type: Escalate. Indicates that the content has been flagged for additional review."
},
"comment": {
"type": "token",
"description": "Moderation action type: Comment. Indicates that no change is being made to the subject or associated reports, just a comment is being added by a human or automated moderator"
},
"label": {
"type": "token",
"description": "Moderation action type: Label. Indicates that labels associated with the subject are being changed."
},
"revert": {
"type": "token",
"description": "Moderation action type: Revert. Indicates that a previously taken action is being reversed."
},
"mute": {
"type": "token",
"description": "Moderation action type: Mute. Indicates that reports/other events on a subject can be muted for a period of time."
},
"report": {
"type": "token",
"description": "Moderation action type: Report. Indicates that a new report was received for the subject."
},
"reportView": {
"type": "object",
"required": [
Expand All @@ -136,7 +171,7 @@
"type": "ref",
"ref": "com.atproto.moderation.defs#reasonType"
},
"reason": { "type": "string" },
"comment": { "type": "string" },
"subjectRepoHandle": { "type": "string" },
"subject": {
"type": "union",
Expand All @@ -150,6 +185,45 @@
}
}
},
"subjectStatusView": {
"type": "object",
"required": ["id", "subject", "createdAt", "updatedAt", "reviewState"],
"properties": {
"id": { "type": "integer" },
"subject": {
"type": "union",
"refs": ["#repoRef", "com.atproto.repo.strongRef"]
},
"updatedAt": { "type": "string", "format": "datetime" },
"createdAt": { "type": "string", "format": "datetime" },
bnewbold marked this conversation as resolved.
Show resolved Hide resolved
"reviewState": {
"type": "ref",
"ref": "#subjectReviewState"
},
"note": {
foysalit marked this conversation as resolved.
Show resolved Hide resolved
"type": "string"
},
"muteUntil": {
"type": "string",
"format": "datetime"
},
"lastReviewedAt": {
"type": "string",
"format": "datetime"
},
"lastReportedAt": {
"type": "string",
"format": "datetime"
},
"takendown": {
"type": "boolean"
},
Comment on lines +170 to +172
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth taking a peek at #statusAttr and getSubjectStatus, updateSubjectStatus lexicons— we may be able to make these more consistent with each other.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if these can be same though? statusAttr is defined as

{
    "applied": { "type": "boolean" },
    "ref": { "type": "string" }
}

but in the appview, takendown: true always means applied: true and ref would just be a reference to the entry's id itself so it's kinda redundant unless we map this ref to reference to the event that changed the status?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They do also have different names in getSubjectStatus / updateSubjectStatus: takedown vs. takendown.

"suspendUntil": {
"type": "string",
"format": "datetime"
}
}
},
"reportViewDetail": {
"type": "object",
"required": [
Expand All @@ -166,7 +240,7 @@
"type": "ref",
"ref": "com.atproto.moderation.defs#reasonType"
},
"reason": { "type": "string" },
"comment": { "type": "string" },
"subject": {
"type": "union",
"refs": [
Expand All @@ -176,6 +250,10 @@
"#recordViewNotFound"
]
},
"subjectStatus": {
"type": "ref",
"ref": "com.atproto.admin.defs#subjectStatusView"
},
"reportedBy": { "type": "string", "format": "did" },
"createdAt": { "type": "string", "format": "datetime" },
"resolvedByActions": {
Expand Down Expand Up @@ -324,16 +402,10 @@
},
"moderationDetail": {
"type": "object",
"required": ["actions", "reports"],
"properties": {
"currentAction": { "type": "ref", "ref": "#actionViewCurrent" },
"actions": {
"type": "array",
"items": { "type": "ref", "ref": "#actionView" }
},
"reports": {
"type": "array",
"items": { "type": "ref", "ref": "#reportView" }
"subjectStatus": {
"type": "ref",
"ref": "#subjectStatusView"
}
}
},
Expand Down Expand Up @@ -368,6 +440,22 @@
"height": { "type": "integer" },
"length": { "type": "integer" }
}
},
"subjectReviewState": {
"type": "string",
"knownValues": ["#reviewOpen", "#reviewEscalated", "#reviewClosed"]
},
"reviewOpen": {
"type": "token",
"description": "Moderator review status of a subject: Open. Indicates that the subject needs to be reviewed by a moderator"
},
"reviewEscalated": {
"type": "token",
"description": "Moderator review status of a subject: Escalated. Indicates that the subject was escalated for review by a moderator"
},
"reviewClosed": {
"type": "token",
"description": "Moderator review status of a subject: Closed. Indicates that the subject was already reviewed and resolved by a moderator"
}
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"lexicon": 1,
"id": "com.atproto.admin.getModerationActions",
"id": "com.atproto.admin.getModerationEvents",
bnewbold marked this conversation as resolved.
Show resolved Hide resolved
"defs": {
"main": {
"type": "query",
Expand Down
72 changes: 72 additions & 0 deletions lexicons/com/atproto/admin/getModerationStatuses.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{
"lexicon": 1,
"id": "com.atproto.admin.getModerationStatuses",
bnewbold marked this conversation as resolved.
Show resolved Hide resolved
"defs": {
"main": {
"type": "query",
"description": "View moderation statuses of subjects (record or repo).",
"parameters": {
"type": "params",
"properties": {
"subject": { "type": "string" },
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to other subject fields, maybe format=uri?

"note": {
"type": "string",
"description": "Search subjects by keyword from notes"
},
"reportedAfter": {
"type": "string",
"format": "datetime",
"description": "Search subjects reported after a given timestamp"
},
"reportedBefore": {
"type": "string",
"format": "datetime",
"description": "Search subjects reported before a given timestamp"
},
"reviewedAfter": {
"type": "string",
"format": "datetime",
"description": "Search subjects reviewed after a given timestamp"
},
"reviewedBefore": {
"type": "string",
"format": "datetime",
"description": "Search subjects reviewed before a given timestamp"
},
"includeMuted": {
"type": "boolean",
"description": "By default, we don't include muted subjects in the results. Set this to true to include them."
},
"reviewState": {
"type": "string",
"description": "Specify when fetching subjects in a certain state"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 50
},
"cursor": { "type": "string" }
}
},
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["subjectStatuses"],
"properties": {
"cursor": { "type": "string" },
"subjectStatuses": {
"type": "array",
"items": {
"type": "ref",
"ref": "com.atproto.admin.defs#subjectStatusView"
}
}
}
}
}
}
}
}
29 changes: 0 additions & 29 deletions lexicons/com/atproto/admin/resolveModerationReports.json

This file was deleted.

29 changes: 0 additions & 29 deletions lexicons/com/atproto/admin/reverseModerationAction.json

This file was deleted.

Loading
Loading