-
Notifications
You must be signed in to change notification settings - Fork 578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Evented architecture for moderation system #1617
Changes from 17 commits
5ea9064
5445a6b
2b5801c
b62a397
8781c87
39a81fd
1637548
e1e09b3
619eed5
c8c1a55
dfc4fec
75625a7
a82d3d6
f734703
2cc6f1d
f8b3b78
84d79a8
6b652ef
cc87dcb
10ee0e5
dacbb3f
5a9bdd9
ddce910
164f391
82870ec
87364a8
79f4c97
a07f64f
d71b910
aea0b55
afb9a8b
cc756ad
f5204b0
23cea8e
85e8322
3b0e41c
3b3a1a9
1ca8427
c442738
9973b8e
62ea6df
efc4eaa
eb766bd
bda203d
aa9ac28
226b87e
8c1f026
1ca527b
be425b9
677e2e8
902e6b4
96395e6
49f381f
39cf166
1bf5acc
143e569
d220257
6e76210
dc57dc8
006e0dc
2792f9b
d44b683
6c034bd
741aef4
f8a9bbe
554af17
56f3a61
90c8241
e4bd6d6
9f11c6c
2617b4d
c25168f
bcb475e
79b0473
954b523
e6e8416
a608eb6
82fb94f
31d1bb2
a286b84
a93a162
22bd5f0
c3d0093
0d1eb3f
404a42f
2d1f542
91d32d4
006372f
a2e6ea6
d21b3d9
0697ffc
99aef0a
655fe09
d69bb10
48253a8
a8b95bf
c75131f
9eebb90
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,10 +9,8 @@ | |
"action", | ||
"subject", | ||
"subjectBlobCids", | ||
"reason", | ||
"createdBy", | ||
"createdAt", | ||
"resolvedReportIds" | ||
"createdAt" | ||
], | ||
"properties": { | ||
"id": { "type": "integer" }, | ||
|
@@ -28,10 +26,10 @@ | |
"subjectBlobCids": { "type": "array", "items": { "type": "string" } }, | ||
"createLabelVals": { "type": "array", "items": { "type": "string" } }, | ||
"negateLabelVals": { "type": "array", "items": { "type": "string" } }, | ||
"reason": { "type": "string" }, | ||
"comment": { "type": "string" }, | ||
"createdBy": { "type": "string", "format": "did" }, | ||
"createdAt": { "type": "string", "format": "datetime" }, | ||
"reversal": { "type": "ref", "ref": "#actionReversal" }, | ||
"meta": { "type": "ref", "ref": "#actionMeta" }, | ||
"resolvedReportIds": { "type": "array", "items": { "type": "integer" } } | ||
} | ||
}, | ||
|
@@ -42,7 +40,6 @@ | |
"action", | ||
"subject", | ||
"subjectBlobs", | ||
"reason", | ||
"createdBy", | ||
"createdAt", | ||
"resolvedReports" | ||
|
@@ -69,10 +66,9 @@ | |
}, | ||
"createLabelVals": { "type": "array", "items": { "type": "string" } }, | ||
"negateLabelVals": { "type": "array", "items": { "type": "string" } }, | ||
"reason": { "type": "string" }, | ||
"comment": { "type": "string" }, | ||
"createdBy": { "type": "string", "format": "did" }, | ||
"createdAt": { "type": "string", "format": "datetime" }, | ||
"reversal": { "type": "ref", "ref": "#actionReversal" }, | ||
"resolvedReports": { | ||
"type": "array", | ||
"items": { "type": "ref", "ref": "#reportView" } | ||
|
@@ -93,16 +89,35 @@ | |
}, | ||
"actionReversal": { | ||
"type": "object", | ||
"required": ["reason", "createdBy", "createdAt"], | ||
"required": ["createdBy", "createdAt"], | ||
"properties": { | ||
"reason": { "type": "string" }, | ||
"comment": { "type": "string" }, | ||
"createdBy": { "type": "string", "format": "did" }, | ||
"createdAt": { "type": "string", "format": "datetime" } | ||
} | ||
}, | ||
"actionMeta": { | ||
"type": "object", | ||
"properties": { | ||
"resolveReportIds": { "type": "array", "items": { "type": "integer" } }, | ||
"reportType": { | ||
"type": "ref", | ||
"ref": "com.atproto.moderation.defs#reasonType" | ||
} | ||
} | ||
}, | ||
"actionType": { | ||
"type": "string", | ||
"knownValues": ["#takedown", "#flag", "#acknowledge", "#escalate"] | ||
"knownValues": [ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In some of the codegen I see a |
||
"#takedown", | ||
"#flag", | ||
"#acknowledge", | ||
"#escalate", | ||
"#comment", | ||
"#label", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As an example, if a moderator adds a comment and a label at the same time then clicks "save", what event(s) are generated, and are those two events bundled together in any way? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Referencing our previous conversations on this: we decided that this not blocking at the moment and we are settling for 1 event at a time model for now. Soon after release, if we see this being a common case, we could either end up adjusting the client to dispatch multiple requests through unified UI/UX or allow this endpoint to receive multiple events as array and process all in one go. |
||
"#revert", | ||
"#mute" | ||
] | ||
}, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the action type imply what content is allowed in the action? E.g. can you include labels in a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I want to keep this flexible for now and not apply too much validation based on action types just yet. Some of these props are kinda composable. for instance, I could #acknowledge for a certain period of time and the action will be reverted after that period or #mute AND apply a label. some are not so composable for instance a #revert with a expiry probably doesn't make too much sense. |
||
"takedown": { | ||
"type": "token", | ||
|
@@ -120,6 +135,26 @@ | |
"type": "token", | ||
"description": "Moderation action type: Escalate. Indicates that the content has been flagged for additional review." | ||
}, | ||
"comment": { | ||
"type": "token", | ||
"description": "Moderation action type: Comment. Indicates that no change is being made to the subject or associated reports, just a comment is being added by a human or automated moderator" | ||
}, | ||
"label": { | ||
"type": "token", | ||
"description": "Moderation action type: Label. Indicates that labels associated with the subject are being changed." | ||
}, | ||
"revert": { | ||
"type": "token", | ||
"description": "Moderation action type: Revert. Indicates that a previously taken action is being reversed." | ||
}, | ||
"mute": { | ||
"type": "token", | ||
"description": "Moderation action type: Mute. Indicates that reports/other events on a subject can be muted for a period of time." | ||
}, | ||
"report": { | ||
"type": "token", | ||
"description": "Moderation action type: Report. Indicates that a new report was received for the subject." | ||
}, | ||
"reportView": { | ||
"type": "object", | ||
"required": [ | ||
|
@@ -136,7 +171,7 @@ | |
"type": "ref", | ||
"ref": "com.atproto.moderation.defs#reasonType" | ||
}, | ||
"reason": { "type": "string" }, | ||
"comment": { "type": "string" }, | ||
"subjectRepoHandle": { "type": "string" }, | ||
"subject": { | ||
"type": "union", | ||
|
@@ -150,6 +185,45 @@ | |
} | ||
} | ||
}, | ||
"subjectStatusView": { | ||
"type": "object", | ||
"required": ["id", "subject", "createdAt", "updatedAt", "reviewState"], | ||
"properties": { | ||
"id": { "type": "integer" }, | ||
"subject": { | ||
"type": "union", | ||
"refs": ["#repoRef", "com.atproto.repo.strongRef"] | ||
}, | ||
"updatedAt": { "type": "string", "format": "datetime" }, | ||
"createdAt": { "type": "string", "format": "datetime" }, | ||
bnewbold marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"reviewState": { | ||
"type": "ref", | ||
"ref": "#subjectReviewState" | ||
}, | ||
"note": { | ||
foysalit marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"type": "string" | ||
}, | ||
"muteUntil": { | ||
"type": "string", | ||
"format": "datetime" | ||
}, | ||
"lastReviewedAt": { | ||
"type": "string", | ||
"format": "datetime" | ||
}, | ||
"lastReportedAt": { | ||
"type": "string", | ||
"format": "datetime" | ||
}, | ||
"takendown": { | ||
"type": "boolean" | ||
}, | ||
Comment on lines
+170
to
+172
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might be worth taking a peek at There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not sure if these can be same though?
but in the appview, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They do also have different names in |
||
"suspendUntil": { | ||
"type": "string", | ||
"format": "datetime" | ||
} | ||
} | ||
}, | ||
"reportViewDetail": { | ||
"type": "object", | ||
"required": [ | ||
|
@@ -166,7 +240,7 @@ | |
"type": "ref", | ||
"ref": "com.atproto.moderation.defs#reasonType" | ||
}, | ||
"reason": { "type": "string" }, | ||
"comment": { "type": "string" }, | ||
"subject": { | ||
"type": "union", | ||
"refs": [ | ||
|
@@ -176,6 +250,10 @@ | |
"#recordViewNotFound" | ||
] | ||
}, | ||
"subjectStatus": { | ||
"type": "ref", | ||
"ref": "com.atproto.admin.defs#subjectStatusView" | ||
}, | ||
"reportedBy": { "type": "string", "format": "did" }, | ||
"createdAt": { "type": "string", "format": "datetime" }, | ||
"resolvedByActions": { | ||
|
@@ -324,16 +402,10 @@ | |
}, | ||
"moderationDetail": { | ||
"type": "object", | ||
"required": ["actions", "reports"], | ||
"properties": { | ||
"currentAction": { "type": "ref", "ref": "#actionViewCurrent" }, | ||
"actions": { | ||
"type": "array", | ||
"items": { "type": "ref", "ref": "#actionView" } | ||
}, | ||
"reports": { | ||
"type": "array", | ||
"items": { "type": "ref", "ref": "#reportView" } | ||
"subjectStatus": { | ||
"type": "ref", | ||
"ref": "#subjectStatusView" | ||
} | ||
} | ||
}, | ||
|
@@ -368,6 +440,22 @@ | |
"height": { "type": "integer" }, | ||
"length": { "type": "integer" } | ||
} | ||
}, | ||
"subjectReviewState": { | ||
"type": "string", | ||
"knownValues": ["#reviewOpen", "#reviewEscalated", "#reviewClosed"] | ||
}, | ||
"reviewOpen": { | ||
"type": "token", | ||
"description": "Moderator review status of a subject: Open. Indicates that the subject needs to be reviewed by a moderator" | ||
}, | ||
"reviewEscalated": { | ||
"type": "token", | ||
"description": "Moderator review status of a subject: Escalated. Indicates that the subject was escalated for review by a moderator" | ||
}, | ||
"reviewClosed": { | ||
"type": "token", | ||
"description": "Moderator review status of a subject: Closed. Indicates that the subject was already reviewed and resolved by a moderator" | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
{ | ||
"lexicon": 1, | ||
"id": "com.atproto.admin.getModerationStatuses", | ||
bnewbold marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"defs": { | ||
"main": { | ||
"type": "query", | ||
"description": "View moderation statuses of subjects (record or repo).", | ||
"parameters": { | ||
"type": "params", | ||
"properties": { | ||
"subject": { "type": "string" }, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. similar to other subject fields, maybe format=uri? |
||
"note": { | ||
"type": "string", | ||
"description": "Search subjects by keyword from notes" | ||
}, | ||
"reportedAfter": { | ||
"type": "string", | ||
"format": "datetime", | ||
"description": "Search subjects reported after a given timestamp" | ||
}, | ||
"reportedBefore": { | ||
"type": "string", | ||
"format": "datetime", | ||
"description": "Search subjects reported before a given timestamp" | ||
}, | ||
"reviewedAfter": { | ||
"type": "string", | ||
"format": "datetime", | ||
"description": "Search subjects reviewed after a given timestamp" | ||
}, | ||
"reviewedBefore": { | ||
"type": "string", | ||
"format": "datetime", | ||
"description": "Search subjects reviewed before a given timestamp" | ||
}, | ||
"includeMuted": { | ||
"type": "boolean", | ||
"description": "By default, we don't include muted subjects in the results. Set this to true to include them." | ||
}, | ||
"reviewState": { | ||
"type": "string", | ||
"description": "Specify when fetching subjects in a certain state" | ||
}, | ||
"limit": { | ||
"type": "integer", | ||
"minimum": 1, | ||
"maximum": 100, | ||
"default": 50 | ||
}, | ||
"cursor": { "type": "string" } | ||
} | ||
}, | ||
"output": { | ||
"encoding": "application/json", | ||
"schema": { | ||
"type": "object", | ||
"required": ["subjectStatuses"], | ||
"properties": { | ||
"cursor": { "type": "string" }, | ||
"subjectStatuses": { | ||
"type": "array", | ||
"items": { | ||
"type": "ref", | ||
"ref": "com.atproto.admin.defs#subjectStatusView" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
} | ||
} | ||
} |
This file was deleted.
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am interested to hear a little more about how to think about these two fields as a part of action metadata.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
resolveReportIds
will go away since we don't need to keep track of individual reports being resolved.reportType
will stay and store the metadata for#report
events only where the user specifies thereportType
on the client.