Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Label expiration #2241

Merged
merged 32 commits into from
Mar 9, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
3084a33
sketching out label sequencer
dholms Jan 12, 2024
c6072fc
refactor sequencer
dholms Jan 12, 2024
72b4511
sequencer tests
dholms Jan 12, 2024
be19aa3
tests
dholms Jan 12, 2024
1c6f494
add query labels endpoint & tests
dholms Jan 13, 2024
a51aaf7
add pagination
dholms Jan 13, 2024
1570fa4
fix label formatting on temp
dholms Jan 13, 2024
8d8e8a9
tidy
dholms Jan 15, 2024
c2c4caa
merge main
dholms Feb 27, 2024
5918ae6
format labels
dholms Feb 27, 2024
3efc6d4
make use listen/notify for sequencer
dholms Feb 27, 2024
c47109e
ensure sig on all outgoing labels from ozone
dholms Feb 28, 2024
faaaba2
fixing up tests
dholms Feb 28, 2024
7c84fc6
fix sequencer tests
dholms Feb 28, 2024
f3400c7
fix hanging server test
dholms Feb 28, 2024
884c690
Merge branch 'label-stream' into signed-labels
dholms Feb 28, 2024
51ee15b
merge main
dholms Feb 29, 2024
78d73d8
add log on failure to update label
dholms Feb 29, 2024
c79aa7b
update description for sig
dholms Feb 29, 2024
221c3f7
add expiration to labels
dholms Feb 29, 2024
773287b
fix test
dholms Feb 29, 2024
14d9ca0
merge main
dholms Mar 7, 2024
e8ab4ac
use bytes for label sigs
dholms Mar 7, 2024
9623e50
fix tests
dholms Mar 7, 2024
06c79c0
add ver to labels
dholms Mar 8, 2024
ced9392
merge
dholms Mar 8, 2024
2a4383b
tidy up background queue
dholms Mar 9, 2024
6cd885f
store signing keys as ids
dholms Mar 9, 2024
80868b7
merge
dholms Mar 9, 2024
9105e56
fix sequencer teest
dholms Mar 9, 2024
fd19f34
Merge branch 'signed-labels' into label-exp
dholms Mar 9, 2024
7879a99
merge
dholms Mar 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions lexicons/com/atproto/label/defs.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
"description": "Metadata tag on an atproto resource (eg, repo or record).",
"required": ["src", "uri", "val", "cts"],
"properties": {
"ver": {
"type": "integer",
"description": "The AT Protocol version of the label object."
},
"src": {
"type": "string",
"format": "did",
Expand Down Expand Up @@ -35,6 +39,15 @@
"type": "string",
"format": "datetime",
"description": "Timestamp when this label was created."
},
"exp": {
"type": "string",
"format": "datetime",
"description": "Timestamp at which this label expires (no longer applies)."
},
"sig": {
"type": "bytes",
"description": "Signature of dag-cbor encoded label."
}
}
},
Expand Down
14 changes: 14 additions & 0 deletions packages/api/src/client/lexicons.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2202,6 +2202,10 @@ export const schemaDict = {
'Metadata tag on an atproto resource (eg, repo or record).',
required: ['src', 'uri', 'val', 'cts'],
properties: {
ver: {
type: 'integer',
description: 'The AT Protocol version of the label object.',
},
src: {
type: 'string',
format: 'did',
Expand Down Expand Up @@ -2235,6 +2239,16 @@ export const schemaDict = {
format: 'datetime',
description: 'Timestamp when this label was created.',
},
exp: {
type: 'string',
format: 'datetime',
description:
'Timestamp at which this label expires (no longer applies).',
},
sig: {
type: 'bytes',
description: 'Signature of dag-cbor encoded label.',
},
},
},
selfLabels: {
Expand Down
6 changes: 6 additions & 0 deletions packages/api/src/client/types/com/atproto/label/defs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import { CID } from 'multiformats/cid'

/** Metadata tag on an atproto resource (eg, repo or record). */
export interface Label {
/** The AT Protocol version of the label object. */
ver?: number
/** DID of the actor who created this label. */
src: string
/** AT URI of the record, repository (account), or other resource that this label applies to. */
Expand All @@ -20,6 +22,10 @@ export interface Label {
neg?: boolean
/** Timestamp when this label was created. */
cts: string
/** Timestamp at which this label expires (no longer applies). */
exp?: string
/** Signature of dag-cbor encoded label. */
sig?: Uint8Array
[k: string]: unknown
}

Expand Down
14 changes: 14 additions & 0 deletions packages/bsky/src/lexicon/lexicons.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2202,6 +2202,10 @@ export const schemaDict = {
'Metadata tag on an atproto resource (eg, repo or record).',
required: ['src', 'uri', 'val', 'cts'],
properties: {
ver: {
type: 'integer',
description: 'The AT Protocol version of the label object.',
},
src: {
type: 'string',
format: 'did',
Expand Down Expand Up @@ -2235,6 +2239,16 @@ export const schemaDict = {
format: 'datetime',
description: 'Timestamp when this label was created.',
},
exp: {
type: 'string',
format: 'datetime',
description:
'Timestamp at which this label expires (no longer applies).',
},
sig: {
type: 'bytes',
description: 'Signature of dag-cbor encoded label.',
},
},
},
selfLabels: {
Expand Down
6 changes: 6 additions & 0 deletions packages/bsky/src/lexicon/types/com/atproto/label/defs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import { CID } from 'multiformats/cid'

/** Metadata tag on an atproto resource (eg, repo or record). */
export interface Label {
/** The AT Protocol version of the label object. */
ver?: number
/** DID of the actor who created this label. */
src: string
/** AT URI of the record, repository (account), or other resource that this label applies to. */
Expand All @@ -20,6 +22,10 @@ export interface Label {
neg?: boolean
/** Timestamp when this label was created. */
cts: string
/** Timestamp at which this label expires (no longer applies). */
exp?: string
/** Signature of dag-cbor encoded label. */
sig?: Uint8Array
[k: string]: unknown
}

Expand Down
6 changes: 4 additions & 2 deletions packages/ozone/src/api/label/queryLabels.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import { Server } from '../../lexicon'
import AppContext from '../../context'
import { InvalidRequestError } from '@atproto/xrpc-server'
import { sql } from 'kysely'
import { formatLabel } from '../../mod-service/util'

export default function (server: Server, ctx: AppContext) {
server.com.atproto.label.queryLabels(async ({ params }) => {
Expand Down Expand Up @@ -44,7 +43,10 @@ export default function (server: Server, ctx: AppContext) {

const res = await builder.execute()

const labels = res.map((l) => formatLabel(l))
const modSrvc = ctx.modService(ctx.db)
const labels = await Promise.all(
res.map((l) => modSrvc.views.formatLabelAndEnsureSig(l)),
)
const resCursor = res.at(-1)?.id.toString(10)

return {
Expand Down
6 changes: 4 additions & 2 deletions packages/ozone/src/api/temp/fetchLabels.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import { Server } from '../../lexicon'
import AppContext from '../../context'
import { formatLabel } from '../../mod-service/util'
import {
UNSPECCED_TAKEDOWN_BLOBS_LABEL,
UNSPECCED_TAKEDOWN_LABEL,
Expand Down Expand Up @@ -29,7 +28,10 @@ export default function (server: Server, ctx: AppContext) {
.limit(limit)
.execute()

const labels = labelRes.map((l) => formatLabel(l))
const modSrvc = ctx.modService(ctx.db)
const labels = await Promise.all(
labelRes.map((l) => modSrvc.views.formatLabelAndEnsureSig(l)),
)

return {
encoding: 'application/json',
Expand Down
12 changes: 9 additions & 3 deletions packages/ozone/src/context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,20 +71,19 @@ export class AppContext {
})

const modService = ModerationService.creator(
signingKey,
cfg,
backgroundQueue,
idResolver,
eventPusher,
appviewAgent,
createAuthHeaders,
cfg.service.did,
overrides?.imgInvalidator,
cfg.cdn.paths,
)

const communicationTemplateService = CommunicationTemplateService.creator()

const sequencer = new Sequencer(db)
const sequencer = new Sequencer(modService(db))

const authVerifier = new AuthVerifier(idResolver, {
serviceDid: cfg.service.did,
Expand Down Expand Up @@ -188,6 +187,13 @@ export class AppContext {
async appviewAuth() {
return this.serviceAuthHeaders(this.cfg.appview.did)
}

devOverride(overrides: Partial<AppContextOptions>) {
this.opts = {
...this.opts,
...overrides,
}
}
}

export default AppContext
2 changes: 1 addition & 1 deletion packages/ozone/src/daemon/context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,13 @@ export class DaemonContext {
})

const modService = ModerationService.creator(
signingKey,
cfg,
backgroundQueue,
idResolver,
eventPusher,
appviewAgent,
createAuthHeaders,
cfg.service.did,
)

const eventReverser = new EventReverser(db, modService)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import { Kysely, sql } from 'kysely'

export async function up(db: Kysely<unknown>): Promise<void> {
await db.schema.alterTable('label').addColumn('exp', 'varchar').execute()
await db.schema
.alterTable('label')
.addColumn('sig', sql`bytea`)
.execute()
await db.schema
.alterTable('label')
.addColumn('signingKey', 'varchar')
.execute()
}

export async function down(db: Kysely<unknown>): Promise<void> {
await db.schema.alterTable('label').dropColumn('exp').execute()
await db.schema.alterTable('label').dropColumn('sig').execute()
await db.schema.alterTable('label').dropColumn('signingKey').execute()
}
1 change: 1 addition & 0 deletions packages/ozone/src/db/migrations/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ export * as _20231219T205730722Z from './20231219T205730722Z-init'
export * as _20240116T085607200Z from './20240116T085607200Z-communication-template'
export * as _20240201T051104136Z from './20240201T051104136Z-mod-event-blobs'
export * as _20240208T213404429Z from './20240208T213404429Z-add-tags-column-to-moderation-subject'
export * as _20240228T003647759Z from './20240228T003647759Z-add-label-sigs'
3 changes: 3 additions & 0 deletions packages/ozone/src/db/schema/label.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ export interface Label {
val: string
neg: boolean
cts: string
exp: string | null
sig: Buffer | null
signingKey: string | null
}

export type LabelRow = Selectable<Label>
Expand Down
14 changes: 14 additions & 0 deletions packages/ozone/src/lexicon/lexicons.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2202,6 +2202,10 @@ export const schemaDict = {
'Metadata tag on an atproto resource (eg, repo or record).',
required: ['src', 'uri', 'val', 'cts'],
properties: {
ver: {
type: 'integer',
description: 'The AT Protocol version of the label object.',
},
src: {
type: 'string',
format: 'did',
Expand Down Expand Up @@ -2235,6 +2239,16 @@ export const schemaDict = {
format: 'datetime',
description: 'Timestamp when this label was created.',
},
exp: {
type: 'string',
format: 'datetime',
description:
'Timestamp at which this label expires (no longer applies).',
},
sig: {
type: 'bytes',
description: 'Signature of dag-cbor encoded label.',
},
},
},
selfLabels: {
Expand Down
6 changes: 6 additions & 0 deletions packages/ozone/src/lexicon/types/com/atproto/label/defs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import { CID } from 'multiformats/cid'

/** Metadata tag on an atproto resource (eg, repo or record). */
export interface Label {
/** The AT Protocol version of the label object. */
ver?: number
/** DID of the actor who created this label. */
src: string
/** AT URI of the record, repository (account), or other resource that this label applies to. */
Expand All @@ -20,6 +22,10 @@ export interface Label {
neg?: boolean
/** Timestamp when this label was created. */
cts: string
/** Timestamp at which this label expires (no longer applies). */
exp?: string
/** Signature of dag-cbor encoded label. */
sig?: Uint8Array
[k: string]: unknown
}

Expand Down
Loading
Loading