Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up role-based auth #2255

Merged
merged 82 commits into from
Mar 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
6a26396
tidy bsky auth
dholms Feb 29, 2024
125f721
hook up new auth verifier
dholms Feb 29, 2024
36da1d9
update auth throughout ozone
dholms Feb 29, 2024
32b3de8
handle mod signing keys
dholms Feb 29, 2024
b1f07d5
add client proxy heads to pds
dholms Feb 29, 2024
ef236a4
hook up rest of routes
dholms Feb 29, 2024
bfbb586
simplify pipethrough & add some SSRF protection
dholms Feb 29, 2024
92d9268
tests
dholms Feb 29, 2024
199b754
fix bad var
dholms Feb 29, 2024
d1d39ff
merge main
dholms Feb 29, 2024
5cea30c
remove basic auth in ozone
dholms Feb 29, 2024
7561b93
wip
dholms Feb 29, 2024
cb53fdc
fix key parsing in pds
dholms Feb 29, 2024
f8145ec
Merge branch 'ozone-acls-take2' into rm-basic-auth
dholms Feb 29, 2024
26b3557
fix up all ozone tests
dholms Mar 1, 2024
e3bfb17
fix admin auth test
dholms Mar 1, 2024
a642063
rename test
dholms Mar 1, 2024
e2c0949
Merge branch 'ozone-acls-take2' into rm-basic-auth
dholms Mar 1, 2024
bac2b57
fix ozone test
dholms Mar 1, 2024
d599dd7
clean up tokens in pds
dholms Mar 1, 2024
08dc9a9
fix up pds tests
dholms Mar 1, 2024
8747869
fix up ozone tests
dholms Mar 1, 2024
88c2412
add pipethrough to write routes
dholms Mar 1, 2024
a30ac47
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 1, 2024
648cf62
merge
dholms Mar 1, 2024
a52f7b6
reenable proxied admin test
dholms Mar 1, 2024
9b322c7
add moderator accounts to ozone in dev-env
dholms Mar 4, 2024
f7ef546
update did doc id values
dholms Mar 4, 2024
dbe9aff
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 4, 2024
6eb72bf
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 4, 2024
0482a92
null creds string -> `none`
dholms Mar 4, 2024
cccade6
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 4, 2024
8a38742
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 4, 2024
5df31de
fix fetchLabels auth check
dholms Mar 5, 2024
04dc443
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
9e290ca
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 5, 2024
dd891d4
:sparkles: Add a couple more proxied requests that we use in ozone ui
foysalit Mar 5, 2024
2ca4fee
Add runit to the services/bsky Dockerfile (#2254)
Jacob2161 Feb 29, 2024
6ba5f6c
Improve tag detection (#2260)
estrattonbailey Mar 1, 2024
9b2500e
Version packages (#2261)
github-actions[bot] Mar 1, 2024
c76fd03
:bug: Increment attempt count after each attempt to push ozone event …
foysalit Mar 4, 2024
87f00f2
Ozone delegates email sending to actor's pds (#2272)
devinivy Mar 5, 2024
ccfc4d9
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
1b1d1a4
merge
dholms Mar 5, 2024
c273f46
add dev dep for nodemailer in ozone
dholms Mar 5, 2024
8341c7a
fix auth verifier method
dholms Mar 5, 2024
207e208
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
9ddf283
merge
dholms Mar 5, 2024
f936105
build branch
dholms Mar 5, 2024
d7682f9
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 5, 2024
971b4b7
build branch
dholms Mar 5, 2024
11b7af2
merge main
dholms Mar 5, 2024
abe4b03
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
3a9661f
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 5, 2024
5f3c91b
fix url check
dholms Mar 5, 2024
037f163
better error handling for get account infos
dholms Mar 5, 2024
fc1c40d
fix labeler service id
dholms Mar 5, 2024
483b71f
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
5e1c5fd
fix iss on auth headers
dholms Mar 5, 2024
64d99dd
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
82acea2
fix dev-env ozone did
dholms Mar 5, 2024
4c7db5c
fix tests & another jwt issuer
dholms Mar 5, 2024
514b437
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
d697105
merge
dholms Mar 5, 2024
9d5d762
fix proxy auth
dholms Mar 5, 2024
81f9d69
ozone: fix ip check
devinivy Mar 5, 2024
c74fd23
Merge remote-tracking branch 'origin/ozone-acls-take2' into pds-proxy…
devinivy Mar 5, 2024
7be8445
fix aud check on pds mod service auth
dholms Mar 5, 2024
7a490d2
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
592518c
tidy
dholms Mar 5, 2024
402cc7b
Merge branch 'ozone-acls-take2' into pds-proxy-headers
dholms Mar 5, 2024
5b6b9ee
Merge branch 'main' into pds-proxy-headers
dholms Mar 6, 2024
f583ba9
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 6, 2024
1ce9e00
Update packages/pds/tests/proxied/admin.test.ts
dholms Mar 6, 2024
5aec85f
merge main
dholms Mar 7, 2024
c5c7a4c
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 7, 2024
b4211ea
fix pipethrough of headers
dholms Mar 7, 2024
0ab7500
Merge branch 'pds-proxy-headers' into rm-basic-auth
dholms Mar 7, 2024
e46397d
fix moderation status tests
dholms Mar 7, 2024
d9b62b9
fix auth on ozone routes
dholms Mar 7, 2024
4d2c17f
update iss on daemon
dholms Mar 7, 2024
accb2a6
merge main
dholms Mar 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/build-and-push-bsky-ghcr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ on:
push:
branches:
- main
- pds-proxy-headers
env:
REGISTRY: ghcr.io
USERNAME: ${{ github.actor }}
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/build-and-push-ozone-aws.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ on:
push:
branches:
- main
- pds-proxy-headers
env:
REGISTRY: ${{ secrets.AWS_ECR_REGISTRY_USEAST2_PACKAGES_REGISTRY }}
USERNAME: ${{ secrets.AWS_ECR_REGISTRY_USEAST2_PACKAGES_USERNAME }}
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/build-and-push-pds-ghcr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ on:
push:
branches:
- main
- pds-proxy-headers
env:
REGISTRY: ghcr.io
USERNAME: ${{ github.actor }}
Expand Down
4 changes: 2 additions & 2 deletions packages/dev-env/src/bsky.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { AtpAgent } from '@atproto/api'
import { Secp256k1Keypair } from '@atproto/crypto'
import { Client as PlcClient } from '@did-plc/lib'
import { BskyConfig } from './types'
import { ADMIN_PASSWORD, MOD_PASSWORD, TRIAGE_PASSWORD } from './const'
import { ADMIN_PASSWORD } from './const'
import { BackgroundQueue } from '@atproto/bsky/src/data-plane/server/background'

export class TestBsky {
Expand Down Expand Up @@ -64,7 +64,7 @@ export class TestBsky {
modServiceDid: cfg.modServiceDid ?? 'did:example:invalidMod',
labelsFromIssuerDids: ['did:example:labeler'], // this did is also used as the labeler in seeds
...cfg,
adminPasswords: [ADMIN_PASSWORD, MOD_PASSWORD, TRIAGE_PASSWORD],
adminPasswords: [ADMIN_PASSWORD],
})

// Separate migration db in case migration changes some connection state that we need in the tests, e.g. "alter database ... set ..."
Expand Down
2 changes: 0 additions & 2 deletions packages/dev-env/src/const.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
export const ADMIN_PASSWORD = 'admin-pass'
export const MOD_PASSWORD = 'mod-pass'
export const TRIAGE_PASSWORD = 'triage-pass'
export const JWT_SECRET = 'jwt-secret'
2 changes: 2 additions & 0 deletions packages/dev-env/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ export * from './network'
export * from './network-no-appview'
export * from './pds'
export * from './plc'
export * from './ozone'
export * from './feed-gen'
export * from './seed'
export * from './moderator-client'
export * from './types'
export * from './util'
23 changes: 23 additions & 0 deletions packages/dev-env/src/mock/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,29 @@ export async function generateMockSetup(env: TestNetwork) {
)
}

// Create moderator accounts
const triageRes =
await clients.loggedout.api.com.atproto.server.createAccount({
email: '[email protected]',
handle: 'triage.test',
password: 'triage-pass',
})
env.ozone.addAdminDid(triageRes.data.did)
const modRes = await clients.loggedout.api.com.atproto.server.createAccount({
email: '[email protected]',
handle: 'mod.test',
password: 'mod-pass',
})
env.ozone.addAdminDid(modRes.data.did)
const adminRes = await clients.loggedout.api.com.atproto.server.createAccount(
{
email: '[email protected]',
handle: 'admin-mod.test',
password: 'admin-mod-pass',
},
)
env.ozone.addAdminDid(adminRes.data.did)

// Report one user
const reporter = picka(users)
await reporter.agent.api.com.atproto.moderation.createReport({
Expand Down
138 changes: 138 additions & 0 deletions packages/dev-env/src/moderator-client.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
import AtpAgent from '@atproto/api'
import { InputSchema as TakeActionInput } from '@atproto/api/src/client/types/com/atproto/admin/emitModerationEvent'
import { QueryParams as QueryStatusesParams } from '@atproto/api/src/client/types/com/atproto/admin/queryModerationStatuses'
import { QueryParams as QueryEventsParams } from '@atproto/api/src/client/types/com/atproto/admin/queryModerationEvents'
import { TestOzone } from './ozone'

type ModLevel = 'admin' | 'moderator' | 'triage'

export class ModeratorClient {
agent: AtpAgent
constructor(public ozone: TestOzone) {
this.agent = ozone.getClient()
}

async getEvent(id: number, role?: ModLevel) {
const result = await this.agent.api.com.atproto.admin.getModerationEvent(
{ id },
{
headers: await this.ozone.modHeaders(role),
},
)
return result.data
}

async queryModerationStatuses(input: QueryStatusesParams, role?: ModLevel) {
const result =
await this.agent.api.com.atproto.admin.queryModerationStatuses(input, {
headers: await this.ozone.modHeaders(role),
})
return result.data
}

async queryModerationEvents(input: QueryEventsParams, role?: ModLevel) {
const result = await this.agent.api.com.atproto.admin.queryModerationEvents(
input,
{
headers: await this.ozone.modHeaders(role),
},
)
return result.data
}

async emitModerationEvent(
opts: {
event: TakeActionInput['event']
subject: TakeActionInput['subject']
subjectBlobCids?: TakeActionInput['subjectBlobCids']
reason?: string
createdBy?: string
meta?: TakeActionInput['meta']
},
role?: ModLevel,
) {
const {
event,
subject,
subjectBlobCids,
reason = 'X',
createdBy = 'did:example:admin',
} = opts
const result = await this.agent.api.com.atproto.admin.emitModerationEvent(
{ event, subject, subjectBlobCids, createdBy, reason },
{
encoding: 'application/json',
headers: await this.ozone.modHeaders(role),
},
)
return result.data
}

async reverseModerationAction(
opts: {
id: number
subject: TakeActionInput['subject']
reason?: string
createdBy?: string
},
role?: ModLevel,
) {
const { subject, reason = 'X', createdBy = 'did:example:admin' } = opts
const result = await this.agent.api.com.atproto.admin.emitModerationEvent(
{
subject,
event: {
$type: 'com.atproto.admin.defs#modEventReverseTakedown',
comment: reason,
},
createdBy,
},
{
encoding: 'application/json',
headers: await this.ozone.modHeaders(role),
},
)
return result.data
}

async performTakedown(
opts: {
subject: TakeActionInput['subject']
subjectBlobCids?: TakeActionInput['subjectBlobCids']
durationInHours?: number
reason?: string
},
role?: ModLevel,
) {
const { durationInHours, ...rest } = opts
return this.emitModerationEvent(
{
event: {
$type: 'com.atproto.admin.defs#modEventTakedown',
durationInHours,
},
...rest,
},
role,
)
}

async performReverseTakedown(
opts: {
subject: TakeActionInput['subject']
subjectBlobCids?: TakeActionInput['subjectBlobCids']
reason?: string
},
role?: ModLevel,
) {
return this.emitModerationEvent(
{
event: {
$type: 'com.atproto.admin.defs#modEventReverseTakedown',
},
...opts,
},
role,
)
}
}
1 change: 1 addition & 0 deletions packages/dev-env/src/network.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ export class TestNetwork extends TestNetworkNoAppView {
await this.pds.processAll()
await this.processFullSubscription(timeout)
await this.bsky.sub.background.processAll()
await this.ozone.processAll()
}

async serviceHeaders(did: string, aud?: string) {
Expand Down
86 changes: 62 additions & 24 deletions packages/dev-env/src/ozone.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
import getPort from 'get-port'
import * as ui8 from 'uint8arrays'
import * as plc from '@did-plc/lib'
import * as ozone from '@atproto/ozone'
import { AtpAgent } from '@atproto/api'
import { createServiceJwt } from '@atproto/xrpc-server'
import { Keypair, Secp256k1Keypair } from '@atproto/crypto'
import * as plc from '@did-plc/lib'
import { OzoneConfig } from './types'
import { ADMIN_PASSWORD, MOD_PASSWORD, TRIAGE_PASSWORD } from './const'
import { DidAndKey, OzoneConfig } from './types'
import { ADMIN_PASSWORD } from './const'
import { createDidAndKey } from './util'
import { ModeratorClient } from './moderator-client'

export class TestOzone {
constructor(
public url: string,
public port: number,
public server: ozone.OzoneService,
public daemon: ozone.OzoneDaemon,
public adminAccnt: DidAndKey,
public moderatorAccnt: DidAndKey,
public triageAccnt: DidAndKey,
) {}

static async create(config: OzoneConfig): Promise<TestOzone> {
Expand All @@ -24,6 +30,24 @@ export class TestOzone {
serverDid = await createOzoneDid(config.plcUrl, serviceKeypair)
}

const admin = await createDidAndKey({
plcUrl: config.plcUrl,
handle: 'admin.ozone',
pds: 'https://pds.invalid',
})

const moderator = await createDidAndKey({
plcUrl: config.plcUrl,
handle: 'moderator.ozone',
pds: 'https://pds.invalid',
})

const triage = await createDidAndKey({
plcUrl: config.plcUrl,
handle: 'triage.ozone',
pds: 'https://pds.invalid',
})

const port = config.port || (await getPort())
const url = `http://localhost:${port}`

Expand All @@ -37,11 +61,13 @@ export class TestOzone {
signingKeyHex,
...config,
adminPassword: ADMIN_PASSWORD,
moderatorPassword: MOD_PASSWORD,
triagePassword: TRIAGE_PASSWORD,
adminDids: [],
moderatorDids: [],
triageDids: [],
adminDids: [...(config.adminDids ?? []), admin.did],
moderatorDids: [
...(config.moderatorDids ?? []),
config.appviewDid,
moderator.did,
],
triageDids: [...(config.triageDids ?? []), triage.did],
}

// Separate migration db in case migration changes some connection state that we need in the tests, e.g. "alter database ... set ..."
Expand Down Expand Up @@ -70,7 +96,7 @@ export class TestOzone {
// don't do event reversal in dev-env
await daemon.ctx.eventReverser.destroy()

return new TestOzone(url, port, server, daemon)
return new TestOzone(url, port, server, daemon, admin, moderator, triage)
}

get ctx(): ozone.AppContext {
Expand All @@ -81,23 +107,35 @@ export class TestOzone {
return new AtpAgent({ service: this.url })
}

adminAuth(role: 'admin' | 'moderator' | 'triage' = 'admin'): string {
const password =
role === 'triage'
? TRIAGE_PASSWORD
: role === 'moderator'
? MOD_PASSWORD
: ADMIN_PASSWORD
return (
'Basic ' +
ui8.toString(ui8.fromString(`admin:${password}`, 'utf8'), 'base64pad')
)
getModClient() {
return new ModeratorClient(this)
}

adminAuthHeaders(role?: 'admin' | 'moderator' | 'triage') {
return {
authorization: this.adminAuth(role),
}
addAdminDid(did: string) {
this.ctx.cfg.access.admins.push(did)
}

addModeratorDid(did: string) {
this.ctx.cfg.access.moderators.push(did)
}

addTriageDid(did: string) {
this.ctx.cfg.access.triage.push(did)
}

async modHeaders(role: 'admin' | 'moderator' | 'triage' = 'moderator') {
const account =
role === 'admin'
? this.adminAccnt
: role === 'moderator'
? this.moderatorAccnt
: this.triageAccnt
const jwt = await createServiceJwt({
iss: account.did,
aud: this.ctx.cfg.service.did,
keypair: account.key,
})
return { authorization: `Bearer ${jwt}` }
}

async processAll() {
Expand Down
Loading
Loading