Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Action !takedown labels in hydrator #2270

Merged
merged 19 commits into from
Mar 12, 2024
67 changes: 59 additions & 8 deletions packages/bsky/src/hydration/hydrator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { ids } from '../lexicon/lexicons'
import { isMain as isEmbedRecord } from '../lexicon/types/app/bsky/embed/record'
import { isMain as isEmbedRecordWithMedia } from '../lexicon/types/app/bsky/embed/recordWithMedia'
import { isListRule } from '../lexicon/types/app/bsky/feed/threadgate'
import type { Label } from '../lexicon/types/com/atproto/label/defs'
import {
ActorHydrator,
ProfileAggs,
Expand Down Expand Up @@ -126,6 +127,9 @@ export class Hydrator {
this.label.getLabelsForSubjects(labelSubjectsForDid(dids)),
viewer ? this.hydrateProfileViewers(dids, viewer) : undefined,
])
if (!includeTakedowns) {
actionTakedownLabels(dids, actors, labels)
}
return mergeStates(profileViewersState ?? {}, {
actors,
labels,
Expand Down Expand Up @@ -170,10 +174,15 @@ export class Hydrator {
async hydrateLists(
uris: string[],
viewer: string | null,
includeTakedowns = false,
): Promise<HydrationState> {
const [listsState, profilesState] = await Promise.all([
await this.hydrateListsBasic(uris, viewer),
await this.hydrateProfilesBasic(uris.map(didFromUri), viewer),
await this.hydrateListsBasic(uris, viewer, includeTakedowns),
await this.hydrateProfilesBasic(
uris.map(didFromUri),
viewer,
includeTakedowns,
),
])
return mergeStates(listsState, profilesState)
}
Expand All @@ -183,12 +192,19 @@ export class Hydrator {
async hydrateListsBasic(
uris: string[],
viewer: string | null,
includeTakedowns = false,
): Promise<HydrationState> {
const [lists, listViewers] = await Promise.all([
const [lists, listViewers, labels] = await Promise.all([
this.graph.getLists(uris),
viewer ? this.graph.getListViewerStates(uris, viewer) : undefined,
this.label.getLabelsForSubjects(uris),
])
return { lists, listViewers, viewer }

if (!includeTakedowns) {
actionTakedownLabels(uris, lists, labels)
}

return { lists, listViewers, labels, viewer }
}

// app.bsky.graph.defs#listItemView
Expand Down Expand Up @@ -286,9 +302,16 @@ export class Hydrator {
viewer,
includeTakedowns,
),
this.hydrateLists([...nestedListUris, ...gateListUris], viewer),
this.hydrateFeedGens(nestedFeedGenUris, viewer),
this.hydrateLists(
[...nestedListUris, ...gateListUris],
viewer,
includeTakedowns,
),
this.hydrateFeedGens(nestedFeedGenUris, viewer, includeTakedowns),
])
if (!includeTakedowns) {
actionTakedownLabels(allPostUris, posts, labels)
}
// combine all hydration state
return mergeManyStates(profileState, listState, feedGenState, {
posts,
Expand Down Expand Up @@ -401,8 +424,9 @@ export class Hydrator {
async hydrateThreadPosts(
refs: ItemRef[],
viewer: string | null,
includeTakedowns = false,
): Promise<HydrationState> {
return this.hydratePosts(refs, viewer)
return this.hydratePosts(refs, viewer, includeTakedowns)
}

// app.bsky.feed.defs#generatorView
Expand All @@ -412,18 +436,24 @@ export class Hydrator {
async hydrateFeedGens(
uris: string[], // @TODO any way to get refs here?
viewer: string | null,
includeTakedowns = false,
): Promise<HydrationState> {
const [feedgens, feedgenAggs, feedgenViewers, profileState] =
const [feedgens, feedgenAggs, feedgenViewers, profileState, labels] =
await Promise.all([
this.feed.getFeedGens(uris),
this.feed.getFeedGenAggregates(uris.map((uri) => ({ uri }))),
viewer ? this.feed.getFeedGenViewerStates(uris, viewer) : undefined,
this.hydrateProfiles(uris.map(didFromUri), viewer),
this.label.getLabelsForSubjects(uris),
])
if (!includeTakedowns) {
actionTakedownLabels(uris, feedgens, labels)
}
return mergeStates(profileState, {
feedgens,
feedgenAggs,
feedgenViewers,
labels,
viewer,
})
}
Expand Down Expand Up @@ -478,6 +508,7 @@ export class Hydrator {
this.label.getLabelsForSubjects(uris),
this.hydrateProfiles(uris.map(didFromUri), viewer),
])
actionTakedownLabels(postUris, posts, labels)
return mergeStates(profileState, {
posts,
likes,
Expand Down Expand Up @@ -724,3 +755,23 @@ const mergeManyStates = (...states: HydrationState[]) => {
const mergeManyMaps = <T>(...maps: HydrationMap<T>[]) => {
return maps.reduce(mergeMaps, undefined as HydrationMap<T> | undefined)
}

const actionTakedownLabels = <T>(
keys: string[],
hydrationMap: HydrationMap<T>,
labels: Labels,
) => {
for (const key of keys) {
const subjectLabels = labels.get(key)
if (!subjectLabels) continue
if (includesTakedownLabel(subjectLabels)) {
hydrationMap.set(key, null)
}
}
}

const includesTakedownLabel = (labels: Label[]) => {
return labels.some((l) => l.val === TAKEDOWN_LABEL)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it guaranteed that none of the labels here have a neg: true?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be, but I'll add a check just in case 👍

}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So many labels pass through here and the hit rate on takedowns is so low that I wouldn't shy away from a little optimization... if it's straightforward. Maybe it would be easy to make the label hydration state a little richer to track which lists have a takedown? If it's not straightforward then I'm sure it's fine for the time being as-is.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I like it - I added a materialized isTakendown flag to the label hydration map

Also makes it easier to extend the functionality with either more labels that toggle the flag or other types of flags 👍


const TAKEDOWN_LABEL = '!takedown'
Loading