Appview v2: don't apply missing modlists #2122
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
devinivy
reviewed
Feb 2, 2024
devinivy
reviewed
Feb 2, 2024
Comment on lines
+107
to
+110
| // if a list no longer exists or is not a mod list, then remove from viewer state | ||
| profileViewers?.forEach((item) => { | ||
| removeNonModListsFromProfileViewer(item, listState) | ||
| }) |
There was a problem hiding this comment.
I'm a little torn on modifying the ProfileViewers in this way. There are implicit dependencies between getProfileViewerStates(), this logic, and mute/block evaluation. We could also handle it on the flip side, when blocks are being evaluated, in viewerBlockExists() and viewerMuteExists(), and in some ways I think that fits better into how hydration vs. view logic currently works. Either case introduces some dependencies, so I'd be for leaving another comment or two to document those. That would be either:
- documenting in
getProfileViewerStates()the dependence on a later call toremoveNonModListsFromProfileViewer()for evaluation of blocks/mutes or - documenting in
viewerMute/BlockExists()that referenced lists must be hydrated.
There was a problem hiding this comment.
yeah i thought about doing it in the view layer
but I think there's a decent argument that this should be handled in hydration. in v1, we were doing this in hydration & the lists on ProfileViewers don't actually "exist". Imo this isn't "view logic", it's an additional piece of logic on hydration that would ideally be handled in the hydration query but isn't because everything is denormalized & this is expensive to materialize 🤔
Def down to rework logic/document it better
There was a problem hiding this comment.
I agree it would be more ideal for this to be in getProfileViewerStates but then it mixes it up with list logic which is kinda pretty strange because we have to jam a list hydration call in there...
maybe we could add a higher level getProfilerViewerStates to the hydrator that handles this logic
and have a lower level getUncheckedProfileViewerStates on the profile hydrator that calls out the response is incomplete
Co-authored-by: devin ivy <[email protected]>
devinivy
approved these changes
Feb 2, 2024
devinivy
approved these changes
Feb 2, 2024
devinivy
added a commit
that referenced
this pull request
Feb 27, 2024
* add buf & connectrpc, codegen client * lint * prettier ignore * fix prettier ignore * tidy & add tests * filler commit * rm filler * server boilerplate * follows impl * posts impl * posts & likes impl * repost impl * profiles & handle null values * list impl * mutes impl * blocks impl * misc * feed gen impl * label impl * notifs impl * feeds impl * threads impl * early sketchwork * wip * stub out thick client * in-progress work on hydrator * tweak * hydrate profile labels, detail lists * feedgen hydration * protobuf tweaks * more protobuf tweaks * wip * snake case * moar snake case * tidy actor hydration * tidy parsing * type fixes, renaming, comments in hydrator * hydrate list items and likes * hydrate notifications * feed hydration * graph & label hydration * more record protobufs * pluralize * tweak pbs * use new methods * Setup dataplane grpc client/mock server (#1921) * add buf & connectrpc, codegen client * lint * prettier ignore * fix prettier ignore * tidy & add tests * add record getter mocks * post view hydration * fix up mock dataplane to match new protos * missed one * wire up dataplane in ctx & dev-env * adding some basic views * feed hydration, add blocks to post hydration * pass over notification hydration * tidy * merge * implement getProfile * hydrate post aggregation and viewer state * fix * fix codegen * get some tests passing! * add takedowns & some like bugfixing * all profile tests passing! * likes test * follow endpoints using data plane * reorg follow block rules * reposts * post views! * implement getList w/ dataplane caveat * adjust dataplane getListMembers to return listitem uris * implement getListMutes and -Blocks w/ dataplane * suggestions * timeline * misc view fixes * view fixes for mutes, self-mute/block * author feed * feed gen routes * tidy * misc block/mute fixes * list feed & actor likes * implement getLists, fix some empty cursors * implement getMutes, empty profile description fix * implement getBlocks, block application fix * implement getSuggestedFollowsByActor, needs some fixes * feed generation * search routes * threads * tidy * fix some snaps * fix getSuggestedFollowsByActor * implement listNotifications * implement getUnreadCount w/ dataplane * implement notifications.updateSeen w/ dataplane * 3rd party blocking tests * blocked profile viewer * add search mocks * refactor getFeed * createPipeline -> createPipelineNew * basic replygating functionality on dataplane w/o filtering violating replies * hack threadgates into dataplane, apply gates * deterministic thread orders in dataplane * misc cleanup around dataplane * upgrade typescript to v5.3 * update typescript linter deps * sync bsky proto, codegen * update dataplane, sync with bsky proto updates * remove indexer, ingester, daemon, moderation services from appview * convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane * remove bsky services. db and indexing logic into mock dataplane. * remove tests not needed by appview v2, misc reorg * add basic in-mem repo subscription to dataplane mock * fix dev-env, bsky tests, bsky build * cull bsky service entrypoint * add bsky service readme * build * tidy * tidy, fix pds proxy tests * fix * fix bsky entrypoint deps * support http2 grpc client * build * fix dataplane bad tls config/default * support multiple dataplane urls, retry when unavailable * build * tidy/fix * move dataplane mock tests into their own dir * cover label hydration through timeline test * bring back labels in appview tests * remove unused db primary/replica/coordinator from bsky dataplane * bsky proto add cids to contracts, buf codegen * sync-up bsky data-plane w/ codegen updates * start using dataplane interaction endpoints * add file * avoid overfetching from dataplane, plumb feed items and cids * pass refs through for post viewer state * switch list feeds to use feed item in dataplane * handle not found err on get-thread dataplane call * support use of search service rather than dataplane methods * mark some appview v2 todos * tidy * still use dataplane on search endpoints when search service is not configured * fix pds test * fix up bsky tests & snaps * tidy migrations * fix appview-v2 docker build * Support label issuer tied to appview v2 (#2033) support label issuer tied to appview * Appview v2: handle empty cursor on list notifications (#2017) handle empty cursor on appview listnotifs * Update appview v2 to use author feed enum (#2047) * update bsky protos with author feed enum, misc feed item changes * support new author feed enums in dataplane * fix build * Appview v2: utilize sorted-at field in bsky protos (#2050) utilize new sorted-at field in bsky protos * remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts * Appview v2, sync w/ changes to protos (#2071) * sync bsky protos * sync-up bsky implementation w/ proto changes * Appview v2 initial implementation for getPopularFeedGenerators (#2072) add an initial implementation for getPopularFeedGenerators on appview v2 * merge * fixes * fix feed tests * fix bsync mock * format * remove unused config * fix lockfile * another lockfile fix * fix duplicate type * fix dupplicate test * Appview v2 handling clearly bad cursors (#2092) * make mock dataplane cursors different from v1 cursors * fail open on clearly bad appview cursors * fix pds appview proxy snaps * Appview v2 no notifs seen behavior (#2096) * alter behavior for presenting notifications w/ no last-seen time * fix pds proxy tests * Appview v2 dataplane retries based on client host (#2098) choose dataplane client for retries based on host when possible/relevant * don't apply negated labels * display suspensions on actor profile in appview v2 * Appview v2 use dataplane for identity lookups (#2095) * update bsky proto w/ identity methods * setup identity endpoints on mock dataplane * move from idresolver to dataplane for identity lookups on appview * tidy * Appview v2: apply safe takedown refs to records, actors (#2107) apply safe takedown refs to records, actors * Fix timing on appview v2 repo rev header (#2113) fix timing on appview repo rev * fix post thread responses * Appview v2 don't apply 3p self blocks (#2112) do not apply 3p self-blocks * Appview v2 search for feed generators (#2118) * add protos for feedgen search * support feed search on getPopularFeedGenerators * Appview v2 config tidy (#2117) * remove mod and triage roles from appview * rename cdn and search config * remove custom feed harness from appview v2 * Appview v2: don't apply missing modlists (#2122) * dont apply missing mod lists * update mock dataplane * Update packages/bsky/src/hydration/hydrator.ts Co-authored-by: devin ivy <[email protected]> * refactor & document a bit better * fix up other routes --------- Co-authored-by: devin ivy <[email protected]> * Appview v2 enforce post thread root boundary (#2120) * enforce post thread root boundary * test thread root boundary * Appview v2 fix admin environment variable (#2137) fix admin env in appview v2 * Remove re-pagination from getSuggestions (#2145) * remove re-pagination from getSuggestions * fix test * Adjust wording for account suspension (#2153) adjust wording for account suspension * Appview v2: fix not-found and blocked uris in threads (#2201) * fix uris of not-found and blocked posts in threads * update snaps * ✨ Show author feed of takendown author to admins only (#2197) * fold in cid, auth, tracing, node version changes * remove dead config from bsky service entrypoint * build * remove ozone test codepaths for appview v2 * tidy, docs fix --------- Co-authored-by: Devin Ivy <[email protected]> Co-authored-by: Foysal Ahamed <[email protected]>
dholms
added a commit
that referenced
this pull request
Mar 6, 2024
* adding some basic views * feed hydration, add blocks to post hydration * pass over notification hydration * tidy * merge * implement getProfile * hydrate post aggregation and viewer state * fix * fix codegen * get some tests passing! * add takedowns & some like bugfixing * all profile tests passing! * likes test * follow endpoints using data plane * reorg follow block rules * reposts * post views! * implement getList w/ dataplane caveat * adjust dataplane getListMembers to return listitem uris * implement getListMutes and -Blocks w/ dataplane * suggestions * timeline * misc view fixes * view fixes for mutes, self-mute/block * author feed * feed gen routes * tidy * misc block/mute fixes * list feed & actor likes * implement getLists, fix some empty cursors * implement getMutes, empty profile description fix * implement getBlocks, block application fix * implement getSuggestedFollowsByActor, needs some fixes * feed generation * search routes * threads * tidy * fix some snaps * fix getSuggestedFollowsByActor * implement listNotifications * implement getUnreadCount w/ dataplane * implement notifications.updateSeen w/ dataplane * 3rd party blocking tests * blocked profile viewer * add search mocks * refactor getFeed * createPipeline -> createPipelineNew * basic replygating functionality on dataplane w/o filtering violating replies * hack threadgates into dataplane, apply gates * deterministic thread orders in dataplane * misc cleanup around dataplane * upgrade typescript to v5.3 * update typescript linter deps * sync bsky proto, codegen * update dataplane, sync with bsky proto updates * remove indexer, ingester, daemon, moderation services from appview * convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane * remove bsky services. db and indexing logic into mock dataplane. * remove tests not needed by appview v2, misc reorg * add basic in-mem repo subscription to dataplane mock * fix dev-env, bsky tests, bsky build * cull bsky service entrypoint * add bsky service readme * build * tidy * tidy, fix pds proxy tests * fix * fix bsky entrypoint deps * support http2 grpc client * build * fix dataplane bad tls config/default * support multiple dataplane urls, retry when unavailable * build * tidy/fix * move dataplane mock tests into their own dir * cover label hydration through timeline test * bring back labels in appview tests * remove unused db primary/replica/coordinator from bsky dataplane * bsky proto add cids to contracts, buf codegen * sync-up bsky data-plane w/ codegen updates * start using dataplane interaction endpoints * add file * avoid overfetching from dataplane, plumb feed items and cids * pass refs through for post viewer state * Lexicons: Add labeler prefs, labeler declaration, and get labeler routes * Add labelerViewBasic and update embed views * Fix typo * switch list feeds to use feed item in dataplane * handle not found err on get-thread dataplane call * support use of search service rather than dataplane methods * mark some appview v2 todos * tidy * still use dataplane on search endpoints when search service is not configured * fix pds test * Switch to labelerViewDetailed * Move label and report values to refs * Add getActorLabelers * lint * fix up bsky tests & snaps * small diff to open pr * rm new line * codegen schemas * tidy migrations * table + indexing * protos * rename lexicons * views, hydration + rename lexicons * rest of routes * data plane routes * parse labelers from req * fix appview-v2 docker build * Support label issuer tied to appview v2 (#2033) support label issuer tied to appview * hydrate context * tidy header logic * integrating into more routes * more routes * wrap up rest * add mock labeler * rework labelerlexicons * tidy lexs * codegen new lexicons * integrate lexicon rework * add proxy logic * forward labeler headers through pds * tweak label header parsing * remove did from scheams * update indexing for lexs * tests for mod service views * label hydration test * Add 'associated' to profileViewDetailed * Rename labelers to mods in preferences * Change uri to did in mod preferences * couple more * syntax tweaks * integrate updated lexicons * update view snap * handle mod service embeds * tidy * fix mock * lint * base default labels of config var * fix label hydration * Appview v2: handle empty cursor on list notifications (#2017) handle empty cursor on appview listnotifs * Update appview v2 to use author feed enum (#2047) * update bsky protos with author feed enum, misc feed item changes * support new author feed enums in dataplane * fix build * Appview v2: utilize sorted-at field in bsky protos (#2050) utilize new sorted-at field in bsky protos * remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts * Appview v2, sync w/ changes to protos (#2071) * sync bsky protos * sync-up bsky implementation w/ proto changes * Appview v2 initial implementation for getPopularFeedGenerators (#2072) add an initial implementation for getPopularFeedGenerators on appview v2 * merge * fixes * fix feed tests * fix bsync mock * format * remove unused config * fix lockfile * another lockfile fix * fix duplicate type * fix dupplicate test * Appview v2 handling clearly bad cursors (#2092) * make mock dataplane cursors different from v1 cursors * fail open on clearly bad appview cursors * fix pds appview proxy snaps * Appview v2 no notifs seen behavior (#2096) * alter behavior for presenting notifications w/ no last-seen time * fix pds proxy tests * Appview v2 dataplane retries based on client host (#2098) choose dataplane client for retries based on host when possible/relevant * don't apply negated labels * display suspensions on actor profile in appview v2 * Appview v2 use dataplane for identity lookups (#2095) * update bsky proto w/ identity methods * setup identity endpoints on mock dataplane * move from idresolver to dataplane for identity lookups on appview * tidy * Appview v2: apply safe takedown refs to records, actors (#2107) apply safe takedown refs to records, actors * Fix timing on appview v2 repo rev header (#2113) fix timing on appview repo rev * fix post thread responses * Appview v2 don't apply 3p self blocks (#2112) do not apply 3p self-blocks * Appview v2 search for feed generators (#2118) * add protos for feedgen search * support feed search on getPopularFeedGenerators * Appview v2 config tidy (#2117) * remove mod and triage roles from appview * rename cdn and search config * remove custom feed harness from appview v2 * Appview v2: don't apply missing modlists (#2122) * dont apply missing mod lists * update mock dataplane * Update packages/bsky/src/hydration/hydrator.ts Co-authored-by: devin ivy <[email protected]> * refactor & document a bit better * fix up other routes --------- Co-authored-by: devin ivy <[email protected]> * Appview v2 enforce post thread root boundary (#2120) * enforce post thread root boundary * test thread root boundary * Appview v2 fix admin environment variable (#2137) fix admin env in appview v2 * Remove re-pagination from getSuggestions (#2145) * remove re-pagination from getSuggestions * fix test * Adjust wording for account suspension (#2153) adjust wording for account suspension * Appview v2: fix not-found and blocked uris in threads (#2201) * fix uris of not-found and blocked posts in threads * update snaps * ✨ Show author feed of takendown author to admins only (#2197) * fold in cid, auth, tracing, node version changes * remove dead config from bsky service entrypoint * build * remove ozone test codepaths for appview v2 * tidy, docs fix * fix test * add additional user counts * add associated data to profiles * update snaps * update to is_mod_service * format * tidy * 3p labeler sdk updates (#2176) * Update sdk to support 3p labeler preferences * Stick with intolerance instead of hate for the label group id * wip expand labels and label groups * Output moderationOpts (computed) and modsPref (unaltered) * Add tests for enabling/disabling mod services * Add atproto-labelers header config * Expand labels and label groups in definitions * Fix tests * Tweaks to labels * Remove label descriptions and improve output types on labels and label groups * Add mocker to exported API * Improve types of label and label group definitions * Rework moderation prefs to continue using global labelgroup settings and only disable label groups per moderator * Simplify encoding of the label preferences in definitions * Add target constraints to labels * Refactor the moderation sdk to derive more behaviors from the definition files * Small cleanup * Add hiding tool to modsdk * Track filter causes * Make mute state an alert * Fix: dont blur profileview for blocks * Prioritize causes by severity * Add moderateNotification() and drop quote post moderation code * Add mocker functions for notifications * Improve mock data * Lexicon: Add custom label definitions and remove modservice descriptions * Lexicon: Update moderation prefs * SDK updates: remove label groups, reduce builtin labels, update mod-preference apis * Lexicon: Update global labels with new reduced set * Lexicon: Remove moderation.getService and add detailed option to getServices * Lexicons: add severity=none to custom label value defs * Implement custom label-value definition tooling * All custom labels are no-self * Backend impl for labeler lexicon updates (#2258) * codegen * clean up impl * fix up tests * Lexicon: modservice -> labeler * Remove x- prefix behavior; add label value syntax rules; add custom label precedence rules * Lexicon: Remove the ability to choose a defaultSetting from custom labels * Rework test suites * Give behaviors to all labels regardless of target * sync up backend with lex changes * fix labelers in dev-env agent * lint protos * update protos & views * small bugfix & update tests * tweak protos * fix build issue from merge --------- Co-authored-by: Devin Ivy <[email protected]> Co-authored-by: Paul Frazee <[email protected]> Co-authored-by: Foysal Ahamed <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When hydrating a profile basic, check to ensure that relevant mod lists both exist & are have a purpose of being modlists. If the list does not exist as a modlist, then remove it from profile viewer state so that it does not affect filtering & views