Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Appview v2 #1924

Merged
merged 201 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
201 commits
Select commit Hold shift + click to select a range
3be2fb7
add buf & connectrpc, codegen client
dholms Dec 5, 2023
15f1571
lint
dholms Dec 5, 2023
3deaa28
prettier ignore
dholms Dec 5, 2023
f6a8a0d
fix prettier ignore
dholms Dec 5, 2023
580ee07
tidy & add tests
dholms Dec 5, 2023
83c768f
filler commit
dholms Dec 5, 2023
832e42b
rm filler
dholms Dec 5, 2023
9c24cd2
server boilerplate
dholms Dec 5, 2023
4fd01a4
follows impl
dholms Dec 5, 2023
78fe428
posts impl
dholms Dec 5, 2023
f9037e6
posts & likes impl
dholms Dec 5, 2023
5782f5b
repost impl
dholms Dec 5, 2023
afd3823
profiles & handle null values
dholms Dec 5, 2023
2c0fc18
list impl
dholms Dec 5, 2023
9976d15
mutes impl
dholms Dec 5, 2023
2935d66
blocks impl
dholms Dec 5, 2023
58ec69e
misc
dholms Dec 5, 2023
f142eb9
feed gen impl
dholms Dec 6, 2023
b6bbee1
label impl
dholms Dec 6, 2023
c88674e
notifs impl
dholms Dec 6, 2023
de217c8
feeds impl
dholms Dec 6, 2023
d9ca2db
threads impl
dholms Dec 6, 2023
29d2de7
early sketchwork
dholms Dec 6, 2023
9fcfdc0
wip
dholms Dec 6, 2023
1d61b4b
stub out thick client
dholms Dec 6, 2023
c4ed481
in-progress work on hydrator
devinivy Dec 6, 2023
b4e6e87
tweak
dholms Dec 6, 2023
90e7930
hydrate profile labels, detail lists
devinivy Dec 6, 2023
40509fa
feedgen hydration
devinivy Dec 7, 2023
eca6a58
protobuf tweaks
dholms Dec 7, 2023
eef3a81
more protobuf tweaks
dholms Dec 7, 2023
92d9060
wip
dholms Dec 7, 2023
ccd3b92
snake case
dholms Dec 7, 2023
cba4c55
moar snake case
dholms Dec 7, 2023
3440b3e
tidy actor hydration
dholms Dec 7, 2023
bf5cb7f
tidy parsing
dholms Dec 7, 2023
eee2864
type fixes, renaming, comments in hydrator
devinivy Dec 7, 2023
4cd3c27
hydrate list items and likes
devinivy Dec 7, 2023
b6d1abe
hydrate notifications
devinivy Dec 7, 2023
4d941c5
feed hydration
dholms Dec 7, 2023
9999f1f
graph & label hydration
dholms Dec 7, 2023
5db9111
more record protobufs
dholms Dec 7, 2023
3349fe2
pluralize
dholms Dec 7, 2023
b9ca447
tweak pbs
dholms Dec 7, 2023
bfd89e2
use new methods
dholms Dec 7, 2023
52580fe
Setup dataplane grpc client/mock server (#1921)
dholms Dec 7, 2023
fadacad
add record getter mocks
dholms Dec 7, 2023
758b69a
merge
dholms Dec 7, 2023
d784a93
Merge branch 'mock-data-plane' into bav-v2-hydration
dholms Dec 7, 2023
a9ff131
Merge pull request #1925 from bluesky-social/mock-data-plane
dholms Dec 7, 2023
0148a65
Merge branch 'appview-v2' into bav-v2-hydration
dholms Dec 7, 2023
a49f483
post view hydration
devinivy Dec 7, 2023
a99f28f
fix up mock dataplane to match new protos
dholms Dec 7, 2023
f3d5c82
missed one
dholms Dec 7, 2023
ce5eecd
wire up dataplane in ctx & dev-env
dholms Dec 7, 2023
d828187
adding some basic views
dholms Dec 7, 2023
a5c227a
feed hydration, add blocks to post hydration
devinivy Dec 7, 2023
bdd2fb8
pass over notification hydration
devinivy Dec 7, 2023
8d669bf
tidy
devinivy Dec 7, 2023
fc70842
merge
dholms Dec 7, 2023
4fb3d24
implement getProfile
dholms Dec 7, 2023
6d162a0
hydrate post aggregation and viewer state
devinivy Dec 7, 2023
95bad68
fix
devinivy Dec 7, 2023
b1dfbde
fix codegen
dholms Dec 7, 2023
2298406
get some tests passing!
dholms Dec 8, 2023
4ab0bbb
add takedowns & some like bugfixing
dholms Dec 8, 2023
9fd84ea
all profile tests passing!
dholms Dec 8, 2023
39aa7b4
likes test
dholms Dec 8, 2023
5cd6fc7
follow endpoints using data plane
devinivy Dec 8, 2023
eedc2ea
reorg follow block rules
devinivy Dec 8, 2023
9d64919
reposts
dholms Dec 8, 2023
7fb5ea9
post views!
dholms Dec 8, 2023
7395f8d
implement getList w/ dataplane caveat
devinivy Dec 8, 2023
571f9f6
adjust dataplane getListMembers to return listitem uris
devinivy Dec 8, 2023
640b792
implement getListMutes and -Blocks w/ dataplane
devinivy Dec 8, 2023
c342d38
suggestions
dholms Dec 8, 2023
0126f08
timeline
dholms Dec 9, 2023
a9e7148
misc view fixes
devinivy Dec 11, 2023
49d79ee
view fixes for mutes, self-mute/block
devinivy Dec 11, 2023
bb3d377
author feed
dholms Dec 12, 2023
396e210
feed gen routes
dholms Dec 12, 2023
1ad4b84
tidy
dholms Dec 12, 2023
bee9c6d
misc block/mute fixes
devinivy Dec 12, 2023
19f2f47
list feed & actor likes
dholms Dec 12, 2023
97fcb3c
implement getLists, fix some empty cursors
devinivy Dec 12, 2023
cd4b277
implement getMutes, empty profile description fix
devinivy Dec 12, 2023
265a2e3
implement getBlocks, block application fix
devinivy Dec 13, 2023
12e8edb
implement getSuggestedFollowsByActor, needs some fixes
devinivy Dec 13, 2023
0db39b6
feed generation
dholms Dec 12, 2023
b6fb606
search routes
dholms Dec 13, 2023
e358500
threads
dholms Dec 13, 2023
defb28c
tidy
dholms Dec 13, 2023
4266b3a
fix some snaps
dholms Dec 13, 2023
d7fdc08
fix getSuggestedFollowsByActor
devinivy Dec 13, 2023
10a10b8
implement listNotifications
devinivy Dec 13, 2023
17bdb01
implement getUnreadCount w/ dataplane
devinivy Dec 13, 2023
3707a75
implement notifications.updateSeen w/ dataplane
devinivy Dec 13, 2023
1f78afa
3rd party blocking tests
dholms Dec 13, 2023
7d249bd
blocked profile viewer
dholms Dec 13, 2023
9f92374
add search mocks
dholms Dec 13, 2023
69d8a2b
refactor getFeed
dholms Dec 14, 2023
c13fca9
createPipeline -> createPipelineNew
dholms Dec 14, 2023
a4dc101
basic replygating functionality on dataplane w/o filtering violating …
devinivy Dec 14, 2023
764e0da
hack threadgates into dataplane, apply gates
devinivy Dec 14, 2023
cecc669
deterministic thread orders in dataplane
devinivy Dec 14, 2023
f089eb5
misc cleanup around dataplane
devinivy Dec 15, 2023
f6de678
upgrade typescript to v5.3
devinivy Dec 15, 2023
549b464
update typescript linter deps
devinivy Dec 15, 2023
70cf5c2
sync bsky proto, codegen
devinivy Dec 19, 2023
9574a86
update dataplane, sync with bsky proto updates
devinivy Dec 19, 2023
d8aa699
remove indexer, ingester, daemon, moderation services from appview
devinivy Dec 20, 2023
61a0389
convert more bsky internals to dataplane, remove custom feedgens, imp…
devinivy Dec 20, 2023
96f3828
remove bsky services. db and indexing logic into mock dataplane.
devinivy Dec 20, 2023
c3fb682
remove tests not needed by appview v2, misc reorg
devinivy Dec 20, 2023
975ba4d
add basic in-mem repo subscription to dataplane mock
devinivy Dec 21, 2023
a48903d
fix dev-env, bsky tests, bsky build
devinivy Dec 21, 2023
8371879
cull bsky service entrypoint
devinivy Dec 21, 2023
012c754
add bsky service readme
devinivy Dec 21, 2023
f2edcfd
build
devinivy Dec 21, 2023
a93ae30
tidy
devinivy Dec 21, 2023
9229cc2
tidy, fix pds proxy tests
devinivy Dec 21, 2023
22b3c2f
Merge pull request #1939 from bluesky-social/bav-v2-hydration
devinivy Dec 21, 2023
3b85cbf
fix
devinivy Dec 22, 2023
3b9a5cb
fix bsky entrypoint deps
devinivy Dec 22, 2023
e8258eb
support http2 grpc client
devinivy Dec 26, 2023
438025e
build
devinivy Dec 27, 2023
f431958
fix dataplane bad tls config/default
devinivy Dec 27, 2023
1dab449
support multiple dataplane urls, retry when unavailable
devinivy Dec 27, 2023
065149a
build
devinivy Dec 27, 2023
6f05b42
tidy/fix
devinivy Dec 27, 2023
1da49cc
move dataplane mock tests into their own dir
devinivy Dec 29, 2023
62603e0
cover label hydration through timeline test
devinivy Dec 29, 2023
b40e8c6
bring back labels in appview tests
devinivy Dec 29, 2023
9b2153f
remove unused db primary/replica/coordinator from bsky dataplane
devinivy Dec 29, 2023
1d8a4f1
Merge pull request #1982 from bluesky-social/bav-v2-drop-pg
devinivy Dec 29, 2023
3feee71
Merge pull request #1992 from bluesky-social/bav-v2-config-pass1
devinivy Dec 29, 2023
c67e1f1
Merge pull request #1993 from bluesky-social/bav-v2-dataplane-clients
devinivy Dec 29, 2023
8668a2f
bsky proto add cids to contracts, buf codegen
devinivy Jan 2, 2024
246fb09
sync-up bsky data-plane w/ codegen updates
devinivy Jan 2, 2024
1ca11d2
start using dataplane interaction endpoints
devinivy Jan 2, 2024
36b72bf
add file
devinivy Jan 2, 2024
dd6fe98
Merge pull request #2008 from bluesky-social/appview-v2-proto-cids
devinivy Jan 2, 2024
5e829ff
avoid overfetching from dataplane, plumb feed items and cids
devinivy Jan 3, 2024
0b6752a
pass refs through for post viewer state
devinivy Jan 3, 2024
53c33f2
switch list feeds to use feed item in dataplane
devinivy Jan 5, 2024
fabb122
handle not found err on get-thread dataplane call
devinivy Jan 5, 2024
34d3a7c
support use of search service rather than dataplane methods
devinivy Jan 5, 2024
deb8c4d
mark some appview v2 todos
devinivy Jan 6, 2024
3d71d89
tidy
devinivy Jan 6, 2024
45c6237
still use dataplane on search endpoints when search service is not co…
devinivy Jan 6, 2024
f006a08
fix pds test
devinivy Jan 6, 2024
88f2f21
merge main
dholms Jan 9, 2024
98613e9
fix up bsky tests & snaps
dholms Jan 9, 2024
14d8773
tidy migrations
dholms Jan 9, 2024
807b38a
fix appview-v2 docker build
devinivy Jan 9, 2024
0a45fea
Support label issuer tied to appview v2 (#2033)
devinivy Jan 9, 2024
cbfe0b9
Merge remote-tracking branch 'origin/appview-v2-fix-overfetching1' in…
devinivy Jan 12, 2024
1c14449
Merge remote-tracking branch 'origin/appview-v2-not-found-search' int…
devinivy Jan 12, 2024
f2cfd45
Appview v2: handle empty cursor on list notifications (#2017)
devinivy Jan 12, 2024
c592ae8
Update appview v2 to use author feed enum (#2047)
devinivy Jan 15, 2024
59704b3
Appview v2: utilize sorted-at field in bsky protos (#2050)
devinivy Jan 15, 2024
75d3079
remove all dataplane usage of GetLikeCounts, switch to GetInteraction…
devinivy Jan 16, 2024
3c6287a
Appview v2, sync w/ changes to protos (#2071)
devinivy Jan 24, 2024
2178b15
Appview v2 initial implementation for getPopularFeedGenerators (#2072)
devinivy Jan 24, 2024
c8129d5
merge
dholms Jan 25, 2024
9935a76
fixes
dholms Jan 25, 2024
3e8431e
fix feed tests
dholms Jan 25, 2024
37ac9b4
fix bsync mock
dholms Jan 25, 2024
714b117
format
dholms Jan 25, 2024
8194bc1
remove unused config
devinivy Jan 25, 2024
96a5a9b
final merge
dholms Jan 25, 2024
827f9e7
fix lockfile
dholms Jan 25, 2024
8f5306f
another lockfile fix
dholms Jan 25, 2024
0fb4e5b
fix duplicate type
dholms Jan 25, 2024
82c0855
fix dupplicate test
dholms Jan 25, 2024
6a664ed
Appview v2 handling clearly bad cursors (#2092)
devinivy Jan 25, 2024
ad72ef7
Appview v2 no notifs seen behavior (#2096)
devinivy Jan 25, 2024
29230db
Appview v2 dataplane retries based on client host (#2098)
devinivy Jan 25, 2024
ed9e234
don't apply negated labels
devinivy Jan 26, 2024
6dd9331
display suspensions on actor profile in appview v2
devinivy Jan 26, 2024
b47e28d
Appview v2 use dataplane for identity lookups (#2095)
devinivy Jan 26, 2024
8e59aad
Appview v2: apply safe takedown refs to records, actors (#2107)
devinivy Jan 29, 2024
abf4d44
Fix timing on appview v2 repo rev header (#2113)
dholms Jan 30, 2024
a2b51fb
fix post thread responses
devinivy Jan 30, 2024
464faba
Appview v2 don't apply 3p self blocks (#2112)
devinivy Jan 31, 2024
1970529
Appview v2 search for feed generators (#2118)
devinivy Jan 31, 2024
be4dc5a
Appview v2 config tidy (#2117)
devinivy Feb 2, 2024
b292023
Appview v2: don't apply missing modlists (#2122)
dholms Feb 2, 2024
374d2b8
Appview v2 enforce post thread root boundary (#2120)
devinivy Feb 2, 2024
5cd08e4
Appview v2 fix admin environment variable (#2137)
devinivy Feb 5, 2024
e610da9
Remove re-pagination from getSuggestions (#2145)
devinivy Feb 7, 2024
94a59e5
Adjust wording for account suspension (#2153)
dholms Feb 8, 2024
8497d18
Appview v2: fix not-found and blocked uris in threads (#2201)
devinivy Feb 20, 2024
c2a6577
:sparkles: Show author feed of takendown author to admins only (#2197)
foysalit Feb 20, 2024
ae0180a
fold in cid, auth, tracing, node version changes
devinivy Feb 26, 2024
d65d814
Merge remote-tracking branch 'origin/main' into appview-v2-merge
devinivy Feb 26, 2024
ed97d0a
remove dead config from bsky service entrypoint
devinivy Feb 26, 2024
bba2eb4
build
devinivy Feb 26, 2024
f855938
remove ozone test codepaths for appview v2
devinivy Feb 26, 2024
5a60f5d
tidy, docs fix
devinivy Feb 26, 2024
4608ad2
Merge pull request #2231 from bluesky-social/appview-v2-merge
devinivy Feb 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 0 additions & 1 deletion .github/workflows/build-and-push-bsky-aws.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ on:
push:
branches:
- main
- appview-v1-courier
env:
REGISTRY: ${{ secrets.AWS_ECR_REGISTRY_USEAST2_PACKAGES_REGISTRY }}
USERNAME: ${{ secrets.AWS_ECR_REGISTRY_USEAST2_PACKAGES_USERNAME }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/build-and-push-bsky-ghcr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ on:
push:
branches:
- main
- appview-v2
env:
REGISTRY: ghcr.io
USERNAME: ${{ github.actor }}
Expand Down
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ pnpm-lock.yaml
.pnpm*
.changeset
*.d.ts
packages/bsky/src/data-plane/gen
10 changes: 9 additions & 1 deletion packages/bsky/bin/migration-create.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,15 @@ export async function main() {
)
}
const filename = `${prefix}-${name}`
const dir = path.join(__dirname, '..', 'src', 'db', 'migrations')
const dir = path.join(
__dirname,
'..',
'src',
'data-plane',
'server',
'db',
'migrations',
)

await fs.writeFile(path.join(dir, `${filename}.ts`), template, { flag: 'wx' })
await fs.writeFile(
Expand Down
2 changes: 1 addition & 1 deletion packages/bsky/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const buildShallow =

require('esbuild').build({
logLevel: 'info',
entryPoints: ['src/index.ts', 'src/db/index.ts'],
entryPoints: ['src/index.ts'],
bundle: true,
sourcemap: true,
outdir: 'dist',
Expand Down
4 changes: 3 additions & 1 deletion packages/bsky/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
"@atproto/xrpc-server": "workspace:^",
"@bufbuild/protobuf": "^1.5.0",
"@connectrpc/connect": "^1.1.4",
"@connectrpc/connect-express": "^1.1.4",
"@connectrpc/connect-node": "^1.1.4",
"@did-plc/lib": "^0.0.1",
"@isaacs/ttlcache": "^1.4.1",
Expand Down Expand Up @@ -80,6 +81,7 @@
"@types/express-serve-static-core": "^4.17.36",
"@types/pg": "^8.6.6",
"@types/qs": "^6.9.7",
"axios": "^0.27.2"
"axios": "^0.27.2",
"http2-express-bridge": "^1.0.7"
}
}
Loading
Loading