Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

wip(sync): POC of using Redis for pubsub #2426

Merged
merged 18 commits into from
Jan 27, 2025
Merged

wip(sync): POC of using Redis for pubsub #2426

merged 18 commits into from
Jan 27, 2025

Conversation

yohanboniface
Copy link
Member

@yohanboniface yohanboniface commented Jan 13, 2025 •
edited
Loading

TODO

  • add expire to peers registry hash in redis, as for now when the server closes the connection we have extra users (edit: we cleaned manually, as HEXPIRE is not available in FOSS version of Redis)
  • make that the peer uuid is created by the client, so when it reconnects, it uses the same, and does not create a new one
  • see if we can use a connection_pool
  • use dynamic websocket_uri (that must include the map id)
  • integrate Redis in playwright tests

@yohanboniface yohanboniface marked this pull request as ready for review January 22, 2025 09:48
@yohanboniface yohanboniface changed the base branch from ws-without-channel to master January 23, 2025 16:13
yohanboniface and others added 15 commits January 23, 2025 17:16
@yohanboniface @davidbgk
wip(sync): use django-channels to serve websockets
31546d6 
Co-authored-by: David Larlet <[email protected]>
@yohanboniface
wip(sync): websocket server with ASGI and PostgreSQL LISTEN/NOTIFY
a29eae1
@yohanboniface
wip(sync): only save Peer after authentication
698c74b
@yohanboniface
wip(sync): only open listen connections after authentication
460a0c9
@yohanboniface
wip(sync): use Daphne as live_server for tests
ab7119e 
Also clean dependencies.

We still use the channels live server for our tests, but do not use it
anymore for the actual websocket handling.
@yohanboniface @davidbgk
wip(sync): POC of using Redis for pubsub
acb2e96 
Co-authored-by: David Larlet <[email protected]>
@yohanboniface @davidbgk
wip(sync): use the correct URL for websocket
36d9e9b 
Co-authored-by: David Larlet <[email protected]>
@yohanboniface
wip(sync): make the client set its peer id
1bf100d
@yohanboniface @davidbgk
wip(sync): use our async_live_server for websocket related PW tests
a07ee48 
As now both http and ws are on the same domain, let's use one test server
able to serve both.

Co-authored-by: David Larlet <[email protected]>
@yohanboniface
wip(sync): add Redis to CI and configure tests settings
7e42331
@yohanboniface
wip(sync): try a better pattern to unsubscribe to pubsub channels
82342ea 
When publishing a "STOP", this would unsubscribe every listener of the
channel.
@yohanboniface
wip(sync): remove a bit of server prints
ef7c769
@yohanboniface
wip(sync): log but do not crash when sending fail
11fb29c 
This should be a race condition when sending to a closed websocket. Let's
log to track them.
@yohanboniface
wip(sync): only return peers with an active connection
0d5e304
@yohanboniface @davidbgk
wip(sync): clean stale username from redis
476c160 
We wanted to use the HEXPIRE command, but discovered that this command
is only available since the Redis 7.4 version (the latest), and this version
does not have an OSI compliant licence, so it is generally not installable
through common packages managers. The OSS fork is Valkey, but it still does
not have the HEXPIRE command.

So we decide to clean those keys manually, and in order no do this clean
task at each websocket connection, we only do it when we are the first user
to connect to a given map.

Co-authored-by: David Larlet <[email protected]>
@yohanboniface yohanboniface merged commit 60918e6 into master Jan 27, 2025
4 checks passed
@yohanboniface yohanboniface deleted the ws-with-redis branch January 27, 2025 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@yohanboniface