[Bug] Python Realtime Client Connection Timeouts and Failed Subscriptions

[jackyliang]

[Bug] Python Realtime Client Connection Timeouts and Failed Subscriptions

• I confirm this is a bug with Supabase, not with my own application.
• I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

The Supabase Realtime client in Python is experiencing connection timeouts and failures when attempting to subscribe to database changes. The client repeatedly fails to maintain a connection with errors like "keepalive ping timeout" and "join push timeout for channel", despite having retry logic implemented.

As a workaround, I've implemented a manual sync on startup that fetches all existing records and syncs them to Redis, but this doesn't solve the real-time updates issue.

Key error messages:

• join push timeout for channel realtime:config-changes
• send push failed: sent 1011 (internal error) keepalive ping timeout
• Error in connection check

To Reproduce

The issue occurs when using the AsyncRealtimeClient to subscribe to Postgres changes in Supabase. The relevant code setup is:

self.realtime_client = AsyncRealtimeClient(
    f"{self.supabase_url}/realtime/v1",
    self.supabase_service_key,
    auto_reconnect=True,
    params={
        'schema': 'public',
        'heartbeat_interval': 15,
        'timeout': 30,
        'retry_after': 1,
        'max_retries': 10
    }
)

The client attempts to subscribe to two channels:

1. Theme changes on the 'themes' table
2. Config changes on the 'assistants' table

Manual sync implementation as fallback:

async def sync_all_existing(self):
    try:
        supabase = get_supabase()
        assistants = supabase.table('assistants').select('*').execute()
        logger.info(f"Syncing {len(assistants.data)} existing assistants")
        
        for record in assistants.data:
            assistant_name = record["formatted_name"]
            config = {
                "BUSINESS_NAME": record.get("formatted_name", ""),
                "COMPANY_API_KEY": record.get("company_api_key", ""),
                # ... other config fields
            }
            logger.info(f"Manual sync - Updating config in Redis: {assistant_name}")
            self.redis.set(f"config:{assistant_name}", json.dumps(config))
    except Exception as e:
        logger.error(f"Error during manual sync: {str(e)}")

Expected behavior

• The Realtime client should maintain a stable connection
• Successfully subscribe to channel changes
• Receive real-time updates when database changes occur
• Automatically reconnect when connection is lost
• Manual sync should only be needed on initial startup, not as a workaround for failed real-time connections

System information

• Using Python Supabase client
• Dependencies:
  • realtime==2.0.0
  • supabase==x.x.x (version from your requirements.txt)

Additional context

1. The issue persists despite having:

• Automatic reconnection enabled
• Exponential backoff retry logic
• Multiple retry attempts configured
• Manual sync fallback implemented

2. Current workarounds attempted:

• Implemented heartbeat checks
• Added connection status monitoring
• Included error logging
• Added exponential backoff for retries
• Added manual sync on startup and reconnection attempts

3. Questions for Supabase team:

• Are there known issues with Python Realtime client stability?
• Are the current timeout values (heartbeat_interval: 15, timeout: 30) appropriate?
• Should we implement different reconnection strategies?
• Are there recommended best practices for handling these specific timeout errors?
• Is there a better way to handle the initial sync that wouldn't require falling back to manual syncs?

4. Error logs:

2024-11-19 14:11:24,098:ERROR - join push timeout for channel realtime:config-changes
2024-11-19 14:11:26,099:INFO - send: {"topic": "realtime:theme-changes", "event": "phx_join"...}
2024-11-19 14:11:26,100:ERROR - send push failed: sent 1011 (internal error) keepalive ping timeout; no close frame received
2024-11-19 14:11:26,100:INFO - send: {"topic": "realtime:config-changes", "event": "phx_join"...}
2024-11-19 14:11:26,100:ERROR - send push failed: sent 1011 (internal error) keepalive ping timeout; no close frame received