feat: redis streams pubsub implementation

[Ubisoft-potato]

Key Changes

1. Global PubSub Configuration

• Added centralized PubSub configuration in values.yaml and values.dev.yaml under global.pubsub
• Standardized configuration includes:
  • PubSub type (google/redis-stream)
  • Redis connection settings (address, pool size, min idle)
  • Project ID
  • Emulator host settings

2. Deployment Template Updates

Updated deployment templates for all components (web, api, batch, subscriber) to use global PubSub values:

• Removed component-specific PubSub configurations
• Added environment variables for PubSub type and Redis settings
• Standardized environment variable naming across components

3. Subscriber Configuration

• Updated subscribers-configmap.yaml to dynamically merge global PubSub settings into subscriber configurations
• Added support for both regular and on-demand subscribers
• Maintained backward compatibility with existing subscriber configurations

4. Redis Stream Implementation

Added new Redis Stream-based PubSub implementation:

• Created stream_client.go for Redis Stream client implementation
• Implemented stream_publisher.go and stream_puller.go for Redis Stream operations
• Added mock implementations for testing

5. Factory Pattern Implementation

• Introduced factory.go for creating PubSub clients
• Added support for multiple PubSub providers (Google Cloud and Redis Stream)
• Implemented provider-specific configuration options

Migration Notes

• Existing deployments will continue to work with default Google Cloud PubSub
• Redis Stream support can be enabled by setting global.pubsub.type: redis-stream
• No manual intervention required for existing configurations

Breaking Changes

None. This change is backward compatible and maintains existing functionality while adding new capabilities.