All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Add support of Rails 8.0
- Dropped support of Rails 5.2
- use Nexus URL environment variable for internal releases
- fix deleting stale items from Postgis
- fix deleting stale items from MySQL and PostgreSQL
- change
DEFAULT_PARTITION_STRATEGYto string
- log OTEL
trace_id
- Renamed
backtracelog tag tostacktrace
- Fixed handling of errors if database is not available
- Replaced index
add_index :#{table_name}, [:status, :bucket, :errors_count]toadd_index :#{table_name}, [:status, :id, :bucket], algorithm: :concurrently, include: [:errors_count]
- Add outbox context to ActiveRecord query log tags
- Implement an ability to create outbox items in batches through
Sbmt::Outbox::CreateOutboxBatch
- add option
strict_order
- fix README
- use db replica to select stale item ids
- fix undefined local variable or method labels for Sbmt::Outbox::V2::Poller
- replace
/with-in labelnamein outbox_last_stored_event_id / outbox_created_counter
- replace
/with-in labelnamein box metrics
- Add process the SIGQUIT signal.
- added autostart of the yabeda server via the
enabledoption - added disabling autorun of probes via the
enabledoption
- Add box_id to make it url safe for use in the API.
- Add support for Outbox UI
- Add ability to pause poller v2
- buckets for the histogram are brought to a single form
- Add
ownerlabel tojob_countermetric
PollThrottler::RateLimitedis now per box-based
BRPOPtimeout default for queue processing
- support of worker v2 architecture, which simplifies outbox configuration (no manual selection of partitions / concurrency anymore)
- worker v2: worker process is now consists of two parts: poller + processor
- worker v2: each worker process contains X (1 by default) poll-thread and Y (4 by default) process-threads
- worker v2: poller is responsible for async polling database (usually RO-replicas) changes, i.e. inbox/outbox items ready for processing
- worker v2: processor is responsible for transactional processing of inbox/outbox items
- worker v2: poller / processor communicate via redis job queue per inbox/outbox item
- worker v2: poller / processor consistency is backed by redis locks per box/bucket
- worker v2: more poller / processor metrics to simplify process scaling (i.e. HPA)
- worker v2: poller's poll tactics for different performance profiles: default, aggressive, low-priority
- Add global config option
disposable_transports, and set tofalseby default.
- A transport might be invoked as a disposable object.
- fixed backward compatibility with proto_payload
- track errors when the worker get a failure
- dropped support of Ruby 2.5 and 2.6
- removed Sbmt::Outbox::Item
- removed after_commit_everywhere
- removed
redis_servers=config option. Configure Redis withredis=config option - changed the expected result of a transport while processing to only False result is consedered as failed
- make ErrorTracker to send errors to Sentry (if available)
- re-plugged Schked as an optional dependency
- replaced Sidekiq with ActiveJob. Don't forget to set
config.active_job.queue_adapter = :sidekiq - accept box names as arguments in maintenance tasks, ex:
rake outbox:retry_failed_items[some/box_item] - rename proto_payload to payload
- replaced gem fabrication with factory_bot_rails
- now item_process_middlewares is only called if
Sbmt::Outbox::Itemis actually being processed - changed item_process_middlewares call signature:
itemis passed to#callmethod instead ofitem_id,jobandoptionsparameter are removed
- don't include buckets into a partition lock key
- optimized removing of obsolete items by using an index on the
created_atfield
- the size of the redis connection pool is now equal to the number of threads, which will avoid issues with connection blocking.
- added a comment to the migration generator for the proto_payload column if a MySQL database is used
- changed a base class for the BaseItem to ApplicationRecord
- add backtrace limit
- reduced the number of logs for the column
error_log
- plug opentelemetry tracing
- don't cache DB errors in the item processor
- deep symbolize config keys when initializing a Redis client
- fix working with connection errors. The previous approach leads to hanging of the thread. Now, it uses
Rails.application.executor.wrapmechanism.
- symbolize config keys when initializing a Redis client
- get correct lock when deleting old items
- handle and return database errors while processing an item, and then try to recover 3 times
- use correct redis client in delete stale items job
- clear all DB connections when a connection corrupted
- support Redlock 2.0
- configure Redis using the
redis=config option - support Redis Sentinel
- deprecate using of the
redis_servers=config option - use RedisClient instead of Redis
- the
ownertag was added to metrics
- use gem
redlockinstead of gemsidekiq-uniq-jobs
bin/rails g outbox:transportgenerator
bin/rails g outbox:inbox_itemandbin/rails g outbox:outbox_itemare now replaced withbin/rails g outbox:item
- merge symbolized keys in options
- correct merge ordering of default options
- changed field 'uuid' type to uuid
- rails generator for initial configuration
- rails generator for inbox/outbox item creation
- configured HttpHealthCheck with default port 5555
- sentry middlewares autoloading issue
- sentry tracing middlewares with optional Sentry dependency
- use wider context for job instrumentation
- Select transports by event name
- Use
camelizeinstead ofclassifyto constantize classes from a string
- Drop supporting of Rails 5.0
- Add
transportsconfig option
- Changed gem specifiers for sidekiq and sidekiq-unique-jobs from
~>to>=