Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Ruby 3.2, 3.3 and Rails 7.1 #3

Merged
merged 9 commits into from
May 26, 2024
16 changes: 11 additions & 5 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,19 @@ jobs:
- '2.7'
- '3.0'
- '3.1'
- '3.2'
- '3.3'
activerecord-version:
- '6'
- '7'
- '6_0'
- '6_1'
- '7_0'
- '7_1'
exclude:
- # activerecord-7 doesn't support Ruby 2.6
ruby-version: '2.6'
activerecord-version: '7'
# activerecord-7 doesn't support Ruby 2.6
- ruby-version: '2.6'
activerecord-version: '7_0'
- ruby-version: '2.6'
activerecord-version: '7_1'

services:
mysql:
Expand Down
2 changes: 0 additions & 2 deletions gemfiles/activerecord_6.gemfile

This file was deleted.

2 changes: 2 additions & 0 deletions gemfiles/activerecord_6_0.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
eval_gemfile("../Gemfile")
gem "activerecord", "~> 6.0.0"
2 changes: 2 additions & 0 deletions gemfiles/activerecord_6_1.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
eval_gemfile("../Gemfile")
gem "activerecord", "~> 6.1.0"
2 changes: 0 additions & 2 deletions gemfiles/activerecord_7.gemfile

This file was deleted.

2 changes: 2 additions & 0 deletions gemfiles/activerecord_7_0.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
eval_gemfile("../Gemfile")
gem "activerecord", "~> 7.0.0"
2 changes: 2 additions & 0 deletions gemfiles/activerecord_7_1.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
eval_gemfile("../Gemfile")
gem "activerecord", "~> 7.1.0"
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,54 @@
module ActiveRecord
module DebugErrors
module DisplayMySQLInformation
# Rails 7 or later never calls ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter#execute
# cf. https://github.com/rails/rails/pull/43097
if ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter.private_method_defined?(:raw_execute)
method_name = :raw_execute
else
method_name = :execute
# For Rails 6.0, 6.1, 7.0.
if ActiveRecord.version < Gem::Version.new("7.1.0")
# Rails 7 or later never calls ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter#execute
# cf. https://github.com/rails/rails/pull/43097
if ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter.private_method_defined?(:raw_execute)
method_name = :raw_execute
else
method_name = :execute
end
define_method(method_name) do |*args, **kwargs|
super(*args, **kwargs)
rescue ActiveRecord::Deadlocked
handle_deadlocked
raise
rescue ActiveRecord::LockWaitTimeout
handle_lock_wait_timeout
raise
end
private method_name if method_name == :raw_execute
elsif ActiveRecord.version >= Gem::Version.new("7.1.0")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As commented in 5f9a8c9, there is a case when neither raw_execute nor execute is called in Rails 7.1.

I'm not sure if this way is the best, I tried to use translate_exception_class as a hook for Rails 7.1.

Copy link
Owner

@abicky abicky May 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the following approach is similar to the original one and more readable. What do you think about them?

diff --git a/lib/activerecord/debug_errors/ext/connection_adapters/abstract_mysql_adapter.rb b/lib/activerecord/debug_errors/ext/connection_adapters/abstract_mysql_adapter.rb
index 77f3f25..680e2e6 100644
--- a/lib/activerecord/debug_errors/ext/connection_adapters/abstract_mysql_adapter.rb
+++ b/lib/activerecord/debug_errors/ext/connection_adapters/abstract_mysql_adapter.rb
@@ -80,6 +80,12 @@ module ActiveRecord
   end
 end

+ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter.descendants.each do |adapter|
+  adapter.prepend(ActiveRecord::DebugErrors::DisplayMySQLInformation)
+end
 class ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter
-  prepend ActiveRecord::DebugErrors::DisplayMySQLInformation
+  def self.inherited(base)
+    super
+    base.prepend(ActiveRecord::DebugErrors::DisplayMySQLInformation)
+  end
 end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the code you presented is quite neat!

I think, I'm probably misunderstanding something big (and my implementation has based on the misunderstanding), but could you please explain to me why your code still works in Rails 7.1 for further studies? (and why the original code doesn't work.)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you can see below, ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter#raw_execute is not implemented, whereas the subclasses have the method raw_execute, so appending the module to the subclasses works expectedly:

% bundle exec irb
irb(main):001> require "active_record"
=> true
irb(main):002> require "active_record/connection_adapters/abstract_mysql_adapter"
=> true
irb(main):003> require "active_record/connection_adapters/mysql2_adapter"
=> true
irb(main):004> require "active_record/connection_adapters/trilogy_adapter"
=> true
irb(main):005> show_source ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter#raw_execute

From: /Users/arabiki/ghq/src/github.com/abicky/activerecord-debug_errors/gemfiles/vendor/bundle/ruby/3.3.0/gems/activerecord-7.1.3.3/lib/active_record/connection_adapters/abstract/database_statements.rb:530

        def raw_execute(sql, name, async: false, allow_retry: false, materialize_transactions: true)
          raise NotImplementedError
        end

=> nil
irb(main):006> show_source ActiveRecord::ConnectionAdapters::Mysql2Adapter#raw_execute

From: /Users/arabiki/ghq/src/github.com/abicky/activerecord-debug_errors/gemfiles/vendor/bundle/ruby/3.3.0/gems/activerecord-7.1.3.3/lib/active_record/connection_adapters/mysql2/database_statements.rb:96

          def raw_execute(sql, name, async: false, allow_retry: false, materialize_transactions: true)
            log(sql, name, async: async) do
              with_raw_connection(allow_retry: allow_retry, materialize_transactions: materialize_transactions) do |conn|
                sync_timezone_changes(conn)
                result = conn.query(sql)
                verified!
                handle_warnings(sql)
                result
              end
            end
          end

=> nil
irb(main):007> show_source ActiveRecord::ConnectionAdapters::TrilogyAdapter#raw_execute

From: /Users/arabiki/ghq/src/github.com/abicky/activerecord-debug_errors/gemfiles/vendor/bundle/ruby/3.3.0/gems/activerecord-7.1.3.3/lib/active_record/connection_adapters/trilogy/database_statements.rb:44

          def raw_execute(sql, name, async: false, allow_retry: false, materialize_transactions: true)
            log(sql, name, async: async) do
              with_raw_connection(allow_retry: allow_retry, materialize_transactions: materialize_transactions) do |conn|
                sync_timezone_changes(conn)
                result = conn.query(sql)
                verified!
                handle_warnings(sql)
                result
              end
            end
          end

=> nil

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I got it now. Thanks for your explanation!

# For Rails 7.1 or later. Override `ActiveRecord::ConnectionAdapters::AbstractAdapter#translate_exception_class`
# so that it obtains an error happened on query executions.
private def translate_exception_class(*args, **kwargs)
if args[0].is_a?(ActiveRecord::Deadlocked)
handle_deadlocked
elsif args[0].is_a?(ActiveRecord::LockWaitTimeout)
handle_lock_wait_timeout
end
super(*args, **kwargs)
end
end
define_method(method_name) do |*args, **kwargs|
super(*args, **kwargs)
rescue ActiveRecord::Deadlocked

private

def handle_deadlocked
if logger
logger.error "ActiveRecord::Deadlocked occurred:"
display_latest_detected_deadlock
end
raise
rescue ActiveRecord::LockWaitTimeout
end

def handle_lock_wait_timeout
if logger
logger.error "ActiveRecord::LockWaitTimeout occurred:"
display_transactions
display_processlist
end
raise
end
private method_name if method_name == :raw_execute

private

def display_latest_detected_deadlock
display_innodb_status_section("LATEST DETECTED DEADLOCK")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,16 @@

describe "#acquire_connection" do
it "displays connection owners and other threads" do
ActiveRecord::Base.connection_pool.checkout_timeout = 0.001 # no need to delay test suite by waiting the whole full default timeout

Thread.new { sleep 10 } # another thread

expect {
ActiveRecord::Base.connection # Ensure to acquire a connection
Array.new(ActiveRecord::Base.connection_pool.size) do
Thread.new do
ActiveRecord::Base.connection_pool.checkout(0.1)
ActiveRecord::Base.connection_pool.checkout
sleep 0.001
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connection pool handling in Rails 7.1 is so smarter that the existing test cannot cause ConnectionTimeoutError. To cause it, I tweaked the code.

ref: 1688bd0

Copy link
Owner

@abicky abicky May 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid that this test becomes flaky because some threads might end before the last thread calls checkout. How about the following changes?

diff --git a/spec/activerecord/debug_errors/ext/connection_adapters/connection_pool_spec.rb b/spec/activerecord/debug_errors/ext/connection_adapters/connection_pool_spec.rb
index 7a86bc7..6f2b1fe 100644
--- a/spec/activerecord/debug_errors/ext/connection_adapters/connection_pool_spec.rb
+++ b/spec/activerecord/debug_errors/ext/connection_adapters/connection_pool_spec.rb
@@ -20,11 +20,20 @@ RSpec.describe ActiveRecord::DebugErrors::DisplayConnectionOwners do
     it "displays connection owners and other threads" do
       Thread.new { sleep 10 } # another thread

+      mutex = Mutex.new
+      cv = ConditionVariable.new
+
       expect {
         ActiveRecord::Base.connection # Ensure to acquire a connection
         Array.new(ActiveRecord::Base.connection_pool.size) do
           Thread.new do
-            ActiveRecord::Base.connection_pool.checkout(0.1)
+            mutex.synchronize do
+              ActiveRecord::Base.connection_pool.checkout(0.1)
+              cv.wait(mutex, 1)
+            rescue
+              cv.broadcast
+              raise
+            end
           end
         end.each(&:join)
       }.to raise_error(ActiveRecord::ConnectionTimeoutError)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. And to be honest, I couldn't think of any other way to do it, so I'm glad to know the cool way you suggested it. Updated d2a4654

end
end.each(&:join)
}.to raise_error(ActiveRecord::ConnectionTimeoutError)
Expand Down
6 changes: 5 additions & 1 deletion spec/spec_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,11 @@
}

user_for_replica = 'activerecord-debug_errors'
ActiveRecord::Base.legacy_connection_handling = false
# For compatibility. Rails deprecated since 6.1 and removed this option since 7.1.
# https://github.com/rails/rails/pull/44827/commits/ad52c0a19714a1b87a7d0c626a8b364cf95414cf
if ActiveRecord::Base.respond_to?(:legacy_connection_handling)
ActiveRecord::Base.legacy_connection_handling = false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is simply broken in Rails 7.1.

end
ActiveRecord::Base.configurations = {
default_env: {
primary: base_db_config,
Expand Down
Loading