Rails 4 multithreading and connection_pool problems

Question

I am trying to multithread my rails application but have run into some issues with connection_pool. I start a thread and execute a database query but it never seems like the database connections created in the thread are closed. This is my code:

class A
def self.foo
    Thread.new do
      nc1 = ActiveRecord::Base.connection_pool.connections.size
      nw = ""
      nc2 = ""

      ActiveRecord::Base.connection_pool.with_connection do |conns|
        nw = Person.count
        nc2 = ActiveRecord::Base.connection_pool.connections.size
      end

      nc3 = ActiveRecord::Base.connection_pool.connections.size
      puts "First there were #{nc1} connections, after things there were #{nc2} and now finally there are #{nc3} connections, there are #{nw} people in the db"  
    end
  end 
end

When I execute 10.times {A.foo} it gives me this output.

First there were 1 connections, after things there were 3 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 4 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 2 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db
First there were 1 connections, after things there were 5 and now finally there are 5 connections, there are 5325 people in the db

And the finally i run:

ActiveRecord::Base.connection_pool.connections.size
5

Now according to the documentation with_connection is supposed to take a block and execute it with a connection which it then closes, but according to my output it doesn't. I really don't get it.

Does anybody have any solutions or ideas why this might be happening? Is using "connection_pool.connections.size" the right way of checking how many connections there are?

Is there some other way to achieve multithreaded database queries in rails?

Answer 1

Ok so I just did not understant how connection_pool works. Connection pool holds the connections that have been opened since the pool was created whether they are being used or not. So the answer to my question is that you cannot (with the way it is implemented today) connection_pool to see which connections are being actively used. Instead I patched the connection pool itself to include this functionality.

If anyone is interested this is my patch placed in config/initializers/connection_pool_patch.rb:

module ActiveRecord
  module ConnectionAdapters
    class ConnectionPool
      def num_available
        @available.size
      end
    end
  end
end

module ActiveRecord
  module ConnectionAdapters
    class ConnectionPool
      class Queue
        def size
          @queue.size
        end
      end
    end
  end
end

It exposes the size of the private list @available that holds the available (ie not currently used but opened) connections in the connection pool. Usage = ActiveRecord::Base.connection_pool.num_available