DB Ownership process error running phoenix test in containerized elixir 1.6.1

Question

I have an umbrella project compose by:

• gateway - Phoenix application
• core - business model layer notifications a dedicated app for delivering sms, email, etc… users
• user management, role system, and authentication.

The three components are connected via AMQP, so they can send and receive messages between them.

We are using Docker, and drone.io hosted on google cloud's kubernetes engine. So, what is happening is that randomly the application raises the following exception while running my tests:

{\"status\":\"error\",\"message\":\"%DBConnection.OwnershipError{message: \\\"cannot find ownership process for #PID<0.716.0>.\\\\n\\\\nWhen using ownership, you must manage connections in one\\\\nof the four ways:\\\\n\\\\n* By explicitly checking out a connection\\\\n* By explicitly allowing a spawned process\\\\n* By running the pool in shared mode\\\\n* By using :caller option with allowed process\\\\n\\\\nThe first two options require every new process to explicitly\\\\ncheck a connection out or be allowed by calling checkout or\\\\nallow respectively.\\\\n\\\\nThe third option requires a {:shared, pid} mode to be set.\\\\nIf using shared mode in tests, make sure your tests are not\\\\nasync.\\\\n\\\\nThe fourth option requires [caller: pid] to be used when\\\\nchecking out a connection from the pool. The caller process\\\\nshould already be allowed on a connection.\\\\n\\\\nIf you are reading this error, it means you have not done one\\\\nof the steps above or that the owner process has crashed.\\\\n\\\\nSee Ecto.Adapters.SQL.Sandbox docs for more information.\\\"}\",\"code\":0}"

It's a Json since we exchange amqp messages in our tests, for instance:

# Given the module FindUserByEmail
def Users.Services.FindUserByEmail do  
    use GenAMQP.Server, event: "find_user_by_email", conn_name: 
        Application.get_env(:gen_amqp, :conn_name)

    alias User.Repo
    alias User.Models.User, as: UserModel

    def execute(payload) do
      %{ "email" => email } = Poison.decode!(payload)

      resp =
          case Repo.get_by(UserModel, email: email) do
              %UserModel{} = user -> %{status: :ok, response: user}
              nil -> ErrorHelper.err(2011)
              true -> ErrorHelper.err(2012)
              {:error, %Ecto.Changeset{} = changeset} -> ViewHelper.translate_errors(changeset)
          end

      {:reply, Poison.encode!(resp)}
  end
end

# and the test
defmodule Users.Services.FindUserByEmailTest do
  use User.ModelCase
  alias GenAMQP.Client

  def execute(payload) do
      resp = Client.call_with_conn(@conn_name, "find_user_by_email", payload)
      data = Poison.decode!(resp, keys: :atoms)

      assert data.status == "ok"
  end
end

Following details my .drone.yaml file:

pipeline:
unit-tests:
image: bitwalker/alpine-elixir-phoenix:1.6.1
environment:
    RABBITCONN: amqp://user:pass@localhost:0000/unit_testing
    DATABASE_URL: ecto://username:password@postgres/testing
commands:
    - mix local.hex --force && mix local.rebar --force
    - mix deps.get
    - mix compile --force
    - mix test

mix.exs file in every app contains the following aliases

defp aliases do
    [
        "test": ["ecto.create", "ecto.migrate", "test"]
    ]
end

All ours model_case files contain this configuration:

setup tags do
    :ok = Sandbox.checkout(User.Repo)

    unless tags[:async] do
        Sandbox.mode(User.Repo, {:shared, self()})
    end

    :ok
end

How can I debug this? it only occurs while testing code inside the container. Would the issue be related to container's resources?

Any tip or hint would be appreciated.

Answer 1

Try setting onwership_timeout and timeout to a large numbers in your config/tests.exs

config :app_name, User.Repo,
adapter: Ecto.Adapters.Postgres,
  username: ...,
  password: ...,
  database: ...,
  hostname: ...,
  pool: Ecto.Adapters.SQL.Sandbox,
  timeout: 120_000, # i think the default is 5000
  pool_timeout: 120_000,
  ownership_timeout: 120_000 #i think the default is 5000