Which functions are called 'under the hood' when using Ecto.Repo in elixir

Question

I am trying to get a better understanding of Ecto adapters in elixir. I have begun trying to build my own adapter using Ecto.Adapters.Postgres as a base. This seemed like a good choice to start with as it is the default adapter used with Phoenix.

I can use my adapter in my own projects now by updating the the following line in my project's repo file...

defmodule UsingTestAdapter.Repo do
  use Ecto.Repo,
    otp_app: :using_test_adapter,
    adapter: TestAdapter  # <------ this line
end

At the moment it has the same functionality as the postgres adapter. I have been trying to edit some of the functions found in the Ecto.Adapters.Postgres.Connection and I have realised that they do not work quite how I expected.

The insert function for example does not actually use the params passed into Repo.insert .

To make this a little more clear imagine we have the following table, Comments ...

| id | comment |
| -- | ------- |

Now Repo.insert(%Comments{comment: "hi"}) is called.

I want to modify the adapter, so that it ignores the "hi" value that is passed in and instead inserts a comment of "I am the adapter, and I control this database. Hahaha (evil laugh)"...

| id | comment                                                            |
| -- | ------------------------------------------------------------------ |
| 1  | I am the adapter and I control this database. Hahaha (evil laugh)" |

However, the insert function does not appear to actually take the data to be stored as an argument.

My initial thought of what happened with ecto adapters was that when a user calls one of the repo functions it called the corresponding function in the Ecto.Adapters.Postgres.Connection module. This does appear to happen but other steps seem to be happening before this.

If anyone has a better understanding of the chain of functions that are called when Repo.insert (and any other Repo function) is called, please explain below.

Answer 1

I have had the time to look into this more deeply and feel that I now have a better understanding.

I'm going to list the steps, in order, that happen when a user calls Repo.insert in an elixir app.

Step 1. Call Repo.insert

AppName.Repo.insert(%AppName.Comments{comment: "hi"})

Step 2. AppName.Repo Module

defmodule AppName.Repo do
  use Ecto.Repo, otp_app: :app_name, adapter: adapter_name
end

(This is the default set up for a phoenix application)

The use Ecto.Repo allows for all the functions defined in that module to be used in the module that calls it. This means that when we call AppName.Repo.insert , it goes to our module, sees there is no function defined as insert, sees the use marco, checks that module, sees a function called insert and calls that function (this is not exactly how it works but I feel it explains it well enough).

Step 3. Ecto.Repo Module

def insert(struct, opts \\ []) do
  Ecto.Repo.Schema.insert(__MODULE__, struct, opts)
end

Where function is defined

Step 4. Ecto.Repo.Schema Module

4.1

# if a changeset was passed in
def insert(name, %Changeset{} = changeset, opts) when is_list(opts) do
  do_insert(name, changeset, opts)
end

# if a struct was passed in
# This will be called in this example
def insert(name, %{__struct__: _} = struct, opts) when is_list(opts) do
  do_insert(name, Ecto.Changeset.change(struct), opts)
end

Where function is defined

This step ensures that the data that is passed to do_insert in the the form of a changeset.

4.2

do_insert(name, Ecto.Changeset.change(struct), opts)

Not pasting whole function as it is very long. Where function is defined

This function does a fair amount of data manipulation and checks for errors. If all goes well it ends up calling the apply function

4.3

defp apply(changeset, adapter, action, args) do
  case apply(adapter, action, args) do # <---- Kernel.apply/3
    {:ok, values} ->
      {:ok, values}
    {:invalid, _} = constraints ->
      constraints
    {:error, :stale} ->
      opts = List.last(args)

      case Keyword.fetch(opts, :stale_error_field) do
        {:ok, stale_error_field} when is_atom(stale_error_field) ->
          stale_message = Keyword.get(opts, :stale_error_message, "is stale")
          changeset = Changeset.add_error(changeset, stale_error_field, stale_message, [stale: true])

          {:error, changeset}

        _other ->
          raise Ecto.StaleEntryError, struct: changeset.data, action: action
      end
  end
end

Where function is defined

This apply/4 function calls the Kernel.apply/3 function with the module , function name and arguments . In our case the module is AdapterName and the function is :insert .

This is where our adapter comes into play :D (finally).

Step 5. AdapterName

The apply/3 function call above takes us to our created adapter.

defmodule AdapterName do
  # Inherit all behaviour from Ecto.Adapters.SQL
  use Ecto.Adapters.SQL, driver: :postgrex, migration_lock: "FOR UPDATE"
end

There is no insert function defined in this module but as it is ' using ' Ecto.Adapters.SQL let's look at this module next.

Step 6. Ecto.Adapters.SQL Module

defmodule Ecto.Adapters.SQL do

...

      @conn __MODULE__.Connection

...

      @impl true
      def insert(adapter_meta, %{source: source, prefix: prefix}, params,
                 {kind, conflict_params, _} = on_conflict, returning, opts) do
        {fields, values} = :lists.unzip(params)
        sql = @conn.insert(prefix, source, fields, [fields], on_conflict, returning)
        Ecto.Adapters.SQL.struct(adapter_meta, @conn, sql, :insert, source, [], values ++ conflict_params, kind, returning, opts)
      end

...
end

@conn is defined as a module attribute and is just the current calling module ( MODULE ) + .Connection.

The calling module, as discussed in point 5 is AdapterName

That means in the insert function, the following line...

@conn.insert(prefix, source, fields, [fields], on_conflict, returning)

is the same as

AdapterName.Connection.insert(prefix, source, fields, [fields], on_conflict, returning)

As our adapter is just the same as the postgres adapter , it takes us to this next function.

Step 7. AdapterName.Connection

def insert(prefix, table, header, rows, on_conflict, returning) do
  values =
    if header == [] do
      [" VALUES " | intersperse_map(rows, ?,, fn _ -> "(DEFAULT)" end)]
    else
      [?\s, ?(, intersperse_map(header, ?,, &quote_name/1), ") VALUES " | insert_all(rows, 1)]
    end

  ["INSERT INTO ", quote_table(prefix, table), insert_as(on_conflict),
   values, on_conflict(on_conflict, header) | returning(returning)]
end

Where the function is defined

To save some text in an answer that is already too long, I won't go into too much detail. This function doesn't actually take the params we passed into Repo.insert (way back in set one).

If we want to edit the params we need to do so in the AdapterName module. We need to define our own insert function so that it no longer calls the insert function defined in step 6.

Step 8. AdapterName - Define our own insert.

For the sake of simplicity, we are going to just copy the insert defined in step 6 into our AdapterName module. Then we can modify that function to update params as we see fit.

If we do this we end up with a function like...

  def insert(adapter_meta, %{source: source, prefix: prefix}, params, on_conflict, returning, opts) do
    Keyword.replace!(params, :comment, "I am the adapter and I control this database. Hahaha (evil laugh)") # <---- changing the comment like we wanted :D

    {kind, conflict_params, _} = on_conflict
    {fields, values} = :lists.unzip(params)
    sql = @conn.insert(prefix, source, fields, [fields], on_conflict, returning)
    Ecto.Adapters.SQL.struct(adapter_meta, @conn, sql, :insert, source, [], values ++ conflict_params, kind, returning, opts)
  end

This now inserts a different value as we originally wanted.

Hopefully someone finds this helpful.