Rails creating schema_migrations - Mysql2::Error: Specified key was too long

Question

I am using Rails 3.2.6 and Mysql 6.0.9 (but I have exactly the same error on MySQL 5.2.25)

When I create new database ( rake db:create ) and then when I try to load the schema ( rake schema:load ) I get this error:

Mysql2::Error: Specified key was too long; max key length is 767 bytes: CREATE UNIQUE INDEX `unique_schema_migrations` ON `schema_migrations` (`version`)

After hours and hours of research I found these solutions:

1. Change MySQL variable innodb_large_prefix to true (or ON)

This didn't work. I tried it on my Linux server, my Mac and even on Windows - it just doesn't work.

2. Monkeypatch ActiveRecord::SchemaMigration.create_table

I do not need the version column to be 255 long (when it is UTF-8, then it takes 4*255 = 1020 bytes and exceeds the MySQL limit of 767 byte for keys). I do not need it to be UTF-8 either, but all other tables in the DB are UTF-8 and I have set utf8_czech_ci to be the default collation.

The method that actually creates the schema_migrations table looks like this:

def self.create_table
  unless connection.table_exists?(table_name)
    connection.create_table(table_name, :id => false) do |t|
      t.column :version, :string, :null => false
    end
    connection.add_index table_name, :version, :unique => true, :name => index_name
  end
end

You can read the whole file on Github rails/rails

So I tried to add :limit => 100 to the t.column statement, but I did not succeed with this solution either. The problem is that I cannot make this patch load when the originial is already in place. In other words - my patch loads before ActiveRecord::SchemaMigration so it is overwritten.

When I put this in config/initializers/patches/schema_migration.rb :

require 'active_record/scoping/default'
require 'active_record/scoping/named'
require 'active_record/base'

module ActiveRecord
  class SchemaMigration < ActiveRecord::Base
    def self.create_table
      unless connection.table_exists?(table_name)
        connection.create_table(table_name, :id => false) do |t|
          t.column :version, :string, :null => false, :limit => 100
        end
        connection.add_index table_name, :version, :unique => true, :name => index_name
      end
    end
  end
end

It is successfully loaded, but the it is overwritten when the original ActiveRecord::SchemaMigration is loaded.

I tried to mess up with ActiveSupport.on_load(:active_record) but that doesn't seem to work either.

Is there a way to load this file after the originial ActiveRecord::SchemaMigration is in place and make this patch work?

Do you have any suggestions? I can clarify any part of this question, if it makes no sense to you. Just ask me. I've been stuck with this for too long.

Answer 1

767 key should work. Make sure you use utf8 encoding, and not utf16 . I had same problem, and my mistake was that I accidently created utf16 database

Answer 2

I suggest you to drop your database and recreate a new one with the following instructions:

 mysql -u root -p -e "CREATE DATABASE {DB_NAME} DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;"

Answer 3

I have always this error, when it was not this error:

Mysql2::Error: Specified key was too long; max key length is 767 bytes:

it was this other:

Mysql2::Error: Specified key was too long; max key length is 3072 bytes

so I as suggested here I create a new database with this settings using docker and the official mysql

docker run -p 3306:3306 --name mysql-name -e MYSQL_ROOT_PASSWORD=name -d mysql:5.6 --innodb-large-prefix=1 --innodb-file-format=barracuda --innodb-file-per-table=1

the inportant thing here is not docker is that the database ahs teh configuration

innodb-large-prefix=true innodb-file-format=barracuda innodb-file-per-table=true

and finally the database.yml as follows change utf8 latin1 the sswedish is because in the server I saw that:

development:
  host: 192.168.99.100
  database: name
  username: root
  password: root
  adapter: mysql2
  charset: latin1
  encoding: latin1
  collation: latin1_swedish_ci


test:
  host: 192.168.99.100
  database: name_test
  username: root
  password: root
  adapter: mysql2
  charset: latin1
  encoding: latin1
  collation: latin1_swedish_ci

production:
  host: 192.168.99.100
  database: name
  username: root
  password: root
  adapter: mysql2
  charset: latin1
  encoding: latin1
  collation: latin1_swedish_ci

after that works

bundle exec rake db:create
bundle exec rake db:migrate

I did it with the monkey patch and without the monkey patch suggested here and it works

config/initializers/ar_innodb_row_format.rb

ActiveSupport.on_load :active_record do
  module ActiveRecord::ConnectionAdapters   
    class AbstractMysqlAdapter 
      def create_table_with_innodb_row_format(table_name, options = {}) 
        table_options = options.reverse_merge(:options => 'ENGINE=InnoDB ROW_FORMAT=DYNAMIC')

        create_table_without_innodb_row_format(table_name, table_options) do |td|
         yield td if block_given? 
        end
      end
      alias_method_chain :create_table, :innodb_row_format 
    end
  end
end

Answer 4

I have the same problem with a column named version for varchar of length 2000

class AddVersionToUsers < ActiveRecord::Migration
  def change
    add_column :users, :version, :string, limit:2000
    add_index  :users, :version
  end
end

I was using this latin 1 1 character 1 byte, but now I want to use utf8mb4 1 character 4 bytes.

Configuring your databse like this you can get index until 3072 bytes:

docker run -p 3309:3306 --name test-mariadb -e MYSQL_ROOT_PASSWORD=Cal1mero. -d mariadb:10.2 --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --innodb-large-prefix=1 --innodb-file-format=barracuda --innodb-file-per-table=1 --innodb-strict-mode=1 --innodb-default-row-format=dynamic

this is enough for latin_1, (will be 2000 bytes), but for utf8mb4 it will be 8000 bytes. In this keys you have some options

Add a column named hash_version and implement the index on that column.

Consistent String#hash based only on the string's content

Make the string shorter, it should work, but depernds on your needs

or use fulltext in your migrations, like this:

class AddVersionToUsers < ActiveRecord::Migration
  def change
    add_column :users, :version, :string, limit:2000
    add_index  :users, :version, type: :fulltext
  end
end

references:

• https://mensfeld.pl/2016/06/ruby-on-rails-mysql2error-incorrect-string-value-and-specified-key-was-too-long/
• https://codex.wordpress.org/Converting_Database_Character_Sets
• https://dev.mysql.com/doc/refman/8.0/en/innodb-restrictions.html
• https://docs.oracle.com/cd/E17952_01/mysql-5.7-en/innodb-restrictions.html
• https://dba.stackexchange.com/questions/35821/possible-index-on-a-varchar-field-in-mysql