Django/MySQL - __istartswith not producing case-insensitive query

Question

I make use of generic views and I am attempting to query my MySQL db (utf8_bin collation) in a case insensitive manor to try to find all my song titles that start with a particular letter.

view.py

def tracks_by_title(request, starts_with):
    return object_list(
        request,
        queryset = Track.objects.filter(title__istartswith=starts_with),
        template_name = 'tlkmusic_base/titles_list.html',
        template_object_name = 'tracks',
        paginate_by = 25,
    )

and my

urls.py

urlpatterns = patterns('tlkmusic.apps.tlkmusic_base.views',
    (r'^titles/(?P<starts_with>\w)/$', tracks_by_title),
)

the query it produces according to the django debug toolbar is:

SELECT `tracks`.`id`, `tracks`.`url`, `tracks`.`artist`, `tracks`.`album`, `tracks`.`genre`, `tracks`.`year`, `tracks`.`title`, `tracks`.`comment`, `tracks`.`tracknumber`, `tracks`.`discnumber`, `tracks`.`bitrate`, `tracks`.`length`, `tracks`.`samplerate`, `tracks`.`filesize`, `tracks`.`createdate`, `tracks`.`modifydate` FROM `tracks` WHERE `tracks`.`title` LIKE a% LIMIT 1

specifically this line:

WHERE `tracks`.`title` LIKE a% LIMIT 1

Why is it not case-insensitive which is what I was expecting by using __istartswith?

I am using Django 1.1.1 on Ubuntu.

EDIT

Running SELECT * FROM tracks WHERE title LIKE 'a%' LIMIT 0 , 30 in phpmyadmin still returns case-sensitive results, changing my collation is something I want to avoid mostly because the database is maintained by Amarok and I don't know the results of changing the collation on it's end.

Answer 1

MySQL does not support ILIKE .

By default MySQL's LIKE compares strings case-insensitively.

Edit:
Thanks to the OP for providing additional information about the collation.
The current collation, utf8_bin is case-sensitive.
In contrast, utf8_general_ci is case-insensitive.

It's probably easiest to modify collation.
Something like this:

ALTER TABLE `mydb`.`mytable` 
MODIFY COLUMN `song_title` VARCHAR(254) 
CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL;

Answer 2

A solution, while not what I was hoping/expecting but still works is:

SELECT * FROM tracks WHERE title REGEXP BINARY '^(a|A)';

To use a REGEXP.

Which means changing my queryset string.

queryset = Track.objects.filter(title__regex=r'^(a|A)'),

Not optimal I am going to have to upper and lower the query string and then write an entirely new queryset for numbers and non-alphanumeric characters.

Answer 3

My solution was to "extend" the queryset overriding some behaviour (django 1.4):

# coding: UTF-8
from django.db.backends.mysql.base import DatabaseOperations as MySqlDatabaseOperations
from django.db.models.query import QuerySet
from django.db.models import sql
from django.db.models.sql.where import WhereNode


class ExtMySqlDatabaseOperations(MySqlDatabaseOperations):
    def lookup_cast(self, lookup_type):
        if lookup_type in ('iexact', 'icontains', 'istartswith', 'iendswith'):
            return "LOWER(%s)"
        return super(ExtMySqlDatabaseOperations, self).lookup_cast(lookup_type)

class ExtWhereNode(WhereNode):

    def make_atom(self, child, qn, connection):
        lvalue, lookup_type, value_annotation, params_or_value = child
        if type(connection.ops) in (MySqlDatabaseOperations, ExtMySqlDatabaseOperations):
            if lookup_type in ('iexact', 'icontains', 'istartswith', 'iendswith'):
                params_or_value = params_or_value.lower()
            connection.ops = ExtMySqlDatabaseOperations(connection)
        return WhereNode.make_atom(self, (lvalue, lookup_type, value_annotation, params_or_value), qn, connection)

class ExtQuerySet(QuerySet):
    def __init__(self, model=None, query=None, using=None):
        query = query or sql.Query(model, where = ExtWhereNode)
        super(ExtQuerySet, self).__init__(model = model, query = query, using = using)
        #self.query = self.query or sql.Query(model, where = ExtWhereNode)

def ext(qs):
    return ExtQuerySet(model=qs.model, using=qs._db)

Answer 4

God damnit me, I found another awkward way of doing it.

from django.db.models import Q

queryset = Track.objects.filter(Q(title__startswith=starts_with.upper) | Q(title__startswith=starts_with.lower)),