Iterating through const unsigned char * containing “Extended ASCII” characters

Question

In my sqlite database for an iPhone app, I encode/compress a long array of integers (up to 5 digits) into a string using the extended ASCII character set to get it down to 2 characters. (In other words, I encode it using base150)

When getting it out of the database, sqlite3_column_text() returns the string as a "const unsigned char *". I can print this string using printf correctly (it shows even the ASCII characters over 128 properly) but when I try to iterate through it and access each character of the string individually to convert back into my integers, characters with ASCII values over 128 fail, because they're multibyte and it's only getting one byte (I think).

Example:

I have this string called encodedString which contains: svÖ)

unsigned char c = encodedString[0];
unsigned char d = encodedString[2];

printf("%c", c);  //outputs "s"
printf("%c", d);  //outputs "\303"
printf("%s", encodedString);  //outputs "svÖ)"

I've also tried wchar_t with the same results. I have gotten it to work using NSStrings, but it's very slow, and I'm doing this many thousands of times (NSMakeRange is the culprit according to the profiler), so I want it to be as fast as possible, hence C.

What's the trick to getting a single multibyte/extended ASCII character out of a string?

Answer 1

Rather than using a TEXT column, I would recommend using a BLOB column where the data contains an array of integers of whatever size you want to use (perhaps 16-bit unsigned).

You can use sqlite_column_bytes() to determine the size of the column, allowing for variable-length columns to be used.

This will avoid the complexity you are currently facing.