Inserting an array of 16-bit ints to MySQL, from C

Question

I have a large array of int16_t that I need to put into a MySQL database. I have constructed a query string by copying the values to a char buffer, but in the end it doesn't work (char is 1 byte, int16_t is two bytes.) The documentation I've found online about inserting arrays to MySQL does so with character arrays.

How can I insert this array of int16_t to a MySQL database?

Answer 1

I think you should look at the BLOB datatype. which can store binary blocks of data into a column. There are 4 different flavors, according to the size you need. Have a look at the following example, where images are stored in the database:

http://zetcode.com/db/mysqlc/

The really interesting part is about 80% down in the page, where an image is stored and later retrieved from the database. Take particular care about the mysql_real_escape_string() function, which makes sure that the data will be legal for the actual query. You should be able to send the int16 array immediately.

Edit... Here is the 'save' routing which works. It saves the ints array as a binary blob into the database. This program is NOT complete. Eg. it doesn't connect to the database,

#define SIZE 20000
int16_t ints[SIZE];

MYSQL *con;

int
ints_saved(void) // Returns 0 if there's an error
{
  char   *chunk, *st, *query;
  size_t ints_size = sizeof(ints), 
         st_len;
  int qlen;

  chunk = (char *)malloc(ints_size*2 + 1);
  // template for creating SQL
  st = "insert into Ints(Id, Data) values(1, '%s')"; 
  st_len = strlen(st);

  // Remove dangerous characters
  mysql_real_escape_string(con, chunk, (char *)ints, ints_size);

  query = (char *)malloc(st_len + 2*ints_size+1);
  // Construct the actual query here
  qlen = snprintf(query, st_len + 2*ints_size+1, st, chunk);

  printf("%s\n", query);
  // Execute the query
  if (mysql_real_query(con, query, qlen)) {
    printf("%s\n", mysql_error(con));
    return(0);
  };

  return(1);
}

Note that the database table was defined with two columns, Id(int), and Data(blob). You may have to modify blob, if you have other size requirements.

Warning: Nobody cried out yet, but do take care that this system might produce different results in different compilers. Some compilers may align 16-bit ints to 4-byte boundaries. So, though I have taken this into account with the sizeof() function, the actually saved record might occupy more space on some machines. If you want to avoid the problem, you can avoid this with 'packed' records - see the docs for your compiler.

The write process: The int16_t's are in an array. If I look at it as integers, I get:

17767 9158 -26519 18547 -9135 23807 -27574 22764 7977 31949 22714 -10325 16882 7931 -22045 -7866 124 25282 2132 10232 8987 -5656 -12825 17293

Looking at the same array as bytes (or characters), I see:

67 45 C6 23 69 98 73 48 51 DC FF 5C 4A 94 EC 58 29 1F CD 7C BA 58 AB D7

(The 67 45 is the first int value, 17767)

I can't write this, as byte 11 is 0x5C, which confuses the query string:

67 45 C6 23 69 98 73 48 51 DC FF 5C 4A 94 EC 58 29 1F CD 7C BA 58 AB D7  
                                 ^^

So, I call the mysql_escape function, and use the result to form the query:

67 45 c6 23 69 98 73 48 51 dc ff 5c 5c 4a 94 ec 58 29 1f cd 7c ba 58 ab 
                                 ^^^^^

Notice the extra 0x5c?

When I execute the query, I send the escaped string to MySQL, and MySQL immediately removes the escapes and writes the UN-ESCAPED string to the database. So, from now on, where are no more extra 0x5c characters. If you read it back, the the data is 'pure' again.

A really minimalist example. Say I want to write just one 0x5c to a table:

I can't write: insert into Ints(Id, Data) values (1, '\\') because MySQL would think I'm escaping the ' after the \\. (remember the '\\' is the 0x5c)

So I escape that \\: insert into Ints(Id, Data) values (1, '\\\\') . When MySQL receives this query, it inserts only one \\ though.

In the table only one character is inserted. And when read back, it produces only one character.