Cannot access Sqlite Database (of Firefox) using System.Data.Sqlite

Question

I wrote a small C#/.Net application that is able to read the cookies.sqlite file of Firefox. Since I upgraded to Firefox 4 my application is not able to open the database file:

Executing the line "connection.Open();" (in the code sample below) there will be an execption that says:

"File opened that is not a database file. file is encrypted or is not a database"

This is my program code:

class Program
{
    static void Main()
    {

        const string PATH_TO_DATABASE = @"C:\Users\Boris\Desktop\TEMP\cookies.sqlite";
        const string CONNECTION_STRING = @"Data Source=" + PATH_TO_DATABASE;

        if (!File.Exists(PATH_TO_DATABASE)) return;

        using (SQLiteConnection connection = new SQLiteConnection(CONNECTION_STRING))
        {
            connection.Open();
            using (SQLiteCommand command = new SQLiteCommand("SELECT id, name, host, path FROM moz_cookies", connection))
            {
                using (SQLiteDataReader read = command.ExecuteReader())
                {
                    while (read.Read())
                    {
                        string id = read[0].ToString();
                        string name = read[1].ToString();
                        string host = read[2].ToString();
                        string path = read[3].ToString();
                        Console.WriteLine("ID: " + id);
                        Console.WriteLine("Name: " + name);
                        Console.WriteLine("Host: " + host);
                        Console.WriteLine("Path: " + path);
                    }
                }
            }
        }
    }
}

I am using the .Net Wrapper DLL for Sqlite v. 3.6.23.1. The target framework of the application is .Net 2.0.

I was able to open the sqlite database without any problems using an application called SqliteExpert.

It would be great if anybody has an idea!

Regards, Boris

Answer 1

Firefox 4.0 uses SQLite version 3.7.4. (To see this, download SQLite Manager and run select sqlite_version(); in the "Execute SQL" tab.)

It appears that v3.7.4 creates databases that can't be read by v3.6.23.1. I can't find this explicitly stated in the release notes , but it's clear from comparing the cookies.sqlite file to the file format documentation that the format has changed. Specifically, according to the documentation, byte 19 (0x13) should be 1 , but that byte in the cookies.sqlite file is 2 . As per the documentation:

If a value greater than 1 is read by SQLite, then the library will refuse to open the database.

Like the "write version" described above, this field exists to facilitate some degree of forwards compatibility, in case it is ever required. If a version of SQLite created in the future uses a file format that may not be safely read by older SQLite versions, then this field will be set to a value greater than 1.

To read the database, you will need to use the latest version of SQLite; unfortunately System.Data.SQLite hasn't been updated in almost a year. The project has since been taken over by sqlite.org, but downloads are not currently available .

If you don't want to wait for sqlite.org to release a v3.7.x-based .NET wrapper, you could try downloading the source and manually upgrading SQLite.NET/SQLite.Interop/src/sqlite3.c to the latest amalgamation C source file (available here ).

EDIT: As noted by sdwilsh , Firefox 4.0 uses Write-Ahead Logging ; as the description of that new journal mode states:

Thus, if an older version of SQLite attempts to connect to an SQLite database that is operating in WAL mode, it will report an error along the lines of "file is encrypted or is not a database".

Answer 2

I had the same problem. Previously I used a Python script (with sqlite integration) to fetch up data from the cookies.sqlite and perms.sqlite files. The latter still works fine. Indeed, all of the other .sqlite files are readable ... except for cookies.sqlite and places.sqlite. Those two produce the "encrypted or is not a database" error.

Rather than hunt down an upgrade for Python (or sqlite3), I created a patch for the .sqlite files. This C-code changes the bytes at offset 18 and 19 from 2 to 1. I run this on Windows under Cygwin, but it should compile and run on Unix/Linux.

Warning: Don't do this to the original Firefox cookies.sqlite file. Instead, copy that to a temp file, then run the patch on the copy.

// ffpatch.c
// Edits the specified Firefox .sqlite file.
// Changes 0x0202 to 0x0101 at offset 18/19.
// BEFORE
//   0000000   S   Q   L   i   t   e       f   o   r   m   a   t       3  \0
//   0000020 004  \0 002 002  \0   @          \0  \0 005 034  \0  \0  \0   N
// AFTER
//   0000000   S   Q   L   i   t   e       f   o   r   m   a   t       3  \0
//   0000020 004  \0 001 001  \0   @          \0  \0 005 034  \0  \0  \0   N

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <fcntl.h>
#define PROGNAME "ffpatch"
#define MY_OFFSET 18

int main (int argc, char *argv[])
{
  int      fd;
  char     buf[2];
  off_t    offset;
  ssize_t  wsize;

  if (argc != 2)
  {
    fprintf(stderr, "usage: %s sqlite-file\n", PROGNAME);
    exit(1);
  }

  if ((fd = open(argv[1], O_RDWR)) == -1)
  {
    fprintf(stderr, "cannot open %s\n", argv[1]);
    exit(1);
  }

  if ((offset = lseek(fd, MY_OFFSET, SEEK_SET)) != MY_OFFSET)
  {
    fprintf(stderr, "lseek() failed\n");
    exit(1);
  }

  buf[0] = 1;
  buf[1] = 1;
  wsize = write(fd, buf, 2);   

  close(fd);

  exit(0);
}

Answer 3

Hello and thanks alot for your answers!

I made my application work this way:

1. I downloaded the sources of the current ADO.NET Connector for .NET 4.0 here: http://system.data.sqlite.org/index.html/timeline?r=trunk

Maybe you will have to logon on the website first using a anonymous user name and a capcha as passoword.

2. I obtained the precompiled version of the sqlite3.dll v.3.7.4 by downloading an installing this ADO.NET connector: http://www.devart.com/dotconnect/sqlite/ (You may use the ADO.NET connector libs as an replacement for connector from system.data.sqlite.org as well. I myself am only interested in the sqlite3.dll.)

3. After compiling the sources from system.data.sqlite.org I copied the resulting System.Data.Sqlite.dll and the sqlite3.dll to my applications output directory. Please note that both DLLs are compiled either for x86 or x64 machines.

Regards

Answer 4

I think I got your problem. Your db connection is OK. First of all you are using which version of dotnetFramework ? Accordingly you can download and use System.Data.SQLite.dll file to your references then your problem may be solved. I think you are using a System.Data.SQLite.dll file that is older one (doesn't match your version of dot net).