Storing a file in postgres using node-postgres

Question

I'm trying to store a small file into a postgres db using the node-postgres module. I understand that I should use the bytea data type to do this. The problem I'm having is when I do some thing like:

fs.readFile path, (err, data) ->
    client.query 'UPDATE file_table SET file = $1 WHERE key = $2', [data, key], (e, result) ->
    ....

The contents of the file column in the db is: \\x and nothing is stored. If I change the data buffer to hex ie data.toString('hex') the file is stored but all formatting is lost when I read the file back out.

What is the correct way of storing a file into postgres using the node-postgres module?

Answer 1

The trick is to encode as hex and prepend the file with \\x. Reading it back out is indeed supported via parseByteA that returns a buffer:

https://github.com/brianc/node-postgres/blob/master/lib/textParsers.js

Here is what I did to read in an image from disk on postgres 9.2.2 and node.js 0.8.16 and node-postgres (npm package='pg') 0.11.2:

      fs.readFile(loc_on_disk, 'hex', function(err, imgData) {
        console.log('imgData',imgData);
        imgData = '\\x' + imgData;
        app.pgClient.query('insert into image_table (image) values ($1)',
                           [imgData],
                           function(err, writeResult) {
          console.log('err',err,'pg writeResult',writeResult);
        });
      });

and what I did to write it back out

app.get('/url/to/get/', function(req, res, next) {
  app.pgClient.query('select image from image_table limit 1',
                     function(err, readResult) {
    console.log('err',err,'pg readResult',readResult);
    fs.writeFile('/tmp/foo.jpg', readResult.rows[0].image);
    res.json(200, {success: true});
  });
});

Answer 2

I suggest you take a look as SQLGrid

From the readme:

• Efficient - Save space with automatic inline deduplication.
• Easy - Read and write files as if they were on disk with developer friendly APIs.
• Revisions - Keeps multiple versions of files.
• Byte-range Capable - Supports byte ranges to allow for streaming media.
• Consistent - Sha256 hashes are calculated when the file is written, and verified when read back out.
• Fast - Automatically caches hot data to save your database unneeded effort.