How to fix 'TypeError: Key must be a buffer at new Hmac (crypto.js:117:16)' when using ibm-cos-sdk

Question

I try to perform a Cloudant backup to IBM COS based on the example code provided for the @cloudant/couchbackup package and receive an error from the dependent Hmac package: "TypeError: Key must be a buffer at new Hmac (crypto.js:117:16)".

I adjusted the sample code to use my specific source (Cloudant) and target (COS) credentials and left the other parts of the sample code mainly unchanged. The result is identical for running it on Ubuntu (on Virtual Box) and on Windows 10. General access to both Cloudant and COS is proven with other test scripts.

My server.js:

const stream = require('stream');
const url = require('url');

const IbmCos = require('ibm-cos-sdk');
const couchbackup = require('@cloudant/couchbackup');
const debug = require('debug')('s3-backup');
const VError = require('verror').VError;

// COS properties 
const ibmAuthEndpointUrl = 'https://iam.ng.bluemix.net/oidc/token';
var backupBucket = process.env.COS_BUCKET;

var s3config = {
    endpoint: process.env.COS_ENDPOINT,
    apiKeyId: process.env.COS_API_KEY,
    ibmAuthEndpoint: ibmAuthEndpointUrl,
    serviceInstanceId: process.env.COS_RES_INST,
};

// Cloudant properties
const sourceUrl = process.env.COUCH_URL;
const sourceDbName = process.env.COUCH_DATABASE;
const sourceUrlExt = sourceUrl + '/' + sourceDbName;
const shallow = false;

// Create the COS client for the configuration:
var ibmS3Client = new IbmCos.S3(s3config);

const backupKeyPrefix = "A";
const backupKey = `${backupKeyPrefix}-${new Date().toISOString()}`;
debug(`Creating a new backup of ${s(sourceUrlExt)} at ${backupBucket}/${backupKey}...`);

bucketAccessible(ibmS3Client, backupBucket)
    .then(() => {
        return backupToS3(sourceUrlExt, ibmS3Client, backupBucket, backupKey, shallow);
    })
    .then(() => {
        debug('done.');
    })
    .catch((reason) => {
        debug(`Error: ${reason}`);
        process.exit(1);
    });

Main part of the code above and the functions bucketAccessible , backupToS3 , and s are from the example script .

Running server.js I would expect a text file exported from Cloudant to be stored in the COS. However, this is the error I get:

$ DEBUG=s3-backup node server.js
  s3-backup Creating a new backup of https://<mytenant>-bluemix.cloudant.com/test-48hg at dch-48h-backup01/A-2019-02-13T13:28:38.719Z... +0ms
  s3-backup Setting up S3 upload to ${s3Bucket}/${s3Key} +1s
  s3-backup Starting streaming data from ${sourceUrl} +1ms
  s3-backup Download from ${sourceUrl} complete. +717ms
  s3-backup S3 upload done +5ms
  s3-backup TypeError: Key must be a buffer
  s3-backup     at new Hmac (crypto.js:117:16)
  s3-backup     at Object.createHmac (crypto.js:643:10)
  s3-backup     at Object.hmac (C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\util.js:400:30)
  s3-backup     at Object.getSigningKey (C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\signers\v4_credentials.js:59:8)
  s3-backup     at V4.signature (C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\signers\v4.js:95:36)
  s3-backup     at V4.authorization (C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\signers\v4.js:90:36)
  s3-backup     at V4.addAuthorization (C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\signers\v4.js:32:12)
  s3-backup     at C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\event_listeners.js:199:18
  s3-backup     at finish (C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\config.js:308:7)
  s3-backup     at C:\Users\StefanVogel\Documents\GitHub\48hg-backup\node_modules\ibm-cos-sdk\lib\config.js:324:9 +0ms
  s3-backup Error: Error: S3 upload failed +2ms

I tried the same thing with using the file instead of the stream version of the example scripts, still with the same error.

Answer 1

Apparently, the ibm-cos-sdk was not installed properly. After running npm install ibm-cos-sdk it worked fine. Not sure why const IbmCos = require('ibm-cos-sdk'); is not sufficient. Maybe this is another question to someone knowledgeable of npm (not me certainly).

I also managed to run the same script using the aws-sdk with slight modification of instantiating the client.