How to override the retry configuration for Google Cloud Tasks Node.js Client

Question

I've been trying explore if there's a way to retry the createTask function . The reason being is because I keep on encountering deadline exceeded error from time to time like so:

Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded at Object.exports.createStatusError (/srv/node_modules/grpc/src/common.js:91:15) at Object.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:1204:28) at InterceptingListener._callNext (/srv/node_modules/grpc/src/client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:618:8) at callback (/srv/node_modules/grpc/src/client_interceptors.js:845:24)

Reading on the code behind the createTask function, I found out that the default timeout configuration was just 10 seconds.

At the moment, I have tried to extend the timeout to 30s (which I believe is the maximum) by doing this:

try {
  console.log("Sending task %j", task);
  const callOptions = {
    timeout: 30000
  };
  // Send create task request.
  const [response] = await client.createTask(request, callOptions);
  const name = response.name;
  console.log(`Created task ${name}`);
} catch (error) {
  console.error("CREATE_TASK_ERROR::", error);
}

And it appears that it works. However, I would also like to cover the case if the API wasn't able to respond within 30 seconds.

I've tried this code:

try {
  console.log("Sending task %j", task);
  const callOptions = {
    timeout: 2000, // I've set it to 2 seconds to be able to reproduce the deadline exceeded error easily
    retry: {
      initial_retry_delay_millis: 100,
      retry_delay_multiplier: 1.3,
      max_retry_delay_millis: 60000,
      initial_rpc_timeout_millis: 20000,
      rpc_timeout_multiplier: 1.0,
      max_rpc_timeout_millis: 20000,
      total_timeout_millis: 300000
    }
  };
  // Send create task request.
  const [response] = await client.createTask(request, callOptions);
  const name = response.name;
  console.log(`Created task ${name}`);
} catch (error) {
  console.error("CREATE_TASK_ERROR::", error);
}

But I don't see the createTask being retried. But based on the comment here , we should be able to override the default settings including retries.

What am I doing wrong? Please help.

Answer 1

It seems to that callOptions is wrong.

  const callOptions = {
    timeout: 2000, // I've set it to 2 seconds to be able to reproduce the deadline exceeded error easily
    retry: {
      backoffSettings: {
        initialRetryDelayMillis: 100,
        retryDelayMultiplier: 1.3,
        maxRetryDelayMillis: 60000,
        initialRpcTimeoutMillis: 20000,
        // rpc_timeout_multiplier: 1.0,  not exists
        maxRpcTimeoutMillis: 20000,
        totalTimeoutMillis: 300000
      }
    }
  };

See:

• https://googleapis.github.io/gax-nodejs/global.html#CallOptions
• https://googleapis.github.io/gax-nodejs/global.html#RetryOptions
• https://googleapis.github.io/gax-nodejs/global.html#BackoffSettings

But I think that to set retry parameters using cli is better.

See:

• https://cloud.google.com/tasks/docs/creating-appengine-queues#retry