未来一段时间在App Engine上执行任务的最有效方法是什么？

Question

I have an application on App Engine which is consuming some data. After parsing that data, it will know that it needs to execute something in a period of time - possibly not for a number of hours or weeks.

What is the best way to execute a piece of code after some arbitrary amount of time on App Engine?

I figured using Countdown Millis or EtaMillis from a TaskQueue would work, but haven't seen any evidence of anyone doing the same thing, especially for such long time frames.

Is that the best approach, or is there a better way?

Answer 1

如果您能够使用所有相关信息将对象持久保存在数据存储中以备将来处理（包括何时开始处理对象数据的时间），则可以让cron作业定期使用日期/时间范围过滤器查询数据存储并在适当的时间触发处理上述任何对象。

Answer 2

We successfully use TaskQueue's countdown parameter for sending emails to customers 7 days after they registered and for number of other needs.

Task queues is core/basic API/service and are pretty reliable - my opinion it's a best way to go with task queues ETA/countdown unless you:

• need ability programmatically see what is in the queue
• need ability programmatically delete certain task from the queue

Answer 3

I'm using the task queue as a scheduler. There is a 30 day max eta declared in QueueConstants and applied in QueueImpl .

  //Returns the maximum time into the future that a task may be scheduled.
  private static final long MAX_ETA_DELTA_MILLIS = 2592000000L;

1000ms * 60s * 60m * 24hr * 30days = 2592000000ms

 private long determineEta(TaskOptions taskOptions) {
Long etaMillis = taskOptions.getEtaMillis();
Long countdownMillis = taskOptions.getCountdownMillis();
if (etaMillis == null) {
  if (countdownMillis == null) {
    return currentTimeMillis();
  } else {
    if (countdownMillis > QueueConstants.getMaxEtaDeltaMillis()) {
      throw new IllegalArgumentException("ETA too far into the future");
    }
    if (countdownMillis < 0) {
      throw new IllegalArgumentException("Negative countdown is not allowed");
    }
    return currentTimeMillis() + countdownMillis;
  }
} else {
  if (countdownMillis == null) {
    if (etaMillis - currentTimeMillis() > QueueConstants.getMaxEtaDeltaMillis()) {
      throw new IllegalArgumentException("ETA too far into the future");
    }
    if (etaMillis < 0) {
      throw new IllegalArgumentException("Negative ETA is invalid");
    }
    return etaMillis;
  } else {
    throw new IllegalArgumentException(
        "Only one or neither of EtaMillis and CountdownMillis may be specified");
  }
}

}

Answer 4

I do the following:

1. Enqueue a task with a delay configured as you mention. Have the task processing change datastore entries in a known way (for example: set a flag).

2. Have a stragglers low frequency cron job, to perform any processing that has somehow been missed by an enqueued task (for example: an uncaught exception happened in the task).

For this to work, ensure that the processing called by the tasks and cron job are idempotent.

Enjoy?

Answer 5

I think taskQueue is a good strategy but has one big problem "If a push task is created successfully, it will eventually be deleted (at most seven days after the task successfully executes)." Source

I would instead use the datastore . here is one strategy you can take:

1. Insert a record into datastore once you completed "parsing that data".
2. Check the current date against the create/insert date to see how much time has passed by since your job was completed/started etc (clearly, you don't want to do every minute etc maybe do it once a day or every hour)
3. Execute the next task that you need to do as soon as condition in step 2 become passed your "arbitrary amount of time" you want.

Here is how you can add a record to data store...to get you started ..

    Entity parsDataHolder = new Entity("parsing_data_done", guestbookKey);
    parsDataHolder.setProperty("date", date);

DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
datastore.put(parsDataHolder)