如何从 GAE Python 上的 cron 启动 mapreduce 作业

Question

I have mapreduce job defined in mapreduce.yaml:

mapreduce:
- name: JobName 
  mapper:
    input_reader: google.appengine.ext.mapreduce.input_readers.DatastoreInputReader
    handler: handler_name
    params:
    - name: entity_kind
      default: KindName

How to start it from cron? Is there some url that can run it?

Answer 1

You can start a mapreduce task from any kind of AppEngine handler using control.py

from mapreduce import control

mapreduce_id = control.start_map(
    "My Mapper",
    "main.my_mapper",
    "mapreduce.input_readers.DatastoreInputReader",
    {"entity_kind": "models.MyEntity"},
    shard_count=10)

Answer 2

Yes, if you look at the Getting Started page, it shows that you set the URL in your app.yaml :

handlers:
- url: /mapreduce(/.*)?
  script: mapreduce/main.py
  login: admin

You then can just cron it in the usual App Engine fashion, which in this example would be writing a cron.yaml like this:

cron:
- description: daily summary job
  url: /mapreduce
  schedule: every 24 hours