running hadoop on Google app engine?

Question

Is it possible to run map reduce jobs on Google app engine?

Any reference or tutorial would help

Thanks

Answer 1

Sort of.

You can't use the actual MapReduce framework - the architecture is too incompatible with AppEngine.

However, there is an equivalent system built specficially for GAE - appengine-mapreduce . That site is a bit confusing, as the first version of the code only supported mappers, without the subsequent reduce step - recently they released a version with full mapreduce support, but some of the documentation still referes to the earlier mapper-only one.

The best introduction is the GoogleIO talk from Mike Aizatskyi .

Answer 2

以下是完整的文档： https : //developers.google.com/appengine/docs/python/dataprocessing/overview

Answer 3

Saw this Google Cloud Platform advertisement:
Hadoop on Google Compute Engine virtual machines
https://cloud.google.com/solutions/hadoop

Answer 4

You cannot run Hadoop on Appengine (No filesystem access as well).

You may want to check AWS ElasticMapreduce. Its a cloud based platform for running Mapreduce jobs.

ElasticMapreduce