[英]MongoDB MapReduce, Date and % operator
I have a problem with a python-script witch i use to aggregate large collections into smaller pieces and have them grouped by a timestamp. 我有一个python脚本的问题,我用它来将大型集合聚合成较小的部分并按时间戳分组。
map = Code("function(number) {"
"emit({"
"ts : new Date(new Date((this.ts - (this.ts % (60 * number))) * 1000).setSeconds(0))"
"}, 1);"
"}")
reduce = Code("function(key, vals) {"
"var sum = 0;"
"for (var i in vals) {"
"sum += vals[i]"
"}"
"return sum;"
"}")
As you can see, it's a pretty simple MapReduce and the timestamp (ts) should be grouped by anny given number of minutes. 正如您所看到的,它是一个非常简单的MapReduce,时间戳(ts)应按给定的分钟数分组。 I've tested the Javascript here http://jsfiddle.net/QgMzK/1/ and it seems to work fine. 我在这里测试了Javascript http://jsfiddle.net/QgMzK/1/ ,它似乎工作正常。 But when i run it in Python all timestamps become ISODate("1970-01-01T00:00:00Z"). 但是当我在Python中运行时,所有时间戳都变为ISODate(“1970-01-01T00:00:00Z”)。
Any ideas? 有任何想法吗?
Your map function takes one parameter: number
, which, when called by map-reduce, will be set to null and after coertion (and some %
by zero) will make the date your map is returning ISODate("0NaN-NaN-NaNTNaN:NaN:NaNZ")
. 你的map函数有一个参数: number
,当map-reduce调用时,将被设置为null,而在coertion之后(和一些%
为零)将使你的地图返回的日期为ISODate("0NaN-NaN-NaNTNaN:NaN:NaNZ")
。 This after type conversion becomes datetime.datetime(1970, 1, 1, 0, 0)
. 此类型转换后变为datetime.datetime(1970, 1, 1, 0, 0)
。
Remove the parameter and it should work. 删除参数,它应该工作。
edit 编辑
To confirm that, try running this code: 要确认这一点,请尝试运行此代码:
from pymongo import Connection
from bson.code import Code
db = Connection().mr_test
for i in xrange(10):
db.things.insert({"x" : i})
map = Code("function(number) {"
"emit({"
"ts : number"
"}, 1);"
"}")
reduce = Code("function(key, vals) {"
"var sum = 0;"
"for (var i in vals) {"
"sum += vals[i]"
"}"
"return sum;"
"}")
result = db.things.map_reduce(map, reduce, "test_results")
for doc in result.find():
print doc
The result on my machine is : 我的机器上的结果是:
{u'_id': {u'ts': None}, u'value': 10.0}
Note that ts
is None
in the result, cause number
was not set when the mapping function executed. 请注意,结果中ts
为None
,执行映射功能时未设置原因number
。
edit 2 编辑2
AFAIK the only way to pass a parameter to map
is by using scope
optional parameter in map_reduce
, but then you'll have to remove it from map signature anyway. AFAIK将参数传递给map
的唯一方法是使用map_reduce
scope
可选参数,但是无论如何你都必须将其从map签名中删除。
So by changing your map to: 因此,将地图更改为:
map = Code("function() {"
"emit({"
"ts : new Date(new Date((this.ts - (this.ts % (60 * number))) * 1000).setSeconds(0))"
"}, 1);"
"}")
and by calling: 并致电:
db.whatever.map_reduce(map, reduce, "collection_name", scope = {"number" : the_value_your_function_needs})
you can get the result you want. 你可以得到你想要的结果。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.