[英]Write data to Redis from PySpark
In Scala, we would write an RDD to Redis like this: 在Scala中,我们会像这样写一个RDD给Redis:
datardd.foreachPartition(iter => {
val r = new RedisClient("hosturl", 6379)
iter.foreach(i => {
val (str, it) = i
val map = it.toMap
r.hmset(str, map)
})
})
I tried doing this in PySpark like this: datardd.foreachPartition(storeToRedis)
, where function storeToRedis
is defined as: 我尝试在PySpark中这样做: datardd.foreachPartition(storeToRedis)
,其中函数storeToRedis
定义为:
def storeToRedis(x):
r = redis.StrictRedis(host = 'hosturl', port = 6379)
for i in x:
r.set(i[0], dict(i[1]))
It gives me this: 它给了我这个:
ImportError: ('No module named redis', function subimport at 0x47879b0, ('redis',)) ImportError:('没有名为redis的模块',函数subimport在0x47879b0,('redis',))
Of course, I have imported redis. 当然,我已经进口了redis。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.