[英]How would the Python pipes module do a 'cat'
I am trying to do this in python: 我试图在python中这样做:
cat foo | ssh me@xxxx hadoop fs -put - bar/foo
I have originally tried a check_call: 我最初尝试过check_call:
foo = 'foo'
subprocess.check_call(['cat', foo, '|','ssh',os.environ['USER']+'@'+hadoopGateway,'hadoop','fs','-put', '-', inputArgs.targetDir+'/'+foo])
which produces the error: 产生错误:
cat: invalid option -- 'p'
I have looked at the python pipes module documentation and played around with it in the shell, but I do not understand how to kick it off without an output file, like the example. 我已经查看了python管道模块文档,并在shell中使用它,但我不明白如何在没有输出文件的情况下启动它,如示例。
>>> t = pipes.Template()
>>> t.prepend('cat foo', '.-')
>>> t.append('hadoop fs -put - bar/foo', '-.') # what next
Clearly I am missing something. 显然我错过了一些东西。
You don't need cat
or a pipeline for this; 你不需要cat
或管道; all you need is to provide the file as standard input to the ssh
command. 您只需要提供文件作为ssh
命令的标准输入。 In shell, that would be 在shell中,那就是
ssh ${USER}@${hadoopGateway} hadoop fs -put - ${targetDir}/foo < foo
and with the Python subprocess module it's only a tiny bit more involved: 并且使用Python子进程模块,它只涉及更多:
foo='foo'
subprocess.check_call(['ssh',
os.environ['USER']+'@'+hadoopGateway,
'hadoop', 'fs', '-put', '-', inputArgs.targetDir+'/'+foo],
stdin=open(foo, 'r'))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.