os.system('echo "%s" | hadoop fs -put -f - /app/hdp/logs/json/a.json' %(json_string))
json_string=json.dumps({"a":"b"})
The file got created in HDFS but with wrong json format. The format it stored was {a:b}
with no doubles quotes which is not a proper json format.
What is wrong in this approach?
Try using the following:
import subprocess, json
json_string=json.dumps({"a":"b"})
proc = subprocess.Popen('echo "{0}" | hadoop fs -put -f - /app/hdp/logs/json/a.json'.format(json_string), shell=True)
The string should be formatted as "{"a":"b"}"
import subprocess, json
json_string=json.dumps({"a":"b"})
#json_string=json_string.replace('"','\"') try escaping quotes too
proc = subprocess.run('echo {0} | hadoop fs -put -f - /app/hdp/logs/json/a.json'.format(json_string), shell=True)
You will need to include escape characters
echo "{"a":"b"}" in shell terminal
Output: {a:b}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.