Python从json文件读取记录并写入两个单独的json文件

Question

I have a twitter json file and I'm and trying to separate the English and French tweets into two separate files. I'm using Python 2.7 with the following code:

import json

with open('tweets.json', 'r') as f:

with open('english.json', 'w') as enF:
  with open('french.json', 'w') as frF:

    for line in f:
        tweet = json.loads(line)

        if tweet["lang"] == "en":
             json.dump(tweet, enF, sort_keys=True)
        elif tweet["lang"] == "fr":
             json.dump(tweet, frF, sort_keys=True)

This produces the two separate json files, one having English tweets and the other French, which I have checked. The original file has one tweet per line. The english.json and the french.json files have just a single line of all the tweets. Not sure if that will be a problem, not even confident that this is correct. So I passed the english.json again through this code (obviously I changed the name of the file) and it gives an error:

Traceback (most recent call last):
File "C:\Users\jack\Desktop\twitClean\j4.py", line 10, in <module>
tweet = json.loads(line)
File "C:\Python27\lib\json\__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 367, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 4926 - line 1 column 691991 (char 4925 - 691990)

I've been working on this for the past three days, and have come up with nothing. Can anyone please help and tell me what I'm doing wrong?

Answer 1

What about loading the json file as such

with open('tweets.json', 'r') as f:
    tweets_dict = json.load(f)

Then, given that the python-native representation of a json is a dictionnary, you can iter over it and build your french and english related dictionnaries as well. I mean, doing

fr_dict, en_dict, ot_dict = {}, {}, {}
for id_,tweet in tweets_dict.items():
    if tweet['lang'] == 'fr':
        fr_dict[id_] = tweet
    elif tweet['lang'] == 'en':
        en_dict[id_] = tweet
    else:
        ot_dict[id_] = tweet 

with open('french.json', 'w') as frF:
    json.dump(fr_dict, frF, sort_keys=True) 

with open('english.json', 'w') as enF:
    json.dump(en_dict, enF, sort_keys=True)

with open('other.json', 'w') as otF:
    json.dump(ot_dict, otF, sort_keys=True)

Answer 2

SOLVED: Unfortunately, being only a python hacker I cannot solve this using python. I'm sure there must be a way using python. So if someone else needs such a solution here it is.The solution I found was using jq as follows:
cat jsonfile | jq '. | select(.lang=="en")' > savefile

Obviously using this code the jsonfile has to be read twice as I need the English and French tweets in separate files.