How can i load JSON data from a text file and then add onto that data in python?

Question

Im looking for a way to save my json data with my program. Right now the code clears the data at every start but the data stays and lets me appended to it while im running the code. I know i have to load the data and write it into the file again before starting to append more data but everything i have tried has failed me. Heres my code:

load_data_profile = {}
load_data_profile['profile'] = []
def save_profile():
    load_data_profile['profile'].append({
        'profile_name': profile_name_entry.get(),
        'first_name': first_name_entry.get(),
        'last_name': last_name_entry.get(),
        'address': house_address_entry.get(),
        'address2': house_address2_entry.get(),
        'city': city_entry.get(),
        'country': country_entry.get(),
        'state': state_entry.get(),
        'zip': zip_entry.get(),
        'card_type': card_type_entry.get(),
        'card_number': card_number_entry.get(),
        'exp_month': card_exp_month_date_entry.get(),
        'exp_year': card_exp_year_date_entry.get(),
        'phone': phone_entry.get(),
        'email': email_entry.get()
    })
    with open('profiles.txt', 'w', encoding='utf-8') as outfile:
        json.dump(load_data_profile, outfile, indent=2)

This only writes the info to the file. I am leaving out parts that i tried due to the fact that i would need to retype it all in here. Any and all help is appreciated!

Answer 1

Use json.load() to get the profiles from the file first.

def save_profile():
    try:
        with open('profiles.txt', 'r', encoding='utf-8') as infile:
            load_data_profile = json.load(infile)
    except:
        load_data_profile = {'profile': []} # default when file can't be read
    load_data_profile['profile'].append({
        'profile_name': profile_name_entry.get(),
        'first_name': first_name_entry.get(),
        'last_name': last_name_entry.get(),
        'address': house_address_entry.get(),
        'address2': house_address2_entry.get(),
        'city': city_entry.get(),
        'country': country_entry.get(),
        'state': state_entry.get(),
        'zip': zip_entry.get(),
        'card_type': card_type_entry.get(),
        'card_number': card_number_entry.get(),
        'exp_month': card_exp_month_date_entry.get(),
        'exp_year': card_exp_year_date_entry.get(),
        'phone': phone_entry.get(),
        'email': email_entry.get()
    })
    with open('profiles.txt', 'w', encoding='utf-8') as outfile:
        json.dump(load_data_profile, outfile, indent=2)

Answer 2

One way is to take Barmar's approach: read it, append, and write it back. This is generally fine, but it does not scale very well because you must parse the entire file, load the entire thing into memory, and then write the entire thing back to disk. This is super noticeable as your list gets larger.

If you're feeling frisky, you can open the file in append mode, and add to the file incrementally with a newline ("\\n") delimiter, and then iterate over file.readlines() to parse each object (or even get a specific item, say profile #40, with file.readlines()[40] ).

Example:

import json

def save_profile():
    with open('profiles.txt', 'a') as file:
        profile = {
            "name": "John",
            "age": 32
        }
        json.dump(profile, file)
        file.write("\n")

def read_profiles():
    with open('profiles.txt', 'r') as file:
        profiles = file.readlines()
        print(f"{len(profiles)} profiles")
        for profile in profiles:
            print(profile)
            parsed = json.loads(profile)

save_profile()
read_profiles()

Output after running a few times:

> python3 test.py 

13 profiles
{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}

{"name": "John", "age": 32}