我的程序有时会在json文件中的数据末尾写一个额外的]或}？

Question

I have written a note taking tool for myself as my first program. Its actually working really well for the most part however sometimes the program will write an extra ] or } at the end of the list or dict stored inside of said json file.

It doesn't happen often and I think it is only happening when I am writing new lines of code or changing existing lines that read/write to said files. I am not 100% sure but that is what it looks like.

For example I have a single list stored in a file and I use the indent="" flag to make sure as it writes the files its a little more readable for me if I ever have to edit said files. Sometimes when running my program after changing up some code or adding code I get an error stating a file has "extra data" in it. The error looks something like this:

    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 6 column 2 (char 5791)

and the cause of the error would be something like this:

[
"Help",
"DataTypes",
"test",
"Variables",
]] # the error would be cause by this extra ] at the end of the list

What I don't understand is why does the program sometimes add and extra ] or } at the end of the data in my json files?

Is there something I am doing wrong when I open the file or dump to the file?

Here are some sections of code I have that are used to open files and dump to files:

path = "./NotesKeys/"
notebook = dict()
currentWorkingLib = ""
currentWorkingKeys = ""
#~~~~~~~~~~~~~~~~~~~< USE TO open all files in Directory >~~~~~~~~~~~~~~~~~~~
with open("%s%s"%(path,"list_of_all_filenames"), "r") as listall:
    list_of_all_filenames = json.load(listall)

def openAllFiles(event=None):
    global path
    for filename in os.listdir(path):
        with open(path+filename, "r+") as f:
            notebook[filename] = json.load(f)
openAllFiles()

And here is how I am updating the data in the file. Just ignore the e1Current, e1allcase, e2Current they are used to keep the format of the users input for filenames (dict key) lower case in the dictionaries where the notes are stored and maintain the case the user imputed for a display list. This should not be related to the file read write issue.:

Edit: removed unrelated code per commenters request.

#~~~~~~~~~~~~~~~~~~~< UPDATE selected_notes! >~~~~~~~~~~~~~~~~~~~

            dict_to_be_updated = notebook[currentWorkingLib]
            dict_to_be_updated[e1Current] = e2Current
            with open("%s%s"%(path,currentWorkingLib),"r+") as working_temp_var:
                json.dump(dict_to_be_updated, working_temp_var, indent = "")

I am aware of how to open a file and use the data and how to dump data to said file and update the content loaded in the variables of the program based off the newly dumped data.

Am I missing something important during this process? Should I be doing something to ensure data integrity in the json files?

Answer 1

You are opening files in read-write mode, r+ :

with open("%s%s"%(path,currentWorkingLib),"r+") as working_temp_var:

This means you'll be writing to a file that already has data in it , and sometimes the existing data is longer than what you are now writing to the file. That means you'll end up with some trailing data at the end.

You can see this by writing a shorter demo string to a file, then using r+ to write less data to the same file, then reading again:

>>> with open('/tmp/demo', 'w') as init:
...     init.write('The quick brown fox jumps over the lazy dog\n')
...
44
>>> with open('/tmp/demo', 'r+') as readwrite:
...     readwrite.write("Monty Python's flying circus\n")
...
29
>>> with open('/tmp/demo', 'r') as result:
...     print(result.read())
...
Monty Python's flying circus
r the lazy dog

Don't do this. Use w write mode so the file is truncated first:

with open("%s%s"%(path,currentWorkingLib), "w") as working_temp_var:

This ensures your file is cut back to size 0 before you write a new JSON document.