How to append nested lists to a list while running a python loop?

Question

I have a function that iterates through files makes some operations and at the end stores them as a list of lists. I did this in R and now I am trying to pass it to python. My current Code is as follows

def read_raw_spec(infile):
## to be filled later
rw_spec =  []
## loop over all files
for file in infile:
    ## Extract metadata
    with open(file, "r") as f:
        lines = f.readlines()[0:38]
        element = str(lines[11][(lines[11].find(":")+2):(len(lines[11])-1)])
        edge = str(lines[10][(lines[10].find(":")+2):(len(lines[10])-1)])
        E_zero_load = float(lines[12][(lines[12].find(":")+2):(len(lines[12])-1)])
        filename = str([os.path.basename(file)])
        rw_specdat = pd.read_csv(file,delim_whitespace = True, skiprows = 39,  engine = "python")
        rw_specdat = rw_specdat.loc[:,'#':'e']
        rw_specdat = (rw_specdat.rename(index=str, columns={"#":"Energy", "e":"raw_abs"}), E_zero_load)
        rw_spectemp = [{'name':filename, 'element' : element, 'edge' : edge, 'data' : list(rw_specdat)}]

        if rw_spec is None: 
            rw_spec = rw_spectemp
        else:
            rw_spec = rw_spec.append(rw_spectemp)
return rw_spec

Nevertheless, when I run the Code it only takes the last item and adds it to the list. When I would expect a several nested lists, from each file. This Approach works with Pandas but guess lists appending while Looping is different?

Answer 1

In python, list.append() is an operation and thus returns None .

if rw_spec is None: 
    rw_spec = rw_spectemp
else:
    rw_spec = rw_spec.append(rw_spectemp)
    # This makes rw_spec None

Presumably you have even number of files, which leads second last iteration to make rw_spec None, then at the last iteration, rw_spec gets assigned rw_spectemp of the last file, returned it.

As opposed to list , pandas.DataFrame.append returns appended DataFrame , not None . Thus reassigning works.

Simply changing the above chunk as below will fix the problem:

if rw_spec is None: 
    rw_spec = rw_spectemp
else:
    rw_spec.append(rw_spectemp)
    # This makes rw_spec None