iloc generates error while iterating through dataframes from a dictionary

Question

I have loaded my text files from a folder as Dataframes in to a dictionary with file name as the key. If I run a loop through my items in the dictionary to perform certain operations individually on my dataframes the iloc functions work for the first operation and then generates a single position index error.

 i = 0
 dict = {}
 for root, dirs, files in os.walk(root):
 for file in os.listdir(root):
    if file.endswith(".tlm-raw"):
        name = os.path.splitext(file)[0]
        dict[name] = pd.read_table(os.path.join(root,file),index_col=False,header=None, skiprows=53,engine='python')   

Then I run a loop through my dictionary and let's say I want the 13th column from each of my dataframe. The first time I call my dataframe and slice using Iloc it works.

for k in dict:
    df=pd.DataFrame.from_dict(dict[k])
    X=df.iloc[:,12]
    print(X)

Until then it works. But if I further wish to slice a column from it. Let's I want the 19th column. It raise the following error.

    y=df.iloc[:,18]
    **IndexError: single positional indexer is out-of-bounds**

My sincere request. What I actually want is load a files from a folder, store them as dataframes in a dictionary, then iterate through each of the dataframe, perform certain operations and store the modified dataframes in a new dictionary. What is the most feasible way of doing that?

Answer 1

I think you need first change dict to d , because dict is python kyeword:

 i = 0
 d = {}
 for root, dirs, files in os.walk(root):
 for file in os.listdir(root):
    if file.endswith(".tlm-raw"):
        name = os.path.splitext(file)[0]
        d[name] = pd.read_table(os.path.join(root,file),index_col=False,header=None, skiprows=53,engine='python')   

So d is dictionary of DataFrame s, so indexing is possible by:

d['file_name'].iloc[:, 9]

EDIT:

d1 = {}
for k, v in d.items()
  #modify v

  df1[k] = v

My edit: Here I am supposing we have some dataframes in dictionary d, I am iterating through each of them by the key(which in my case is the Filename). Performing n operation on each them. Let's suppose we're trying to slice columns between 31st-94th column.

d1={}
for k in d:
  df=pd.DataFrame.from_dict(d[k])
  Result=pd.DataFrame(df.iloc[:,30:93])
  d1[k]=pd.DataFrame(Result)

This way you have modified dictionary with same keys.

I think need:

d1={}
for k, v in d.items():
  d1[k] = v.iloc[:,30:93]

Or:

d1 = {k: v.iloc[:,30:93] for k, v in d.items()}