Append loop output in column pandas python

Question

I am working with the code below to append output to empty dataframe

image_data = pd.DataFrame()

for i in words:
    y = re.findall('{} ([^ ]*)'.format(re.escape(i)), data)
    x = y
    image_data = image_data.append(x, ignore_index = True)

i am getting output as below but i want

        0
0   30708
1      15
2    1800
0   19200
1      50
2    1180

What i want the output to be

        0    1       2
0   30708   15    1800
1   19200   50    1180

How can i make 3 rows to 3 columns every time the loop repeats.

Answer 1

If x is a list of values, use:

image_data = image_data.append([x], ignore_index = True)

to append all the values as a new row instead of appending a single element as a row. Look here for more details about the append method.

Answer 2

# replicating your dataframe
data = [30708, 15, 1800, 19200, 50, 1180]
df = pd.DataFrame(data)

you could first convert to numpy.ndarry in order to perform reshape():

vals = df[0].values.reshape(2, 3)

then back to pd.DataFrame if you really need it to be a pandas dataframe

df = pd.DataFrame(vals)

Answer 3

It perplexes me when you write x = y without doing any manipulation on x . Seems like a redundant operation. Another problem with your code is that image_data.append is slow since it has to copy the backing memory. Repeatedly calling it in a loop is a guarantee of performance bottleneck.

Try this instead:

# image_data starts as a list
image_data = []

for i in words:
    y = re.findall('{} ([^ ]*)'.format(re.escape(i)), data)
    image_data.append(y)

# And it ends as a DataFrame
image_data = pd.DataFrame(image_data)