Append columns to Pandas Dataframe

Question

Pretty stuck with appending extra columns at a Pandas Dataframe

As a test I setup the code below:

def dataSerialize(TagModel):

    test = """Month; Col_1; Col_2
            1; 0,121; 0,123;
            2; 0,231; 0,356;
            3; 0,150; 0,156;
            4; 0,264; 0,426;
            4; 0,264; 0,426"""

    df = pd.read_csv(StringIO(test), decimal=',',sep=';')
    df = df.set_index('Month')
    df['Test'] = ['3','6','8','78','10']
    df['Test2'] = ['3','6','8','78','10']
    df.to_csv('SerializeTest.csv',sep=';')

In general this seems to work good. Only in the.csv file, my data from "Col_1" moved to the data from "Month". Data from "Col_2" moved to "Col_1". "Col_2" stays empty. The new columns "Test" and "Test2" where nicely added to it. Why the existing data moves to the left? I believe it is something very simple/stupid but it bothers me for hours. Some help would be appreciated very much.

enter image description here

Answer 1

The problem are the delimiters ; at the end of the lines 2-5 in the test string. They trick the parser into thinking that there is an additional column (without content).

If you remove them, the result is as expected (I only print to screen, don't write to file):

import pandas as pd
from io import StringIO

test = """Month; Col_1; Col_2
            1; 0,121; 0,123
            2; 0,231; 0,356
            3; 0,150; 0,156
            4; 0,264; 0,426
            4; 0,264; 0,426"""

df = pd.read_csv(StringIO(test), decimal=',',sep=';')
df = df.set_index('Month')

df['Test'] = ['3','6','8','78','10']
df['Test2'] = ['3','6','8','78','10']

print(df)

# output
        Col_1   Col_2 Test Test2
Month                           
1       0.121   0.123    3     3
2       0.231   0.356    6     6
3       0.150   0.156    8     8
4       0.264   0.426   78    78
4       0.264   0.426   10    10