python pandas changes column value when setting with enlargement

Question

I've boiled my problem down to this example. I am running Python 2.7 on Windows 7 and using pandas 0.17.1.

>>> import pandas as pd
>>> print pd.__version__
0.17.1
>>> x = pd.DataFrame({"a":["james"], "b":[True], "c":["john"]})
>>> x
       a     b     c
0  james  True  john
>>> x.loc[1] = None
>>> x
       a   b     c
0  james   1  john     ##<< THE TYPE HAS CHANGED
1    NaN NaN   NaN
>>> x.loc[1,"a"] = "james1"
>>> x.loc[1,"c"] = "john1"
>>> x.loc[1,"b"] = True
>>> x
        a     b      c
0   james     1   john
1  james1  True  john1 ## << BUT THE NEXT INSERT WAS OK
>>> x.loc[2] = None    ## << AND THIS DOESN'T REPLICATE THE ISSUE
>>> x
        a     b      c
0   james     1   john
1  james1  True  john1
2     NaN   NaN    NaN

In the above I try to add an extra row to the DF. I fill the row "by hand", so to speak, because in my actual situation I don't know all the column values yet and I might need to add an extra column so I fill in the known data first.

The first enlargement however changes the True to a 1 . Subsequent enlargements don't have this issue. But this does mean eventually I have a DF with a column of 0's, 1's and bools, which is annoying.

Does anyone know why this is occurring?

Interestingly, if I add the complete row at a time in the above example, the problem doesn't occur...

>>> x = pd.DataFrame({"a":["james"], "b":[True], "c":["john"]})
>>> x
       a     b     c
0  james  True  john
>>> x.loc[1] = {"a": "james", "b" : False, "c" : "henry"}
>>> x
       a      b      c
0  james   True   john
1  james  False  henry

Answer 1

Just in case anyone comes across the same problem. I reported the issue and the response was this:

This is as expected. np.nan is the missing value indicator, not None. We very rarely allow native None; non-performant as its a python object.

So it looks like what I was trying to do just wouldn't work. The way I have worked around it is to contruct a dictionary with the values I have and then add keys mapping to None for all the other columns in the new row.