Python Pandas GroupBy equivalent of If A and not B where clause in SQL

Question

I am using pandas groupby and was wondering how to implement the following:

1. Dataframes A and B have the same variable to index on, but A has 20 unique index values and B has 5.

2. I want to create a dataframe C that contains rows whose indices are present in A and not in B.

3. Assume that the 5 unique index values in B are all present in A. C in this case would have only those rows associated with index values in A and not in B (ie 15).

4. Using inner, outer, left and right do not do this (unless I misread something).

In SQL I might do this as where A.index <> (not equal) B.index

My Left handed solution:

a) get the respective index columns from each data set, say x and y.

def match(x,y,compareCol):

"""

x and y are series

compare col is the name to the series being returned .

It is the same name as the name of x and y in their respective dataframes"""

x = x.unique()

y = y.unique()

""" Need to compare arrays x.unique() returns arrays"""

new = []

for item in (x):

    if item not in y:

        new.append(item)

returnADataFrame = pa.DataFrame(pa.Series(new, name = compareCol))

return returnADataFrame

b) now do a left join on this on the data set A.

I am reasonably confident that my elementwise comparison is slow as a tortoise on weed with no motivation.

Answer 1

What about something like:

A.ix[A.index - B.index]

A.index - B.index is a set difference:

    In [30]: A.index
    Out[30]: Int64Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19], dtype=int64)

    In [31]: B.index
    Out[31]: Int64Index([  0,   1,   2,   3, 999], dtype=int64)

    In [32]: A.index - B.index
    Out[32]: Int64Index([ 4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19], dtype=int64)

    In [33]: B.index - A.index
    Out[33]: Int64Index([999], dtype=int64)