Selecting rows from a Dataframe based on values from multiple columns in pandas

Question

This question is very related to these two questions another and thisone , and I'll even use the example from the very helpful accepted solution on that question. Here's the example from the accepted solution (credit to unutbu):

import pandas as pd
import numpy as np
df = pd.DataFrame({'A': 'foo bar foo bar foo bar foo foo'.split(),
                   'B': 'one one two three two two one three'.split(),
                   'C': np.arange(8), 'D': np.arange(8) * 2})
print(df)
#      A      B  C   D
# 0  foo    one  0   0
# 1  bar    one  1   2
# 2  foo    two  2   4
# 3  bar  three  3   6
# 4  foo    two  4   8
# 5  bar    two  5  10
# 6  foo    one  6  12
# 7  foo  three  7  14

print(df.loc[df['A'] == 'foo'])

yields

     A      B  C   D
0  foo    one  0   0
2  foo    two  2   4
4  foo    two  4   8
6  foo    one  6  12
7  foo  three  7  14

But I want to have all rows of A and only the arrows in B that have 'two' in them. My attempt at it is to try

print(df.loc[df['A']) & df['B'] == 'two'])

This does not work, unfortunately. Can anybody suggest a way to implement something like this? it would be of a great help if the solution is somewhat general where for example column A doesn't have the same value which is 'foo' but has different values and you still want the whole column.

Answer 1

I think I understand your modified question. After sub-selecting on a condition of B , then you can select the columns you want, such as:

In [1]: df.loc[df.B =='two'][['A', 'B']]
Out[1]: 
     A    B
2  foo  two
4  foo  two
5  bar  two

For example, if I wanted to concatenate all the string of column A, for which column B had value 'two' , then I could do:

In [2]: df.loc[df.B =='two'].A.sum()  # <-- use .mean() for your quarterly data
Out[2]: 'foofoobar'

You could also groupby the values of column B and get such a concatenation result for every different B-group from one expression:

In [3]: df.groupby('B').apply(lambda x: x.A.sum())
Out[3]: 
B
one      foobarfoo
three       barfoo
two      foofoobar
dtype: object

To filter on A and B use numpy.logical_and :

In [1]: df.loc[np.logical_and(df.A == 'foo', df.B == 'two')]
Out[1]: 
     A    B  C  D
2  foo  two  2  4
4  foo  two  4  8

Answer 2

Row subsetting: Isn't this you are looking for ?

df.loc[(df['A'] == 'foo') & (df['B'] == 'two')]

   A   B  C D
2 foo two 2 4
4 foo two 4 8

You can also add .reset_index() at the end to initialize indexes from zero.

Answer 3

Easy , if you do

     df[['A','B']][df['B']=='two']

you will get:

    A    B

2  foo  two
4  foo  two
5  bar  two

To filter on both A and B:

    df[['A','B']][(df['B']=='two') & (df['A']=='foo')]

You get:

        A    B
    2  foo  two
    4  foo  two

and if you want all the columns :

        df[df['B']=='two']

you will get:

            A    B  C   D
        2  foo  two  2   4
        4  foo  two  4   8
        5  bar  two  5  10