Pandas 删除重复的列

Question

I am playing with an excel spreadsheet that is comparing the the value of two items across different currencies. The spreadsheet headings are as below:

USD USD DIFFERENCE | GBP GBP DIFFERENCE | JPY JPY DIFFERENCE.......

When I import this to pandas and create a dataframe it creates headings named Difference.1, Difference.2, Difference.3...., Difference.n

I want to remove all headings named DIFFERENCE

Note all the difference headings are uniquely named

Answer 1

I think you can refer to this link for pandas df.drop functions what we will do is below.

1. search for all the columns name with DIFFERENCE
2. generate a list with all these names in it
3. drop columns that contain these names

// get a new df that contains only name with DIFFERENCE in it

df2 = df.filter(like='DIFFERENCE ', axis=1)

// generate a list for all these column name

x = []
for col in df2.columns:
    x.append(col)

// drop columns with these names in x

df.drop(columns=x)

// OR you can keep a new df for these updated information

df3 = df.drop(columns=x)

Answer 2

I believe all you need is remove columns that contain 'DIFFERENCE' in it names. In this case you can simply do:

    df = pd.read_csv("../path/to/your/file.csv")
    df = df[df.columns.drop(list(df.filter(regex='DIFFERENCE')))]

Example

If you have something like this:

df = pd.DataFrame({"a1": [1,2,3], "a2":[1,1,1], "b":[2,4,6]})
print(df)

Out:
   a1  a2  b
0   1   1  2
1   2   1  4
2   3   1  6

Further step is

df = df[df.columns.drop(list(df.filter(regex='a')))]
print(df)

Out:
   b
0  2
1  4
2  6

You can read more about pandas.DataFrame.drop