A simple IF statement in Python 3.X Pandas not working

Question

This is supposed to be a simple IF statement that is updating based on a condition but it is not working.

Here is my code

df["Category"].fillna("999", inplace = True)

for index, row in df.iterrows():
    if (str(row['Category']).strip()=="11"):
        print(str(row['Category']).strip())
        df["Category_Description"] = "Agriculture, Forestry, Fishing and Hunting"
    elif (str(row['Category']).strip()=="21"):
        df["Category_Description"] = "Mining, Quarrying, and Oil and Gas Extraction"  

The print statement

print(str(row['Category']).strip()) 

is working fine but updates to the Category_Description column are not working.

The input data has the following codes

Category Count of Records
48    17845
42     2024
99     1582
23     1058
54     1032
56      990
32      916
33      874
44      695
11      630
53      421
81      395
31      353
49      336
21      171
45      171
52      116
71      108
61       77
51       64
62       54
72       51
92       36
55       35
22       14

The update resulted in

Agriculture, Forestry, Fishing and Hunting    41183

Here is a small sample of the dataset and code on repl.it https://repl.it/@RamprasadRengan/SimpleIF#main.py When I run the code above with this data I still see the same issue.

What am I missing here?

Answer 1

You are performing a row operation but applying a dataframe change in the "IF" statement. This will apply the values to all the records.

Try sometime like:

def get_category_for_record(rec): 
    if (str(row['Category']).strip()=="11"):
        return "Agriculture, Forestry, Fishing and Hunting"
    elif (str(row['Category']).strip()=="21"):
        return "Mining, Quarrying, and Oil and Gas Extraction"  

df["category"] = df.apply(get_category_for_record, axis = 1)

Answer 2

I think you want to add a column to the dataframe that maps category to a longer description. As mentioned in the comments, assignment to a column affects the entire column. But if you use a list, each row in the column gets the corresponding value.

So use a dictionary to map name to description, build a list, and assign it.

import pandas as pd

category_map = {
    "11":"Agriculture, Forestry, Fishing and Hunting",
    "21":"Mining, Quarrying, and Oil and Gas Extraction"}

df = pd.DataFrame([["48", 17845],
                   ["  11 ", 88888], 
                   ["12", 33333],
                   ["21", 999]], 
    columns=["category", "count of records"])

# cleanup category and add description
df["category"] = df["category"].str.strip()
df["Category_Description"] = [category_map.get(cat, "") 
        for cat in df["category"]]
# alternately....
#df.insert(2, "Category_Description", 
#        [category_map.get(cat, "") for cat in df["category"]])
print(df)