Pandas: Change dataframe values based on dictionary and remove rows with no match

Question

I have a pandas dataframe with a row containing chromosomes listed in the format of: "chr1", "chr2"...

I have a dictionary to Convert these values to an integer - such as:

HashTable = {"chr1" : 1, "chr2" : 2, "chr3" : 3, "chr4" : 4, "chr5" : 5, "chr6" : 6, "chr7" : 7, "chr8" : 8, "chr9" : 9, "chr10" : 10, "chr11" : 11, "chr12" : 12, "chr13" : 13, "chr14" : 14, "chr15" : 15, "chr16" : 16, "chr17" : 17, "chr18" : 18, "chr19" : 19, "chrX" : 20, "chrY" : 21, "chrM" : 22, 'chrMT': 23}

I would like to convert the chromosomes in the dataframe "Chrom" column to the integer values. There are also some chromosomes that are not found in the dictionary that I would like to remove from the dataframe. Is there a simple way to do this?

Answer 1

You can use isin to filter for valid rows, and then use replace to replace the values:

import pandas as pd
HashTable = {"chr1" : 1, "chr2" : 2, "chr3" : 3, "chr4" : 4, "chr5" : 5, "chr6" : 6, "chr7" : 7, "chr8" : 8, "chr9" : 9, "chr10" : 10, "chr11" : 11, "chr12" : 12, "chr13" : 13, "chr14" : 14, "chr15" : 15, "chr16" : 16, "chr17" : 17, "chr18" : 18, "chr19" : 19, "chrX" : 20, "chrY" : 21, "chrM" : 22, 'chrMT': 23}
# A dummy DataFrame with all the valid chromosomes and one unknown chromosome
df = pd.DataFrame({"Chrom": HashTable.keys() + ["unknown_chr"]})
# Filter for valid rows
df = df[df["Chrom"].isin(HashTable.keys())]
# Replace the values according to dict
df["Chrom"].replace(HashTable, inplace=True)
print df

Input (the dummy df above):

          Chrom
0         chrMT
1          chrY
2          chrX
3         chr13
4         chr12
5         chr11
6         chr10
7         chr17
8         chr16
9         chr15
10        chr14
11        chr19
12        chr18
13         chrM
14         chr7
15         chr6
16         chr5
17         chr4
18         chr3
19         chr2
20         chr1
21         chr9
22         chr8
23  unknown_chr

Output DataFrame:

   Chrom
0     23
1     21
2     20
3     13
4     12
5     11
6     10
7     17
8     16
9     15
10    14
11    19
12    18
13    22
14     7
15     6
16     5
17     4
18     3
19     2
20     1
21     9
22     8

If the resulting values are all integers, you change the above replace line to enforce the correct dtype :

df["Chrom"] = df["Chrom"].replace(HashTable).astype(int)