I have a nestled list containing various regions for each sample. I would like to make a dataframe such that each row (sample) has the presence or absence of the corresponding region (column). For example, the data might look like this:
region_list = [['North America'], ['North America', 'South America'], ['Asia'], ['North America', 'Asia', 'Australia']]
And the end dataframe would look something like this:
North America South America Asia Australia
1 0 0 0
1 1 0 0
0 0 1 0
1 0 1 1
I think I could probably figure out a way using nestled loops and appends, but is there be a more pythonic way to do this? Perhaps with numpy.where
?
pandas
str.get_dummies
pd.Series(region_list).str.join('|').str.get_dummies()
Asia Australia North America South America
0 0 0 1 0
1 0 0 1 1
2 1 0 0 0
3 1 1 1 0
numpy
np.bincount
with pd.factorize
n = len(region_list)
i = np.arange(n).repeat([len(x) for x in region_list])
f, u = pd.factorize(np.concatenate(region_list))
m = u.size
pd.DataFrame(
np.bincount(i * m + f, minlength=n * m).reshape(n, m),
columns=u
)
North America South America Asia Australia
0 1 0 0 0
1 1 1 0 0
2 0 0 1 0
3 1 0 1 1
Timing
%timeit pd.Series(region_list).str.join('|').str.get_dummies()
1000 loops, best of 3: 1.42 ms per loop
%%timeit
n = len(region_list)
i = np.arange(n).repeat([len(x) for x in region_list])
f, u = pd.factorize(np.concatenate(region_list))
m = u.size
pd.DataFrame(
np.bincount(i * m + f, minlength=n * m).reshape(n, m),
columns=u
)
1000 loops, best of 3: 204 µs per loop
Let's try:
df = pd.DataFrame(region_list)
df2 = df.stack().reset_index(name='region')
df_out = pd.get_dummies(df2.set_index('level_0')['region']).groupby(level=0).sum().rename_axis(None)
print(df_out)
Output:
Asia Australia North America South America
0 0 0 1 0
1 0 0 1 1
2 1 0 0 0
3 1 1 1 0
This will do the job!
import pandas as pd
import itertools
pd.get_dummies(pd.DataFrame(list(itertools.chain(*region_list)))
Output
0_Asia 0_Australia 0_North America 0_South America
0 0 0 1 0
1 0 0 1 0
2 0 0 0 1
3 1 0 0 0
4 0 0 1 0
5 1 0 0 0
6 0 1 0 0
You can use chain.from_iterable
from itertools
module and list comprehension
:
from itertools import chain
region_list = [['North America'], ['North America', 'South America'], ['Asia'], ['North America', 'Asia', 'Australia']]
regions = list(set(chain.from_iterable(region_list)))
vals = [[1 if j in k else 0 for j in regions] for k in region_list]
df = pd.DataFrame(vals, columns=regions)
print(df)
Output:
Australia Asia North America South America
0 0 0 1 0
1 0 0 1 1
2 0 1 0 0
3 1 1 1 0
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.