[英]Nested dict of lists to pandas DataFrame
I have a rather messy nested dictionary that I am trying to convert to a pandas data frame. 我有一个凌乱的嵌套字典,我试图将其转换为熊猫数据框。 The data is stored in a dictionary of lists contained in a broader dictionary, where each key/value breakdown follows: {userID_key: {postID_key: [list of hash tags]}}
数据存储在包含在更广泛的字典中的列表的字典中,其中每个键/值细分如下: {userID_key: {postID_key: [list of hash tags]}}
Here's a more specific example of what the data looks like: 这是数据看起来更具体的示例:
{'user_1': {'postID_1': ['#fitfam',
'#gym',
'#bro'],
'postID_2': ['#swol',
'#anotherhashtag']},
'user_2': {'postID_78': ['#ripped',
'#bro',
'#morehashtags'],
'postID_1': ['#buff',
'#othertags']},
'user_3': ...and so on }
I want to create a data frame that gives me the frequency counts of each hashtag for each (userID,postID) pair like below: 我想创建一个数据框,为我提供每个(userID,postID)对的每个主题标签的频率计数,如下所示:
+------------+------------+--------+-----+-----+------+-----+
| UserID_key | PostID_key | fitfam | gym | bro | swol | ... |
+------------+------------+--------+-----+-----+------+-----+
| user_1 | postID_1 | 1 | 1 | 1 | 0 | ... |
| user_1 | postID_2 | 0 | 0 | 0 | 1 | ... |
| user_2 | postID_78 | 0 | 0 | 1 | 0 | ... |
| user_2 | postID_1 | 0 | 0 | 0 | 0 | ... |
| user_3 | ... | ... | ... | ... | ... | ... |
+------------+------------+--------+-----+-----+------+-----+
I had scikit-learn's CountVectorizer
as an idea but it's not going to be able to process a nested dictionary. 我将scikit-learn的CountVectorizer
作为一个想法,但是它无法处理嵌套的字典。 Would appreciate any help getting it into that desired form. 希望能有帮助将其转换为所需的形式。
Building on my answer to another question , you can build and concatenate sub-frames using pd.concat
, then use stack
and get_dummies
: 在我对另一个问题的回答的基础上,您可以使用pd.concat
构建和连接子帧,然后使用stack
和get_dummies
:
(pd.concat({k: pd.DataFrame.from_dict(v, orient='index') for k, v in dct.items()})
.stack()
.str.get_dummies()
.sum(level=[0, 1]))
#anotherhashtag #bro #buff #fitfam #gym #morehashtags #othertags #ripped #swol
user_1 postID_1 0 1 0 1 1 0 0 0 0
postID_2 1 0 0 0 0 0 0 0 1
user_2 postID_78 0 1 0 0 0 1 0 1 0
postID_1 0 0 1 0 0 0 1 0 0
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.