如何从字典字符串中打印特定键。

Question

I wanted to get a one hot data based on the number of elements in the list when using sklearn transform.

Code:

from sklearn.feature_extraction.text import CountVectorizer
from itertools import chain


x = [['1234', '5678', '910', 'baba'], ['8', '1'], 
     [], ['9', '3'], [], ['7', '6'], [], []]
vector = CountVectorizer(token_pattern=r".+",  min_df=1, max_df=1.0, lowercase=False,
                 max_features=None)
vec = [xxx for xx in x for xxx in xx]
vector.fit(chain.from_iterable([vec]))
print(vector.get_feature_names())
new = []
for xx in x:
    new.append(vector.transform(xx))
for x in new:
    for xx in x.toarray():
        print(xx)

Current output:

['1', '1234', '3', '5678', '6', '7', '8', '9', '910', 'baba']
[0 1 0 0 0 0 0 0 0 0]
[0 0 0 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 0]
[0 0 0 0 0 0 0 0 0 1]
[0 0 0 0 0 0 1 0 0 0]
[1 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 0 0]
[0 0 1 0 0 0 0 0 0 0]
[0 0 0 0 0 1 0 0 0 0]
[0 0 0 0 1 0 0 0 0 0]

My expected output:

['1', '1234', '3', '5678', '6', '7', '8', '9', '910', 'baba']
[0 1 0 1 0 0 0 0 1 1]
[1 0 0 0 0 0 1 0 0 0]
[0 0 1 0 0 0 0 1 0 0]
[0 0 0 0 1 1 0 0 0 0]

Is there a way to do it using my code? I have tried to change it many times but unfortunately to no luck. Somehow, my brain stops to process anything now.

Answer 1

You shouldn't need explicit for loops for this task. You can use MultiLabelBinarizer instead, also from the sklearn library. It doesn't handle empty lists, so just filter those out first.

Here's an example with Pandas:

import pandas as pd
from sklearn.preprocessing import MultiLabelBinarizer

L = [['1234', '5678', '910', 'baba'], ['8', '1'], 
     [], ['9', '3'], [], ['7', '6'], [], []]

s = pd.Series(list(filter(None, L)))

mlb = MultiLabelBinarizer()

res = pd.DataFrame(mlb.fit_transform(s),
                   columns=mlb.classes_,
                   index=s.index)

print(res)

   1  1234  3  5678  6  7  8  9  910  baba
0  0     1  0     1  0  0  0  0    1     1
1  1     0  0     0  0  0  1  0    0     0
2  0     0  1     0  0  0  0  1    0     0
3  0     0  0     0  1  1  0  0    0     0

Answer 2

You can try of using intersect and np isin

intersect function will give closed elements and isin will create boolean list

mask = ['1', '1234', '3', '5678', '6', '7', '8', '9', '910', 'baba']
for xx in x:
    if len(xx)>1:
        print(np.isin(mask,np.array(list(set(xx).intersection(set(mask))))).astype(int))

Out:

[0 1 0 1 0 0 0 0 1 1]
[1 0 0 0 0 0 1 0 0 0]
[0 0 1 0 0 0 0 1 0 0]
[0 0 0 0 1 1 0 0 0 0]

Flattening the lists

#if you have big lists of elements you can flatten by 
sum(x,[])

Out:

['1234', '5678', '910', 'baba', '8', '1', '9', '3', '7', '6']

Answer 3

For future readers:

I somehow solved it with a SUPER NAIVE way.

Here is the codes:

from sklearn.feature_extraction.text import CountVectorizer from itertools import chain

x = [['1234', '5678', '910', 'baba'], ['8', '1'], 
     [], ['9', '3'], [], ['7', '6'], [], []]
vector = CountVectorizer(token_pattern=r"\S*\d+\S*",  min_df=1, max_df=1.0, lowercase=False,
                 max_features=None)
vec = [xxx for xx in x for xxx in xx]
vector.fit(chain.from_iterable([vec]))
print(vector.get_feature_names())
new = []
for xx in x:
    new.append(" ".join(xx))

neww = vector.transform(new)

print(neww.toarray())