如果值存在于另一个 df.Series 描述中，则从值列表中填充 pd.Series

Question

I need to solve tricky problem, and minimize big O notation problem.

I have two pandas dataframes:

The first df is like as:

| source | searchTermsList |

|:---- |:------:|

| A  | [t1, t2, t3,...tn] |
| B  | [t4, t5, t6,...tn] |
| C  | [t7, t8, t9,...tn] |

Where the first column is string, the second one is a list of strings no duplicated, just unique values.

The second dataframe, which I need to create a new pd.Series with first column (df1.source) if term in searchTerm list, exist in the follow df2.Series, called "description".

Example.
| objID | dataDescr |

|:---- |:------:|

| 1  | The first description name has t2 | 
| 2  | The second description name has t6 and t7| 
| 3  | The third description name has t8, t1, t9| 

Expected results

| objID | dataDescr | source |

|:---- |:------:| -----:|

| 1  | The first description name    | A |
| 2  | The second description name    | B |
| 3  | The third description name    | C |

Explanation

• The first description has t2, so the column filled with A, because t2 appears in the term list.

• The second description has two terms, t6 and t7, in that case match only the first one with the second list, so the source will be filled B

• The third description has three terms, as above, only get the first one with the list and source will be filled with C.

My approach

If I split descrName and finally search that word in the all lists, maybe the computational cost will be very huge. The idea with map, doesn't work, because with haven't ordered dataframe, in the first just we have 10-20 rows, only unique values, in the second will be to matching with each terms n times.

Any suggestion,please?

Answer 1

Use:

s = df1.explode('searchTermsList').set_index('searchTermsList')['source']
print (s)
t1    A
t2    A
t3    A
t4    B
t5    B
t6    B
t7    C
t8    C
t9    C
Name: source, dtype: object

pat = r"\b({})\b".format("|".join(s.index))

df2['searchTermsList'] = df2['dataDescr'].str.extract(pat, expand=False).map(s)
print (df2)
   objID                                  dataDescr searchTermsList
0      1          The first description name has t2               A
1      2  The second description name has t6 and t7               B
2      3  The third description name has t8, t1, t9               C