Use one data-frame (used as a dictionary) to fill in the main data-frame (Python, Pandas)

Question

I have a central DataFrame called "cases" (5000000 rows × 5 columns) and a secondary DataFrame, called "relevant information", which is a kind of dictionary in relation to the central DataFrame (300 rows × 6 columns). I am trying to fill in the central DataFrame based on a common column called "Verdict_type". And, if the value does not appear in the secondary DataFrame it fill in "not_relevant" in all the rows that will be added. I used all sorts of directions without success. I would love to get a good direction.

The DataFrames

Answer 1

import pandas as pd

# this is a mockup of the raw data
cases = [
    [1, "1", "v1"],
    [2, "2", "v2"],
    [3, "3", "v3"]
]

relevant_info = [
    ["v1", "info1"],
    ["v3", "info3"]
]

# these are the data from screenshot
df_cases = pd.DataFrame(cases, columns=["id", "verdict_name", "verdict_type"]).set_index("id")
df_relevant_info = pd.DataFrame(relevant_info, columns=["verdict_type", "features"])

Input:
df_cases <-- note here the index marked as 'id'
df_relevant_info

# first, flatten the index of the cases ( this is probably what you were missing )
df_cases = df_cases.reset_index()
# then, merge the two sets on the verdict_type
df_merge = pd.merge(df_cases, df_relevant_info, on="verdict_type", how="outer")
# finally, mark missing values as non relevant
df_merge["features"] = df_merge["features"].fillna(value="not_relevant")

Output:

merged set:
+----+------+----------------+----------------+--------------+
|    |   id |   verdict_name | verdict_type   | features     |
|----+------+----------------+----------------+--------------|
|  0 |    1 |              1 | v1             | info1        |
|  1 |    2 |              2 | v2             | not_relevant |
|  2 |    3 |              3 | v3             | info3        |
+----+------+----------------+----------------+--------------+