如何将 plot 与 Pandas 随时间变化的变量之间的相关性折线图？

Question

I have three variables (x, y and z) collected at different times (30, 60 and 120 days). I have a correlation dataframe between the three variables, separated by the collected days.

I would like to plot a line graph, to try to understand the behavior of the correlation between the same variables over time.

On the graph's X axis, the times 30, 60 and 120 days and on the Y axis, the correlation values for each pair of variables (without repeating the combination between them or the correlation with itself (1.00)), that is, only the correlations between x and y, x and z, and y and z.

Below I made a reproducible example of the three dataframes I have.

import pandas as pd

day30_dict = {
    "Index": [
        "x30, y30",
        "x30, z30",
        "x30, x30",
        "y30, x30",
        "y30, z30",
        "y30, y30",
        "z30, x30",
        "z30, y30",
        "z30, z30",
    ],
    "cor": [0.50, 0.11, 1.00, 0.50, 0.22, 1.00, 0.11, 0.22, 1.00],
}

day30_df = pd.DataFrame(day30_dict)
day30_df = day30_df.set_index("Index")

day60_dict = {
    "Index": [
        "x60, y60",
        "x60, z60",
        "x60, x60",
        "y60, x60",
        "y60, z60",
        "y60, y60",
        "z60, x60",
        "z60, y60",
        "z60, z60",
    ],
    "cor": [0.10, 0.15, 1.00, 0.10, 0.77, 1.00, 0.15, 0.77, 1.00],
}

day60_df = pd.DataFrame(day60_dict)
day60_df = day60_df.set_index("Index")

day120_dict = {
    "Index": [
        "x120, y120",
        "x120, z120",
        "x120, x120",
        "y120, x120",
        "y120, z120",
        "y120, y120",
        "z120, x120",
        "z120, y120",
        "z120, z120",
    ],
    "cor": [0.01, 0.03, 1.00, 0.01, 0.90, 1.00, 0.03, 0.90, 1.00],
}

day120_df = pd.DataFrame(day120_dict)
day120_df = day120_df.set_index("Index")```

Answer 1

Here is one way to do it with Pandas drop_duplicates , string methods , andconcat :

# Remove duplicates and self correlations in each dataframe,
# Add a new column for time
# Remove numeric values from Index column
# Store dataframes in a list
dfs = []
for df in [day30_df, day60_df, day120_df]:
    df = df[df["cor"] < 1].reset_index()
    df["temp"] = df["Index"].apply(sorted)
    df = df.drop_duplicates("temp").drop(columns="temp")
    df["time"] = df["Index"].str.extract(r"(\d+)")
    df["Index"] = df["Index"].str.replace(r"\d+", "", regex=True)
    dfs.append(df)

new_df = pd.concat(dfs).sort_values("Index", ignore_index=True)

Then, running this code in a Jupyter notebook cell:

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(8,6))
for i, df_ in new_df.groupby("Index"):
    df_.plot(x="time", y="cor", label=i, ax=ax)

Outputs: