TypeError：遍历每一行以获得特定列值时，字符串索引必须是整数

Question

I want to perform linear regression analysis on time for each gene taking all the variables present in the model, hence using all the genes.

In df5, the x-axis represents "Gene Symbol" and y-axis represent "Time".

import pandas as pd
import numpy as np
import plotly.express as px
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

# Split data into training and test splits
train_idx, test_idx = train_test_split(df5.index, test_size=.25, random_state=0)
df5["split"] = "train"
df5.loc[test_idx, "split"] = "test"

# Inputs and targets
X = df5.iloc[:, 1:-1]
y = df5.iloc[:, 0]

X_train = df5.loc[train_idx, ["4", "8", "12", "24", "48"]]
y_train = df5.loc[train_idx, "0"]

# Linear regression prediction
model = LinearRegression()
model.fit(X_train, y_train)
df5['prediction'] = model.predict(X)

I get a typeerror when I want to set y variable as the prediction column value for each row using y=i["prediction"] .

# Scatter plot
for i, j in df5.iterrows():
  for col in df5.columns:
    fig = px.scatter(df5[col], x=df5.iloc[:,0], y=i["prediction"], marginal_x='histogram', marginal_y='histogram', color='split', trendline='ols')
    fig.update_traces(histnorm='probability', selector={'type':'histogram'})
    fig.add_shape(type="line", line=dict(dash='dash'), x0=y.min(), y0=y.min(), x1=y.max(), y1=y.max())
    fig.show()

Traceback:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-395-6ef08290c83a> in <module>()
      2 for i, j in df5.iterrows():
      3   for col in df5.columns:
----> 4     fig = px.scatter(df5[col], x=df5.iloc[:,0], y=i["prediction"], marginal_x='histogram', marginal_y='histogram', color='split', trendline='ols')
      5     fig.update_traces(histnorm='probability', selector={'type':'histogram'})
      6     fig.add_shape(type="line", line=dict(dash='dash'), x0=y.min(), y0=y.min(), x1=y.max(), y1=y.max())

TypeError: string indices must be integers

Data:

df5.head().to_dict()

{'0': {'DNAJB6 /// TMEM135': 0.30131649339447103,
  'DNAJC14': 0.2255444383216058,
  'DNAJC15': 0.25789169794229455,
  'DNAJC30': 0.11388797858763917,
  'DNAJC9': 0.11205541676885071},
 '12': {'DNAJB6 /// TMEM135': 0.28354614480145346,
  'DNAJC14': 0.2343653660720247,
  'DNAJC15': 0.2406210529534205,
  'DNAJC30': 0.11229754447748205,
  'DNAJC9': 0.12045170255898871},
 '24': {'DNAJB6 /// TMEM135': 0.27395808285292367,
  'DNAJC14': 0.2246018336027369,
  'DNAJC15': 0.22347959865906092,
  'DNAJC30': 0.11379897713291527,
  'DNAJC9': 0.10622530623273815},
 '4': {'DNAJB6 /// TMEM135': 0.2949284643966144,
  'DNAJC14': 0.22905481299223704,
  'DNAJC15': 0.22312009403152122,
  'DNAJC30': 0.13114878202076288,
  'DNAJC9': 0.12991396178392187},
 '48': {'DNAJB6 /// TMEM135': 0.289873135093664,
  'DNAJC14': 0.2349502215468218,
  'DNAJC15': 0.17706771640592167,
  'DNAJC30': 0.10857074282633467,
  'DNAJC9': 0.13001391250069522},
 '8': {'DNAJB6 /// TMEM135': 0.2794865791356734,
  'DNAJC14': 0.22228815371920396,
  'DNAJC15': 0.22912018863353348,
  'DNAJC30': 0.11799998627920205,
  'DNAJC9': 0.10520854728987451}}

Answer 1

First: If error shows you which line makes problem then first you could use print(), print(type(...)), etc to check what you have in variables in this line.

It seems you use wrong variable. I think wrong is i["prediction"] because i should be index of row , not row with data . Maybe if you would use more readable variables for index, row in df5.iterrow() instead of for i,j in df.iterrow() then you would see that you run index["prediction"] instead of row["prediction"]

---

But frankly I don't understand what you try to plot.

x=df5.iloc[:,0] should give all data in column, not in row, but y=row["prediction"] should give single value from one row. It makes no sense. You should rather use y=df5["prediction"] and run it without df5.iterrows() - or even use only columns names instead of data px.scatter(df5, x=col, y="prediction", ...)

for col in ["4", "8", "12", "24", "48"]:  # without "0"
    fig = px.scatter(df5, x=col, y="prediction", marginal_x='histogram', marginal_y='histogram', color='split')#, trendline='ols')
    fig.update_traces(histnorm='probability', selector={'type':'histogram'})
    fig.add_shape(type="line", line=dict(dash='dash'), x0=y.min(), y0=y.min(), x1=y.max(), y1=y.max())
    fig.show()

---

Full working code with example data in code - so everyone can simply copy and run it

BTW: it opens every plot on separated page. And I had to skip trendline='ols' in scatter because it gives me error ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/usr/local/lib/python3.8/dist-packages/scipy/signal/signaltools.py)

data = {'0': {'DNAJB6 /// TMEM135': 0.30131649339447103,
  'DNAJC14': 0.2255444383216058,
  'DNAJC15': 0.25789169794229455,
  'DNAJC30': 0.11388797858763917,
  'DNAJC9': 0.11205541676885071},
 '12': {'DNAJB6 /// TMEM135': 0.28354614480145346,
  'DNAJC14': 0.2343653660720247,
  'DNAJC15': 0.2406210529534205,
  'DNAJC30': 0.11229754447748205,
  'DNAJC9': 0.12045170255898871},
 '24': {'DNAJB6 /// TMEM135': 0.27395808285292367,
  'DNAJC14': 0.2246018336027369,
  'DNAJC15': 0.22347959865906092,
  'DNAJC30': 0.11379897713291527,
  'DNAJC9': 0.10622530623273815},
 '4': {'DNAJB6 /// TMEM135': 0.2949284643966144,
  'DNAJC14': 0.22905481299223704,
  'DNAJC15': 0.22312009403152122,
  'DNAJC30': 0.13114878202076288,
  'DNAJC9': 0.12991396178392187},
 '48': {'DNAJB6 /// TMEM135': 0.289873135093664,
  'DNAJC14': 0.2349502215468218,
  'DNAJC15': 0.17706771640592167,
  'DNAJC30': 0.10857074282633467,
  'DNAJC9': 0.13001391250069522},
 '8': {'DNAJB6 /// TMEM135': 0.2794865791356734,
  'DNAJC14': 0.22228815371920396,
  'DNAJC15': 0.22912018863353348,
  'DNAJC30': 0.11799998627920205,
  'DNAJC9': 0.10520854728987451}
}

import pandas as pd
import numpy as np
import plotly.express as px
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

df5 = pd.DataFrame(data)

# Split data into training and test splits
train_idx, test_idx = train_test_split(df5.index, test_size=.25, random_state=0)
df5["split"] = "train"
df5.loc[test_idx, "split"] = "test"

# Inputs and targets
X = df5.iloc[:, 1:-1]
y = df5.iloc[:, 0]

X_train = df5.loc[train_idx, ["4", "8", "12", "24", "48"]]
y_train = df5.loc[train_idx, "0"]

# Linear regression prediction
model = LinearRegression()
model.fit(X_train, y_train)
df5['prediction'] = model.predict(X)

for col in ["4", "8", "12", "24", "48"]:  # without "0"
    fig = px.scatter(df5, x=col, y="prediction", marginal_x='histogram', marginal_y='histogram', color='split')#, trendline='ols')
    fig.update_traces(histnorm='probability', selector={'type':'histogram'})
    fig.add_shape(type="line", line=dict(dash='dash'), x0=y.min(), y0=y.min(), x1=y.max(), y1=y.max())
    fig.show()

Plot for column "4"