[英]Keras custom loss function per tensor group
I am writing a custom loss function that requires calculating ratios of predicted values per group.我正在编写一个自定义损失函数,需要计算每组预测值的比率。 As a simplified example, here is what my Data and model code looks like:
作为一个简化的例子,这里是我的数据和模型代码的样子:
def main():
df = pd.DataFrame(columns=["feature_1", "feature_2", "condition_1", "condition_2", "label"],
data=[[5, 10, "a", "1", 0],
[30, 20, "a", "1", 1],
[50, 40, "a", "1", 0],
[15, 20, "a", "2", 0],
[25, 30, "b", "2", 1],
[35, 40, "b", "1", 0],
[10, 80, "b", "1", 1]])
features = ["feature_1", "feature_2"]
conds_and_label = ["condition_1", "condition_2", "label"]
X = df[features]
Y = df[conds_and_label]
model = my_model(input_shape=len(features))
model.fit(X, Y, epochs=10, batch_size=128)
model.evaluate(X, Y)
def custom_loss(conditions, y_pred): # this is what I need help with
conds = ["condition_1", "condition_2"]
conditions["label_pred"] = y_pred
g = conditions.groupby(by=conds,
as_index=False).apply(lambda x: x["label_pred"].sum() /
len(x)).reset_index(name="pred_ratio")
# true_ratios will be a constant, external DataFrame. Simplified example here:
true_ratios = pd.DataFrame(columns=["condition_1", "condition_2", "true_ratio"],
data=[["a", "1", 0.1],
["a", "2", 0.2],
["b", "1", 0.8],
["b", "2", 0.9]])
merged = pd.merge(g, true_ratios, on=conds)
merged["diff"] = merged["pred_ratio"] - merged["true_ratio"]
return K.mean(K.abs(merged["diff"]))
def joint_loss(conds_and_label, y_pred):
y_true = conds_and_label[:, 2]
conditions = tf.gather(conds_and_label, [0, 1], axis=1)
loss_1 = standard_loss(y_true=y_true, y_pred=y_pred) # not shown
loss_2 = custom_loss(conditions=conditions, y_pred=y_pred)
return 0.5 * loss_1 + 0.5 * loss_2
def my_model(input_shape=None):
model = Sequential()
model.add(Dense(units=2, activation="relu"), input_shape=(input_shape,))
model.add(Dense(units=1, activation='sigmoid'))
model.add(Flatten())
model.compile(loss=joint_loss, optimizer="Adam",
metrics=[joint_loss, custom_loss, "accuracy"])
return model
What I need help with is the custom_loss
function.我需要帮助的是
custom_loss
函数。 As you can see, it is currently written as if the inputs are Pandas DataFrames.如您所见,它目前被编写为好像输入是 Pandas DataFrames。 However, the inputs will be Keras Tensors (with tensorflow backend), so I am trying to figure out how to convert the current code in
custom_loss
to use Keras/TF backend functions .但是,输入将是 Keras Tensors(带有 tensorflow 后端),因此我试图弄清楚如何将
custom_loss
的当前代码转换为使用custom_loss
/TF 后端函数。 For example, I searched online and couldn't find out a way to do a groupby in Keras/TF to get the ratios I need...例如,我在网上搜索并找不到在 Keras/TF 中进行 groupby 以获得我需要的比率的方法......
Some context/explanation that might be helpful to you:一些可能对您有帮助的上下文/解释:
joint_loss
, which consists of standard_loss
(not shown) and custom_loss
.joint_loss
,它由standard_loss
(未显示)和custom_loss
。 But I only need help converting custom_loss
.custom_loss
。custom_loss
does is: custom_loss
作用是:
I ended up figuring out a solution to this, though I would like some feedback on it (specifically some parts).我最终想出了一个解决方案,尽管我希望得到一些反馈(特别是某些部分)。 Here is the solution:
这是解决方案:
import pandas as pd
import tensorflow as tf
import keras.backend as K
from keras.models import Sequential
from keras.layers import Dense, Flatten, Dropout
from tensorflow.python.ops import gen_array_ops
def main():
df = pd.DataFrame(columns=["feature_1", "feature_2", "condition_1", "condition_2", "label"],
data=[[5, 10, "a", "1", 0],
[30, 20, "a", "1", 1],
[50, 40, "a", "1", 0],
[15, 20, "a", "2", 0],
[25, 30, "b", "2", 1],
[35, 40, "b", "1", 0],
[10, 80, "b", "1", 1]])
df = pd.concat([df] * 500) # making data artificially larger
true_ratios = pd.DataFrame(columns=["condition_1", "condition_2", "true_ratio"],
data=[["a", "1", 0.1],
["a", "2", 0.2],
["b", "1", 0.8],
["b", "2", 0.9]])
features = ["feature_1", "feature_2"]
conditions = ["condition_1", "condition_2"]
conds_ratios_label = conditions + ["true_ratio", "label"]
df = pd.merge(df, true_ratios, on=conditions, how="left")
X = df[features]
Y = df[conds_ratios_label]
# need to convert strings to ints because tensors can't mix strings with floats/ints
mapping_1 = {"a": 1, "b": 2}
mapping_2 = {"1": 1, "2": 2}
Y.replace({"condition_1": mapping_1}, inplace=True)
Y.replace({"condition_2": mapping_2}, inplace=True)
X = tf.convert_to_tensor(X)
Y = tf.convert_to_tensor(Y)
model = my_model(input_shape=len(features))
model.fit(X, Y, epochs=1, batch_size=64)
print()
print(model.evaluate(X, Y))
def custom_loss(conditions, true_ratios, y_pred):
y_pred = tf.sigmoid((y_pred - 0.5) * 1000)
uniques, idx, count = gen_array_ops.unique_with_counts_v2(conditions, [0])
num_unique = tf.size(count)
sums = tf.math.unsorted_segment_sum(data=y_pred, segment_ids=idx, num_segments=num_unique)
lengths = tf.cast(count, tf.float32)
pred_ratios = tf.divide(sums, lengths)
mean_pred_ratios = tf.math.reduce_mean(pred_ratios)
mean_true_ratios = tf.math.reduce_mean(true_ratios)
diff = mean_pred_ratios - mean_true_ratios
return K.mean(K.abs(diff))
def standard_loss(y_true, y_pred):
return tf.losses.binary_crossentropy(y_true=y_true, y_pred=y_pred)
def joint_loss(conds_ratios_label, y_pred):
y_true = conds_ratios_label[:, 3]
true_ratios = conds_ratios_label[:, 2]
conditions = tf.gather(conds_ratios_label, [0, 1], axis=1)
loss_1 = standard_loss(y_true=y_true, y_pred=y_pred)
loss_2 = custom_loss(conditions=conditions, true_ratios=true_ratios, y_pred=y_pred)
return 0.5 * loss_1 + 0.5 * loss_2
def my_model(input_shape=None):
model = Sequential()
model.add(Dropout(0, input_shape=(input_shape,)))
model.add(Dense(units=2, activation="relu"))
model.add(Dense(units=1, activation='sigmoid'))
model.add(Flatten())
model.compile(loss=joint_loss, optimizer="Adam",
metrics=[joint_loss, "accuracy"], # had to remove custom_loss because it takes 3 args now
run_eagerly=True)
return model
if __name__ == '__main__':
main()
The main updates are to custom_loss
.主要更新是
custom_loss
。 I removed creating the true_ratios DataFrame from custom_loss
and instead appended it to my Y
in main.我从
custom_loss
删除了创建 true_ratios DataFrame,而是将它附加到我的Y
中。 Now custom_loss
takes 3 arguments, one of which is the true_ratios
tensor.现在
custom_loss
有 3 个参数,其中之一是true_ratios
张量。 I had to use gen_array_ops.unique_with_counts_v2
and unsorted_segment_sum
to get sums per group of conditions.我不得不使用
gen_array_ops.unique_with_counts_v2
和unsorted_segment_sum
来获得每组条件的总和。 And then I got the lengths of each group in order to create pred_ratios
(calculated ratios per group based on y_pred
).然后我得到了每组的长度以创建
pred_ratios
(基于y_pred
计算每组的y_pred
)。 Finally I get the mean predicted ratios and mean true ratios, and take the absolute difference to get my custom loss.最后我得到平均预测比率和平均真实比率,并取绝对差异来获得我的自定义损失。
Some things of note:一些注意事项:
tf.round
, but I realized that is not differentiable.tf.round
,但我意识到这是不可微的。 So instead I replaced it with y_pred = tf.sigmoid((y_pred - 0.5) * 1000)
inside of custom_loss
.y_pred = tf.sigmoid((y_pred - 0.5) * 1000)
的内部custom_loss
。 This essentially takes all the y_pred
values to 0 and 1, but in a differentiable way.y_pred
值取为 0 和 1,但以可微分的方式。 It seems like a bit of a "hack" though, so please let me know if you have any feedback on this.run_eagerly=True
in model.compile()
.model.compile()
使用run_eagerly=True
model.compile()
。 Otherwise I get this error: "ValueError: Dimensions must be equal, but are 1 and 2 for ...".tf.unsorted_segment_sum
.tf.unsorted_segment_sum
的行。unique_with_counts_v2
does not actually exist in tensorflow API yet, but it exists in the source code. unique_with_counts_v2
实际上并不存在于 tensorflow API 中,但它存在于源代码中。 I needed this to be able to group by multiple conditions (not just a single one). Feel free to comment if you have any feedback on this, in general, or in response to the bullets above.如果您对此有任何反馈,或者对上述要点有任何反馈,请随时发表评论。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.