[英]Training the model gives ValueError when I add class weights
I was using multi class U-Net segmentation where I am encountering value error while training by data model.我正在使用多类 U-Net 分段,在通过数据模型进行训练时遇到值错误。 My multi class model is divided into 4 classes.
我的多类模型分为 4 个类。
Code for training model:训练模型代码:
from simple_multi_unet_model import multi_unet_model #Uses softmax
from tensorflow.keras.utils import normalize
import os
import glob
import cv2
import numpy as np
from matplotlib import pyplot as plt
from sklearn.model_selection import train_test_split
import tensorflow as tf
#Resizing images, if needed
SIZE_X = 128
SIZE_Y = 128
n_classes=4 #Number of classes for segmentation
#Capture training image info as a list
train_images = []
directory_path = '/home/Documents/Multiclass/images/'
list_of_files = sorted( filter( os.path.isfile, glob.glob(directory_path + '*.png', recursive=True) ) )
for img_path in list_of_files:
img = cv2.imread(img_path, 0)
img = cv2.resize(img, (SIZE_Y, SIZE_X))
train_images.append(img)
#Convert list to array for machine learning processing
train_images = np.array(train_images)
#Capture mask/label info as a list
train_masks = []
labels_path = '/home/Documents/Multiclass/labels/'
list_of_labels = sorted( filter( os.path.isfile, glob.glob(labels_path + '*.png', recursive=True) ) )
for mask_path in list_of_labels:
mask = cv2.imread(mask_path, 0)
mask = cv2.resize(mask, (SIZE_Y, SIZE_X), interpolation = cv2.INTER_NEAREST) #Otherwise ground truth changes due to interpolation
train_masks.append(mask)
#Convert list to array for machine learning processing
train_masks = np.array(train_masks)
###############################################
#Encode labels... but multi dim array so need to flatten, encode and reshape
from sklearn.preprocessing import LabelEncoder
labelencoder = LabelEncoder()
n, h, w = train_masks.shape
train_masks_reshaped = train_masks.reshape(-1,1)
train_masks_reshaped_encoded = labelencoder.fit_transform(train_masks_reshaped)
train_masks_encoded_original_shape = train_masks_reshaped_encoded.reshape(n, h, w)
np.unique(train_masks_encoded_original_shape)
#################################################
train_images = np.expand_dims(train_images, axis=3)
train_images = normalize(train_images, axis=1)
train_masks_input = np.expand_dims(train_masks_encoded_original_shape, axis=3)
#Create a subset of data for quick testing
#Picking 10% for testing and remaining for training
from sklearn.model_selection import train_test_split
x_train, X_test, y_train, y_test = train_test_split(train_images, train_masks_input, test_size = 0.10, random_state = 0)
print("Class values in the dataset are ... ", np.unique(y_train)) # 0 is the background/few unlabeled
from tensorflow.keras.utils import to_categorical
train_masks_cat = to_categorical(y_train, num_classes=n_classes)
y_train_cat = train_masks_cat.reshape((y_train.shape[0], y_train.shape[1], y_train.shape[2], n_classes))
test_masks_cat = to_categorical(y_test, num_classes=n_classes)
y_test_cat = test_masks_cat.reshape((y_test.shape[0], y_test.shape[1], y_test.shape[2], n_classes))
###############################################################
from sklearn.utils import class_weight
class_weights = class_weight.compute_class_weight('balanced',
np.unique(train_masks_reshaped_encoded),
train_masks_reshaped_encoded)
print("Class weights are...:", class_weights)
IMG_HEIGHT = x_train.shape[1]
IMG_WIDTH = x_train.shape[2]
IMG_CHANNELS = x_train.shape[3]
def get_model():
return multi_unet_model(n_classes=n_classes, IMG_HEIGHT=IMG_HEIGHT, IMG_WIDTH=IMG_WIDTH, IMG_CHANNELS=IMG_CHANNELS)
model = get_model()
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
#If starting with pre-trained weights.
#model.load_weights('???.hdf5')
history = model.fit(x_train, y_train_cat,
batch_size = 16,
verbose=1,
epochs=100,
validation_data=(X_test, y_test_cat),
class_weight=class_weights,
shuffle=False)
I used following approach to define class weights:我使用以下方法来定义类权重:
from sklearn.utils import class_weight
class_weights = class_weight.compute_class_weight('balanced',
np.unique(train_masks_reshaped_encoded),
train_masks_reshaped_encoded)
print("Class weights are...:", class_weights)
The result is class_weights : 0.276965 ,13.5112 ,5.80929,6.97915
.结果是
class_weights : 0.276965 ,13.5112 ,5.80929,6.97915
。
I am encountering ValueError
when I train my model.我在训练模型时遇到
ValueError
。 How can I possibly resolve it?我怎么可能解决它? Please suggest a better approach of using class weights if you think my approach is not viable.
如果您认为我的方法不可行,请提出使用类权重的更好方法。
File "/home/anaconda3/lib/python3.8/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 1185, in _configure_dataset_and_inferred_steps
if class_weight:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
My understanding of this error message is that numpy
does not know whether to evaluate an array as True
if any element is true, or as True
only if all elements are true.我对这条错误消息的理解是,
numpy
不知道是在任何元素为真时将数组评估为True
,还是仅当所有元素都为True
时才将数组评估为True
。 Hence, a ValueError
is returned because the boolean evaluation is ambiguous in this regard.因此,返回
ValueError
是因为布尔评估在这方面是不明确的。
Therefore, when evaluating an array, you should use a.any()
or a.all()
, as indicated in the error message attached.因此,在评估数组时,您应该使用
a.any()
或a.all()
,如附加的错误消息中所示。
The error is likely to be occurring from somewhere else in your code (not shared?) when you try to evaluate the class weights in a boolean context.当您尝试在布尔上下文中评估类权重时,错误很可能发生在代码中的其他地方(未共享?)。
I had the same problem but solved it with this!!!我有同样的问题,但用这个解决了它!!!
You have to zip it together as a dictionary你必须把它压缩成字典
Try The below code:试试下面的代码:
from sklearn.utils import class_weight从 sklearn.utils 导入 class_weight
class_weights = dict(zip(np.unique(train_masks_reshaped_encoded), class_weight.compute_class_weight('balanced', np.unique(train_masks_reshaped_encoded), train_masks_reshaped_encoded))) class_weights = dict(zip(np.unique(train_masks_reshape_encoded), class_weight.compute_class_weight('平衡', np.unique(train_masks_reshape_encoded), train_masks_reshape_encoded)))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.