[英]What is the best way of combining clinical and Image data in a Deep Learning approach?
I have CT images from patients and applied a CNN to those images to predict diseases. 我有来自患者的CT图像,并将CNN应用于这些图像以预测疾病。 I would like to combine my clinical data with my image data in a CNN approach, is that possible?
我想将我的临床数据与CNN方法中的图像数据结合起来,这可能吗? My clinical data has information like age, sex, dates, smoker, all of them are numbers, like 1 for smoker and 0 not.
我的临床数据包含年龄,性别,日期,吸烟者等信息,所有这些信息都是数字,例如1表示吸烟者,0表示不吸烟。
Have a look at, for example, this paper where they combine features from a CNN with text data. 例如,看一下这篇论文 ,它们将CNN的特征与文本数据结合起来。 In that paper, the CNN is already pre-trained (ie, the CNN is essentially a featurizer), but you could clearly learn all in one go.
在那篇论文中,CNN已经过预先训练(即,CNN本质上是一个特征),但你可以一口气清楚地学习。 The idea would always be to
这个想法总是如此
So the answer is "yes, certainly", the details depend on which framework you are using. 所以答案是“肯定,肯定”,细节取决于您使用的框架。
As far as I know CNN is extremely suitable for image data, but not for other data. 据我所知,CNN非常适合图像数据,但不适用于其他数据。
A solution to your problem would be to 'color' your images with the clinical data. 解决您的问题的方法是使用临床数据为您的图像“着色”。 (In image recognition CNNs, usually an input image is split into 3 color layers: red, grey and blue. See: http://cs231n.github.io/convolutional-networks/ )
(在图像识别CNN中,通常输入图像分为3个颜色层:红色,灰色和蓝色。请参阅: http : //cs231n.github.io/convolutional-networks/ )
Let's say your input data is a 32x32 pixel 8-bit greyscale image (so 1 color layer). 假设您的输入数据是一个32x32像素的8位灰度图像(因此为1色图层)。 I propose to add each clinical data variable as a 'color' layer.
我建议将每个临床数据变量添加为“颜色”层。 All input values in the same color layer should be the same.
同一颜色层中的所有输入值应相同。
Whether each layer should be the same size as the image, or if you can get away with a single pixel, I'm not sure, but at least you can treat the clinical data as an 'image' alongside the CT images. 每个图层是否应该与图像大小相同,或者如果您可以使用单个像素,我不确定,但至少您可以将临床数据视为CT图像旁边的“图像”。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.