Best way of extracting features from SuperPixels such as gabor and HOG features

Question

I've used the slic clustering algorithm to create superpixels of a biomedical image (whole slide image for the biomedical imaging experts). I want to extract different features, texture and spacial for the superpixels to create a feature representation and then feed that into a classifier (SVM, RF) to try and classify each superpixel as I have the labels for each one. The end goal is to to classify every superpixel and then use this to build a segmentation.

For each superpixel I draw a bounding box around it with a consistent size across all based on the average height and width of all superpixels since the distribution of sizes is fairly peaked around the average (some will have small parts cut out and others will include some padding. I have a couple of questions

1. In regards to the gabor filter for each superpixel I get a gabor feature with a value for each of its individual pixels , I then take the average of these to get a superpixel gabor feature value. Is this the right approach? code below

   def getGabor(img, ksize, sigma, theta, lamda, gamma, l, ktype): kernel=cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, l, ktype=ktype) fimg = cv2.filter2D(img, cv2.CV_8UC3, kernel) filteredImage=fimg.reshape(-1) return filteredImage def getGabors(img): ksize=5 thetas = list(map(lambda x: x*0.25*np.pi, [1, 2])) gabors=[] for theta in thetas: for sigma in (1,3): for lamda in np.arange(0, np.pi, np.pi*0.25): for gamma in (0.05, 0.5): gabor = getGabor(img.reshape(-1), ksize, sigma, theta, lamda, gamma, 0, cv2.CV_32F) . gabors.append(np.mean(gabor)) return gabors

2. How would this work with HOG? would I take the same approach of averaging the feature vector and how would I keep the HOG descriptor from being too large?

3. Would it be sensible to feed the superpixels into a CNN to learn the feature representation?

4. If anyone has worked with this kind of data before any suggestions over other useful image feature descriptors that would be a good approach for the type of data?

Any advice on building the features or what type of features to look at for superpixels would be much appreciated!

Thanks

Answer 1

I am not sure what the state of the art is in medical image segmentation, but superpixels, HOG and gabor sure sound like pre-2012, feature engineering approaches, before all the deep learning models appeared. The results are bound to depend heavily on the way the superpixels segment the image (probably not too robust).

Why not use a recent image segmentation CNN, like DeepLab ( https://github.com/tensorflow/models/tree/master/research/deeplab )? Just feed it with training examples of the segmentation (a lot of them, admittedly) and see if the model learns the right features itself. Chances are pretty high that it will.