Select complex colour range for skin-colour detection in OpenCV with Python

Question

I am trying to make skin-colour detection program. Basically, it takes video from webcamera, and then creates a mask, after which only the skin should be visible. I have found a criterium for detecting skin-colour ranges in a paper. It looks like this:

The skin colour at uniform daylight illumination rule is defined as (R > 95 ) AND (G > 40 ) AND (B > 20 ) AND (max{R, G, B} - min{R, G, B} 15) AND (|R - G| > 15 ) AND (R > G) AND (R > B) (1) while the skin colour under flashlight or daylight lateral illumination rule is given by (R > 220 ) AND (G > 210 ) AND (B > 170 ) AND (|R - G| <= 15 ) AND (R > B) AND (G > B)

What I did in Python is:

def check(list):
return ( ( (list[2]>95) and (list[1]>40) and (list[0]>20)) and ((max(list)-min(list))>15)       
        and (abs(list[2]-list[1])>15) and (list[2]>list[1]) and (list[2]>list[0]))

def check2(list):
    return (list[2]>220) and (list[1]>210) and (list[0]>170) and (abs(list[2]-list[1])<=15) and ((list[2]>list[0]) and (list[1]>list[0]))

(grabbed, frame) = camera.read()
img=frame   
img=img.tolist()
skinmask =  [[(1 if (check(list) or check2(list)) else 0) for list in l1] for l1 in img]
mask=np.array(skinmask, dtype = "uint8")
skin = cv2.bitwise_and(frame, frame, mask = mask)
cv2.imshow("images", np.hstack([frame, skin]))

But it is not, what I really expected. It slows the process. I found cv2.inRange(image, lower, upper) but it can not handle such complex colour-range rules. Is there any other method to do this in more efficient way?

Answer 1

The bottleneck here is that you are converting the numpy array frame returned by camera.read() (ie the snapshot made by the camera) into an ordinary Python list. You're then iterating over the elements using ordinary for-loops, which are relatively slow by comparison.

What you should do is use numpy's vectorized operations to get the execution time down.

Your example, slightly rewritten ( list -> px ) for clarity and corrected for the color channels (the red pixel is at px[0] , not px[2] ):

import cv2
import numpy as np

camera = cv2.VideoCapture(0)
(grabbed, frame) = camera.read()

def check(px):
    R, G, B = px
    return ( ((R > 95) and (G > 40) and (B > 20))
        and ((max(px)-min(px))>15) and (abs(R - G) > 15) and
        (R > G) and (R > B))

def check2(px):
    R, G, B = px
    return ((R >220) and (G > 210) and (B > 170) and
        (abs(R - G) <= 15) and (R > B) and (G > B))

def iterate_over_list(img):  # your method
    img = img.tolist()
    skinmask =  [[(1 if (check(px) or check2(px)) else 0) for px in row] for row in img]
    return skinmask

This can be rewritten in a vectorized form as:

def vectorized_form(img):
    R,G,B = [img[:,:,x] for x in range(3)]
    delta15 = np.abs(R.astype(np.int8) - G.astype(np.int8)) > 15  # watch out for np.abs(R-G): because of the UNsigned numbers, they could get clipped!
    more_R_than_B = (R > B)
    is_skin_coloured_during_daytime = ((R > 95) & (G > 40) & (B > 20) &
        (img.ptp(axis=-1) > 15) & delta15 & (R > G) & more_R_than_B)
    is_skin_coloured_under_flashlight = ((R > 220) & (G > 210) & (B > 170) &
        ~delta15 & more_R_than_B & (G > B))
    return np.logical_or(is_skin_coloured_during_daytime, is_skin_coloured_under_flashlight)

Note that you can get rid of at least one logical and operation: more_R_than_B appears in each of the checks, which are then combined using the logical or operation. In truth table syntax: (A & B) | (C & B) == (A | C) & B (A & B) | (C & B) == (A | C) & B . But now I'm micro-optimizing, and I wanted to preserve the original form, because it would show the 1-to-1 map with the paper you cited.

Timing considerations on my system show a speed increase of a factor ~19. Remark, my test image had shape (480, 640, 3) . The speed increase will be considerably larger for larger images, because in your method, you iterate over the pixels using the standard Python for-loops, whereas I'm simply using vectorized routines.

In [27]: %timeit iterate_over_list(frame)
1 loops, best of 3: 321 ms per loop

In [28]: %timeit vectorized(frame)
100 loops, best of 3: 16.8 ms per loop