在保留特定指标的同时最小化图像区域

Question

I have a picture and I want to find the crop with the minimal area that at the same time retains a certain percentage of the edge energy.

My take on this was to formulate this as a optimization problem and let scipy's constrained optimizers solve it [code see below]. This apparently is problematic since it is an integer problem (croping takes the integer coordinates of the top-left and down-right corners as parameters). And indeed fmin_cobyla fails to find a solution after some 20s runtime, while fmin_slsqp fails after one iteration with " Singular matrix C in LSQ subproblem (Exit mode 6) ".

Any ideas on how I might tackle this problem otherwise? Is there by chance a library that handles optimization problems on images?

---

from skimage.filters import sobel
from PIL import Image
from scipy.optimize import fmin_slsqp

def objective(x):
    # minimize the area
    return abs((x[2] - x[0]) * (x[3] - x[1]))

def create_ratio_constr(img):
    def constr(x):
        # 81% of the image energy should be contained
        x = tuple(map(int, x))
        crop = img.crop((x[0], x[1], x[2], x[3]))
        area_ratio = round(sum(list(crop.getdata())) /
                           float(sum(list(img.getdata()))), 2)
        if area_ratio == 0.81:
            return 0.0
        return -1
    return constr

def borders_constr(x):
    x = tuple(map(int, x))
    # 1st point is up and left of 2nd point
    rectangle = x[0] < x[2] and x[1] < x[3]

    # only positive values valid
    positive = x[0] > 0 and x[1] > 0

    if rectangle and positive:
        return 0.0
    return -1

img = Image.open("/some/path.jpg")
# get the edges
edges = Image.fromarray(sobel(img.convert("L")))

ratio_constr = create_ratio_constr(edges)
x = fmin_slsqp(objective,
               (0, 0, edges.size[0]-1, edges.size[1]-1),
               [borders_constr, ratio_constr],
               disp=1)

print x

Answer 1

I would probably ignore the integer requirement for the corners of the cropped area and solve the relaxed problem. Then consider moving the cropped edges in or out to the nearest whole pixel. If it is for aesthetic purposes, the +/- part pixel probably doesn't matter. If it has to be correct, there are only four edges to consider in two places each so should not be a big deal to find the one out of 16 choices which is best.