How to fix constraints for allocation optimisation in PuLP python

Question

Background:

I am trying to allocate customers Ci to financial advisers Pj. Each customer has a policy value xi. I'm assuming that the number of customers (n) allocated to each adviser is the same, and that the same customer cannot be assigned to multiple advisers. Therefore each partner will have an allocation of policy values like so:

P1=[x1,x2,x3] , P2=[x4,x5,x6], P3=[x7,x8,x9]

I am trying to find the optimal allocation to minimise dispersion in fund value between the advisers. I am defining dispersion as the difference between the adviser with the highest fund value (z_max) and the lowest fund value (z_min).

The formulation for this problem is therefore:

where yij=1 if we allocate customer Ci to adviser Pj, 0 otherwise

The first constraint says that zmax has to be greater than or equal to each policy value; since the objective function encourages smaller values of zmax, this means that zmax will equal the largest policy value. Similarly, the second constraint sets zmin equal to the smallest policy value. The third constraint says that each customer must be assigned to exactly one adviser. The fourth says that each adviser must have n customers assigned to him/her. Credit: @LarrySnyder610

Problem:

When implementing this problem in PulP, I expect 1740 (nxp) customers to be allocated across the 173 advisers based on constraint 3 and 4. However 72036 and no optimal allocation is obtained.

import random 
import pandas as pd
import pulp 


=============================================================================
# SAMPLE DATA 
=============================================================================

n = 10 # number of customers for each financial adviser
c = 414 #number of customers 
p = 174 #number of financial adviser
policy_values = random.sample(range(1, 1000000), c)


set_I = range(c)
set_J = range(p)
set_N = range(n)
x = {i: policy_values[i] for i in set_I} #customer policy values
y = {(i,j): random.randint(0, 1) for i in set_I for j in set_J} # allocation dummies 


model = pulp.LpProblem("Allocation Model", pulp.LpMinimize)

# =============================================================================
# DECISION VARIABLES
# =============================================================================

y_vars  = {(i,j): pulp.LpVariable(cat=pulp.LpBinary, name="y_{0}_{1}".format(i,j)) for i in set_I for j in set_J}
z_max = pulp.LpVariable("Max Policy Value", 0)
z_min = pulp.LpVariable("Min Policy Value", 0)

# =============================================================================
# OBJECTIVE FUNCTION
# =============================================================================

model += z_max - z_min

# =============================================================================
# CONSTRAINTS
# =============================================================================
model += {j: pulp.lpSum(y_vars[i,j] * x[i] for i in set_I) for j in set_J} <= z_max # constraint 1
model += {j: pulp.lpSum(y_vars[i,j] * x[i] for i in set_I) for j in set_J} >= z_min # constraint 2
model += {i: pulp.lpSum(y_vars[i,j] for j in set_J) for i in set_I} == 1 # constraint 3
model += {j: pulp.lpSum(y_vars[i,j] for i in set_I) for j in set_J} == n #constraint 4

# =============================================================================
# SOLVE MODEL
# =============================================================================

model.solve()
print('Optimised model status: '+str(pulp.LpStatus[model.status]))

count=0
for v in model.variables():
    if v.varValue == 1.0:
        count+=1
        #print(v.name, "=", v.varValue)
print(count)
#>>> 72036 # expecting 1740

print('Optimal difference between highest and lowest summed policy_value: ' + str(pulp.value(model.objective)))

Do I need to make changes to the objective function/constraints to implement the above equations?

EDIT: Below snippet to try and implement @Erwin Kalvelagen's suggested changes. Still has extremely long durations for higher values of n,p and c:

y_sum = {}

# =============================================================================
# DECISION VARIABLES
# =============================================================================

model = pulp.LpProblem("Allocation Model", pulp.LpMinimize)

y_vars = pulp.LpVariable.dicts('y_vars',((i,j) for i in set_I for j in set_J), lowBound=0, upBound = 1, cat=pulp.LpInteger)
z_max = pulp.LpVariable("Max Policy Value")
z_min = pulp.LpVariable("Min Policy Value")
for j in set_J:
    y_sum[j] = pulp.lpSum([y_vars[i,j] * x[i] for i in set_I])

# =============================================================================
# OBJECTIVE FUNCTION
# =============================================================================

model += z_max - z_min

# =============================================================================
# CONSTRAINTS
# =============================================================================

for j in set_J:
    model += pulp.lpSum([y_vars[i,j] for i in set_I]) == n 
    model += y_sum[j] <= z_max
    model += y_sum[j] >= z_min 


for i in set_I:
    model += pulp.lpSum([y_vars[i,j] for j in set_J]) == 1 

Answer 1

Some hints:

1. Debug by print(model)
2. Start with a small data set

3. The constraints are not correctly formulated. It should be something like

    for j in set_J: model += 1.0e-6 * pulp.lpSum(y_vars[i,j] * x[i] for i in set_I) <= z_max # constraint 1 model += 1.0e-6 * pulp.lpSum(y_vars[i,j] * x[i] for i in set_I) >= z_min # constraint 2 model += pulp.lpSum(y_vars[i,j] for i in set_I) == n #constraint 4 for i in set_I: model += pulp.lpSum(y_vars[i,j] for j in set_J) == 1 # constraint 3 

   4. The model will be infeasible if n*p <> c
   5. Detail: we should probably rewrite constraints 1 and 2. Repeating long summations will create a large number of nonzero elements.

Answer 2

I don't think you can use that format to add constraints. Try this format instead:

for j in set_J:
    model += pulp.lpSum([y_vars[i,j] * x[i] for i in set_I]) <= z_max

etc.

Note also the [...] inside the lpSum(...) .

Finally, I don't think you can declare variables the way you did. I usually use LpVariable.dicts() , as in:

y_vars = pulp.lpVariable.dicts('y_vars', set_I, 0, 1, pulp.LpInteger)