Python: Optimize weights in portfolio

Question

I have the following dataframe with weights:

df = pd.DataFrame({'a': [0.1, 0.5, 0.1, 0.3], 'b': [0.2, 0.4, 0.2, 0.2], 'c': [0.3, 0.2, 0.4, 0.1],
           'd': [0.1, 0.1, 0.1, 0.7], 'e': [0.2, 0.1, 0.3, 0.4], 'f': [0.7, 0.1, 0.1, 0.1]})

and then I normalize each row using:

df = df.div(df.sum(axis=1), axis=0)

I want to optimize the normalized weights of each row such that no weight is less than 0 or greater than 0.4.

If the weight is greater than 0.4, it will be clipped to 0.4 and the additional weight will be distributed to the other entries in a pro-rata fashion (meaning the second largest weight will receive more weight so it gets close to 0.4, and if there is any remaining weight, it will be distributed to the third and so on).

Can this be done using the "optimize" function?

Thank you.

Answer 1

Unfortunately, I can only find a loop solution to this problem. When you trim off the excess weight and redistribute it proportionally, the underweight may go over the limit. Then they have to be trimmed off. And the cycle keep repeating until no value is overweight.

The code below works if all items are greater than 0. Handling items with 0 weight is tricky because you can't shift the excess weight to them (as per your proportional formula). You have to define a "seed" weight for them somehow.

# The original data frame. No normalization yet
df = pd.DataFrame({
    'a': [0.1, 0.5, 0.1, 0.3],
    'b': [0.2, 0.4, 0.2, 0.2],
    'c': [0.3, 0.2, 0.4, 0.1],
    'd': [0.1, 0.1, 0.1, 0.7],
    'e': [0.2, 0.1, 0.3, 0.4],
    'f': [0.7, 0.1, 0.1, 0.1]}
)

values = df.to_numpy()
normalized = values / values.sum(axis=1)[:, None]
max_weight = 0.4

for i in range(len(values)):
    row = normalized[i]
    while True:
        overweight = row > max_weight
        if not overweight.any():
            break

        # Calculate the excess weight
        excess_weight = row[overweight].sum() - (max_weight * overweight.sum())
        # Distribute this excess weight proportionally to the underweight items
        row[~overweight] += excess_weight / row[~overweight].sum() * row[~overweight]
        # Clip the overweight to max_weight
        row[overweight] = max_weight

    # Floating point math is not exact. We test for "close to 1"
    # as opposed to "exactly equal 1"
    assert np.isclose(row.sum(), 1)
    normalized[i] = row

values = normalized * values.sum(axis=1)[:, None]

Output:

> df
  a   b   c   d   e   f
0.1 0.2 0.3 0.1 0.2 0.7
0.5 0.4 0.2 0.1 0.1 0.1
0.1 0.2 0.4 0.1 0.3 0.1
0.3 0.2 0.1 0.7 0.4 0.1

> pd.DataFrame(values, columns=df.columns)
       a        b    c        d        e    f
0.106667 0.213333 0.32 0.106667 0.213333 0.64
0.500000 0.400000 0.20 0.100000 0.100000 0.10
0.100000 0.200000 0.40 0.100000 0.300000 0.10
0.300000 0.200000 0.10 0.700000 0.400000 0.10