How to backtest portfolio compositions using backtrader?

Question

I have a csv file / pandas dataframe which looks like this. It contains various portfolio compositions for a portfolio which is re-balanced everyday according to my own calculations.

date        asset   percentage
4-Jan-21    AAPL    12.00%
4-Jan-21    TSM     1.00%
4-Jan-21    IBM     31.00%
4-Jan-21    KO      15.00%
4-Jan-21    AMD     41.00%
5-Jan-21    DELL    23.00%
5-Jan-21    TSM     12.20%  
5-Jan-21    IBM     15.24%  
5-Jan-21    KO      1.50%   
5-Jan-21    NKE     7.50%   
5-Jan-21    TSLA    9.50%   
5-Jan-21    CSCO    3.30%   
5-Jan-21    JPM     27.76%  
6-Jan-21    AMD     45% 
6-Jan-21    BA      0.50%   
6-Jan-21    ORCL    54.50%  
7-Jan-21    AAPL    50.00%  
7-Jan-21    KO      50.00%  
...

I want to test a strategy with a 12 asset portfolio.

AAPL,TSM,IBM,KO,AMD,DELL,NKE,TSLA,CSCO,JPM,BA,ORCL

So let's say on 4Jan2021, the portfolio's composition would be 12% in apple, 1% in TSM.. etc. I want to be able to check the prices and know how many I should be holding.

The next day, 5Jan2021, the composition will change to 23% in Dell.. etc, if the stock isn't in this list means its 0% for that day.

I have been looking at backtrader as a backtesting platform, however, the code I have seen in the repo mostly shows how to do stuff with indicators, like SMA cross over, RSI...

My question is: Is it possible to create and test a portfolio based on these compositions I have so I can check the return of this strategy? It would check this frame, and know how many stocks in a ticker to buy or sell on that particular day.

So the universe of stocks I am buying or sell is AAPL,TSM,IBM,KO,AMD,DELL,NKE,TSLA,CSCO,JPM,BA,ORCL

So on 4-Jan-21 it might look like,

dictionary['4Jan2021'] = {'AAPL':0.12,
                          'TSM':0.01,
                          'IBM':0.31,
                          'KO':0.15,
                          'AMD':0.41,}

On 5-Jan-21 it will look like,

dictionary['5Jan2021'] = {'DELL':0.23,
                          'TSM':0.122,
                          'IBM':0.1524,
                          'KO':0.015,
                          'NKE':0.075,
                          'TSLA':0.095,
                          'CSCO':0.033,
                          'JPM':0.2776,}    

If the ticker isnt there means its 0%. The portfolio composition needs to change everyday.

Answer 1

The first thing you will want to do it load your targets with your datas. I like personally to attach the target to the dataline as I add it to backtrader.

tickers = {"FB": 0.25, "MSFT": 0.4, "TSLA": 0.35}

for ticker, target in tickers.items():
    data = bt.feeds.YahooFinanceData(
        dataname=ticker,
        timeframe=bt.TimeFrame.Days,
        fromdate=datetime.datetime(2019, 1, 1),
        todate=datetime.datetime(2020, 12, 31),
        reverse=False,
    )
    data.target = target
    cerebro.adddata(data, name=ticker)

In next you will want to go through each data, and determine the current allocation. If the current allocation is too far from the desired allocation (threshold) you trade all datas.

Notice there is a buffer variable. This will reduce the overall value of the account for calculating units to trade. This helps avoid margin.

You will use a dictionary to track this information.

def next(self):
    track_trades = dict()
    total_value = self.broker.get_value() * (1 - self.p.buffer)

    for d in self.datas:
        track_trades[d] = dict()
        value = self.broker.get_value(datas=[d])
        allocation = value / total_value
        units_to_trade = (d.target - allocation) * total_value / d.close[0]
        track_trades[d]["units"] = units_to_trade

        # Can check to make sure there is enough distance away from ideal to trade.
        track_trades[d]["threshold"] = abs(d.target - allocation) > self.p.threshold

Check all the thresholds to determine if trading. If any of datas need trading, then all need trading.

rebalance = False
for values in track_trades.values():
    if values['threshold']:
        rebalance = True

if not rebalance:
    return

Finally, execute your trades. Always sell first to generate cash in the account and avoid margins.

# Sell shares first
for d, value in track_trades.items():
    if value["units"] < 0:
        self.sell(d, size=value["units"])

# Buy shares second
for d, value in track_trades.items():
    if value["units"] > 0:
        self.buy(d, size=value["units"])

Here is the all of the code for your reference.

import datetime
import backtrader as bt

class Strategy(bt.Strategy):

    params = (
        ("buffer", 0.05),
        ("threshold", 0.025),
    )

    def log(self, txt, dt=None):
        """ Logging function fot this strategy"""
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        print("%s, %s" % (dt.date(), txt))

    def print_signal(self):
        self.log(
            f"o {self.datas[0].open[0]:7.2f} "
            f"h {self.datas[0].high[0]:7.2f} "
            f"l {self.datas[0].low[0]:7.2f} "
            f"c {self.datas[0].close[0]:7.2f} "
            f"v {self.datas[0].volume[0]:7.0f} "
        )

    def notify_order(self, order):
        """ Triggered upon changes to orders. """
        # Suppress notification if it is just a submitted order.
        if order.status == order.Submitted:
            return

        # Print out the date, security name, order number and status.
        type = "Buy" if order.isbuy() else "Sell"
        self.log(
            f"{order.data._name:<6} Order: {order.ref:3d} "
            f"Type: {type:<5}\tStatus"
            f" {order.getstatusname():<8} \t"
            f"Size: {order.created.size:9.4f} Price: {order.created.price:9.4f} "
            f"Position: {self.getposition(order.data).size:5.2f}"
        )
        if order.status == order.Margin:
            return

        # Check if an order has been completed
        if order.status in [order.Completed]:
            self.log(
                f"{order.data._name:<6} {('BUY' if order.isbuy() else 'SELL'):<5} "
                # f"EXECUTED for: {dn} "
                f"Price: {order.executed.price:6.2f} "
                f"Cost: {order.executed.value:6.2f} "
                f"Comm: {order.executed.comm:4.2f} "
                f"Size: {order.created.size:9.4f} "
            )

    def notify_trade(self, trade):
        """Provides notification of closed trades."""
        if trade.isclosed:
            self.log(
                "{} Closed: PnL Gross {}, Net {},".format(
                    trade.data._name,
                    round(trade.pnl, 2),
                    round(trade.pnlcomm, 1),
                )
            )

    def next(self):
        track_trades = dict()
        total_value = self.broker.get_value() * (1 - self.p.buffer)

        for d in self.datas:
            track_trades[d] = dict()
            value = self.broker.get_value(datas=[d])
            allocation = value / total_value
            units_to_trade = (d.target - allocation) * total_value / d.close[0]
            track_trades[d]["units"] = units_to_trade

            # Can check to make sure there is enough distance away from ideal to trade.
            track_trades[d]["threshold"] = abs(d.target - allocation) > self.p.threshold

        rebalance = False
        for values in track_trades.values():
            if values['threshold']:
                rebalance = True

        if not rebalance:
            return

        # Sell shares first
        for d, value in track_trades.items():
            if value["units"] < 0:
                self.sell(d, size=value["units"])

        # Buy shares second
        for d, value in track_trades.items():
            if value["units"] > 0:
                self.buy(d, size=value["units"])


if __name__ == "__main__":

    cerebro = bt.Cerebro()

    tickers = {"FB": 0.25, "MSFT": 0.4, "TSLA": 0.35}

    for ticker, target in tickers.items():
        data = bt.feeds.YahooFinanceData(
            dataname=ticker,
            timeframe=bt.TimeFrame.Days,
            fromdate=datetime.datetime(2019, 1, 1),
            todate=datetime.datetime(2020, 12, 31),
            reverse=False,
        )
        data.target = target
        cerebro.adddata(data, name=ticker)

    cerebro.addstrategy(Strategy)

    # Execute
    cerebro.run()

####################################
############# EDIT ###############
####################################
There was an additional requiest for adding in variable allocations per day per security. The following code accomplishes that.

import datetime
import backtrader as bt


class Strategy(bt.Strategy):

    params = (
        ("buffer", 0.05),
        ("threshold", 0.025),
    )

    def log(self, txt, dt=None):
        """ Logging function fot this strategy"""
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        print("%s, %s" % (dt.date(), txt))

    def print_signal(self):
        self.log(
            f"o {self.datas[0].open[0]:7.2f} "
            f"h {self.datas[0].high[0]:7.2f} "
            f"l {self.datas[0].low[0]:7.2f} "
            f"c {self.datas[0].close[0]:7.2f} "
            f"v {self.datas[0].volume[0]:7.0f} "
        )

    def notify_order(self, order):
        """ Triggered upon changes to orders. """
        # Suppress notification if it is just a submitted order.
        if order.status == order.Submitted:
            return

        # Print out the date, security name, order number and status.
        type = "Buy" if order.isbuy() else "Sell"
        self.log(
            f"{order.data._name:<6} Order: {order.ref:3d} "
            f"Type: {type:<5}\tStatus"
            f" {order.getstatusname():<8} \t"
            f"Size: {order.created.size:9.4f} Price: {order.created.price:9.4f} "
            f"Position: {self.getposition(order.data).size:5.2f}"
        )
        if order.status == order.Margin:
            return

        # Check if an order has been completed
        if order.status in [order.Completed]:
            self.log(
                f"{order.data._name:<6} {('BUY' if order.isbuy() else 'SELL'):<5} "
                # f"EXECUTED for: {dn} "
                f"Price: {order.executed.price:6.2f} "
                f"Cost: {order.executed.value:6.2f} "
                f"Comm: {order.executed.comm:4.2f} "
                f"Size: {order.created.size:9.4f} "
            )

    def notify_trade(self, trade):
        """Provides notification of closed trades."""
        if trade.isclosed:
            self.log(
                "{} Closed: PnL Gross {}, Net {},".format(
                    trade.data._name,
                    round(trade.pnl, 2),
                    round(trade.pnlcomm, 1),
                )
            )

    def __init__(self):
        for d in self.datas:
            d.target = {
                datetime.datetime.strptime(date, "%d-%b-%y").date(): allocation
                for date, allocation in d.target.items()
            }

    def next(self):
        date = self.data.datetime.date()
        track_trades = dict()
        total_value = self.broker.get_value() * (1 - self.p.buffer)

        for d in self.datas:
            if date not in d.target:
                if self.getposition(d):
                    self.close(d)
                continue
            target_allocation = d.target[date]
            track_trades[d] = dict()
            value = self.broker.get_value(datas=[d])
            current_allocation = value / total_value
            net_allocation = target_allocation - current_allocation
            units_to_trade = (
                (net_allocation) * total_value / d.close[0]
            )
            track_trades[d]["units"] = units_to_trade

            # Can check to make sure there is enough distance away from ideal to trade.
            track_trades[d]["threshold"] = abs(net_allocation) > self.p.threshold

        rebalance = False
        for values in track_trades.values():
            if values["threshold"]:
                rebalance = True

        if not rebalance:
            return

        # Sell shares first
        for d, value in track_trades.items():
            if value["units"] < 0:
                self.sell(d, size=value["units"])

        # Buy shares second
        for d, value in track_trades.items():
            if value["units"] > 0:
                self.buy(d, size=value["units"])


if __name__ == "__main__":

    cerebro = bt.Cerebro()

    allocations = [
        ("AAPL", "4-Jan-21", 0.300),
        ("TSM", "4-Jan-21", 0.200),
        ("IBM", "4-Jan-21", 0.300),
        ("KO", "4-Jan-21", 0.2000),
        ("AMD", "4-Jan-21", 0.1000),
        ("DELL", "5-Jan-21", 0.200),
        ("TSM", "5-Jan-21", 0.20),
        ("IBM", "5-Jan-21", 0.1),
        ("KO", "5-Jan-21", 0.1),
        ("NKE", "5-Jan-21", 0.15),
        ("TSLA", "5-Jan-21", 0.10),
        ("CSCO", "5-Jan-21", 0.050),
        ("JPM", "5-Jan-21", 0.1),
        ("AMD", "6-Jan-21", 0.25),
        ("BA", "6-Jan-21", 0.25),
        ("ORCL", "6-Jan-21", 0.50),
        ("AAPL", "7-Jan-21", 0.5000),
        ("KO", "7-Jan-21", 0.5000),
    ]
    ticker_names = list(set([alls[0] for alls in allocations]))
    targets = {ticker: {} for ticker in ticker_names}
    for all in allocations:
        targets[all[0]].update({all[1]: all[2]})

    for ticker, target in targets.items():
        data = bt.feeds.YahooFinanceData(
            dataname=ticker,
            timeframe=bt.TimeFrame.Days,
            fromdate=datetime.datetime(2020, 12, 21),
            todate=datetime.datetime(2021, 1, 8),
            reverse=False,
        )
        data.target = target
        cerebro.adddata(data, name=ticker)

    cerebro.addstrategy(Strategy)
    cerebro.broker.setcash(1000000)

    # Execute
    cerebro.run()