Rolling over Futures
Not every provider offers acontinuous future for the instruments with whichone can trade. Sometimes the data offered is that of the still valid expirationdates, i.e.: those still being traded
This is not so helpful when it comes tobacktesting because the data isscattered over several different instruments which additionally …overlapin time.
Being able to properly join the data of those instruments, from the past, intoa continuous stream alleviates the pain. The problem:
- There is no law as to how best join the different expiration dates into a continuous future
Some literature, courtesy ofSierraChart at:
The RollOver Data Feed
backtrader has added with1.8.10.99 the possibility to join futures’ datafrom different expiration dates into a continuous future:
importbacktraderasbtcerebro=bt.Cerebro()data0=bt.feeds.MyFeed(dataname='Expiry0')data1=bt.feeds.MyFeed(dataname='Expiry1')...dataN=bt.feeds.MyFeed(dataname='ExpiryN')drollover=cerebro.rolloverdata(data0,data1,...,dataN,name='MyRoll',**kwargs)cerebro.run()
Note
The possible**kwargs are explained below
It can also be done by directly accessing theRollOver feed (which ishelpful if subclassing is done):
importbacktraderasbtcerebro=bt.Cerebro()data0=bt.feeds.MyFeed(dataname='Expiry0')data1=bt.feeds.MyFeed(dataname='Expiry1')...dataN=bt.feeds.MyFeed(dataname='ExpiryN')drollover=bt.feeds.RollOver(data0,data1,...,dataN,dataname='MyRoll',**kwargs)cerebro.adddata(drollover)cerebro.run()
Note
The possible**kwargs are explained below
Note
When usingRollOver the name is assigned usingdataname. Thisis the standard parameter used for all data feeds to pass thename/ticker. In this case it is reused to assign a common name tothe complete set of rolled over futures.
In the case ofcerebro.rolloverdata, the name is assigned to afeed usingname, which is already one named argument of that method
Bottomline:
Data Feeds are created as usual butARE NOT added tocerebro
Those data feeds are given as input tobt.feeds.RollOver
Adataname is also given, mostly for identification purposes.
Thisroll over data feed is then added tocerebro
Options for the Roll-Over
Two parameters are provided to control the roll-over process
checkdate (default:None)
This must be acallable with the following signature:
Where:
Expected Return Values:
True: as long as the callable returns this, a switchover can happen to the next future
If a commodity expires on the 3rd Friday of March,checkdate couldreturnTrue for the entire week in which the expiration takesplace.
False: the expiration cannot take place
checkcondition (default:None)
Note
This will only be called ifcheckdate has returnedTrue
IfNone this will evaluate toTrue (execute roll over) internally
Else this must be acallable with this signature:
Where:
Expected Return Values:
True: roll-over to the next future
Following with the example fromcheckdate, this could say that theroll-over can only happen if thevolume fromd0 is already lessthan the volume fromd1
False: the expiration cannot take place
SubclassingRollOver
If specifying thecallables isn’t enough, there is always the chance tosubclassRollOver. The methods to subclass:
def _checkdate(self, dt, d):
Which matches thesignature of the parameter of the same name above. Theexpected return values are also the saame.
def _checkcondition(self, d0, d1)
Which matches thesignature of the parameter of the same name above. Theexpected return values are also the saame.
Let’s Roll
Note
The default behavior in the sample is to usecerebro.rolloverdata. This can be changed by passing the-no-cerebro flag. In this case the sample usesRollOver andcerebro.adddata
The implementation includes a sample which is available in thebacktradersources.
Futures concatenation
Let’s start by looking at a pure concatenation by running the sample with noarguments.
$ ./rollover.pyLen, Name, RollName, Datetime, WeekDay, Open, High, Low, Close, Volume, OpenInterest0001, FESX, 199FESXM4, 2013-09-26, Thu, 2829.0, 2843.0, 2829.0, 2843.0, 3.0, 1000.00002, FESX, 199FESXM4, 2013-09-27, Fri, 2842.0, 2842.0, 2832.0, 2841.0, 16.0, 1101.0...0176, FESX, 199FESXM4, 2014-06-20, Fri, 3315.0, 3324.0, 3307.0, 3322.0, 134777.0, 520978.00177, FESX, 199FESXU4, 2014-06-23, Mon, 3301.0, 3305.0, 3265.0, 3285.0, 730211.0, 3003692.0...0241, FESX, 199FESXU4, 2014-09-19, Fri, 3287.0, 3308.0, 3286.0, 3294.0, 144692.0, 566249.00242, FESX, 199FESXZ4, 2014-09-22, Mon, 3248.0, 3263.0, 3231.0, 3240.0, 582077.0, 2976624.0...0306, FESX, 199FESXZ4, 2014-12-19, Fri, 3196.0, 3202.0, 3131.0, 3132.0, 226415.0, 677924.00307, FESX, 199FESXH5, 2014-12-22, Mon, 3151.0, 3177.0, 3139.0, 3168.0, 547095.0, 2952769.0...0366, FESX, 199FESXH5, 2015-03-20, Fri, 3680.0, 3698.0, 3672.0, 3695.0, 147632.0, 887205.00367, FESX, 199FESXM5, 2015-03-23, Mon, 3654.0, 3655.0, 3608.0, 3618.0, 802344.0, 3521988.0...0426, FESX, 199FESXM5, 2015-06-18, Thu, 3398.0, 3540.0, 3373.0, 3465.0, 1173246.0, 811805.00427, FESX, 199FESXM5, 2015-06-19, Fri, 3443.0, 3499.0, 3440.0, 3488.0, 104096.0, 516792.0
This usescerebro.chaindata and the result should be clear:
Futures roll-over with no checks
Let’s execute with--rollover
$ ./rollover.py --rollover --plotLen, Name, RollName, Datetime, WeekDay, Open, High, Low, Close, Volume, OpenInterest0001, FESX, 199FESXM4, 2013-09-26, Thu, 2829.0, 2843.0, 2829.0, 2843.0, 3.0, 1000.00002, FESX, 199FESXM4, 2013-09-27, Fri, 2842.0, 2842.0, 2832.0, 2841.0, 16.0, 1101.0...0176, FESX, 199FESXM4, 2014-06-20, Fri, 3315.0, 3324.0, 3307.0, 3322.0, 134777.0, 520978.00177, FESX, 199FESXU4, 2014-06-23, Mon, 3301.0, 3305.0, 3265.0, 3285.0, 730211.0, 3003692.0...0241, FESX, 199FESXU4, 2014-09-19, Fri, 3287.0, 3308.0, 3286.0, 3294.0, 144692.0, 566249.00242, FESX, 199FESXZ4, 2014-09-22, Mon, 3248.0, 3263.0, 3231.0, 3240.0, 582077.0, 2976624.0...0306, FESX, 199FESXZ4, 2014-12-19, Fri, 3196.0, 3202.0, 3131.0, 3132.0, 226415.0, 677924.00307, FESX, 199FESXH5, 2014-12-22, Mon, 3151.0, 3177.0, 3139.0, 3168.0, 547095.0, 2952769.0...0366, FESX, 199FESXH5, 2015-03-20, Fri, 3680.0, 3698.0, 3672.0, 3695.0, 147632.0, 887205.00367, FESX, 199FESXM5, 2015-03-23, Mon, 3654.0, 3655.0, 3608.0, 3618.0, 802344.0, 3521988.0...0426, FESX, 199FESXM5, 2015-06-18, Thu, 3398.0, 3540.0, 3373.0, 3465.0, 1173246.0, 811805.00427, FESX, 199FESXM5, 2015-06-19, Fri, 3443.0, 3499.0, 3440.0, 3488.0, 104096.0, 516792.0
The same behavior. It can clearly be seen that contract changes are being madeon the 3rd Friday of either Mar, Jun, Sep, Dec.
But this is mostly WRONG.backtrader cannot know it, but the author knows thattheEuroStoxx 50 futures stop trading at12:00 CET. So even if there is adaily bar for the 3rd Friday of the expiration month, the change is happeningtoo late.
Changing during the Week
Acheckdate callable is implemented in the sample, which calculates the dateof expiration for the currently active contract.
checkdate will allow a roll over as soon as the week of the 3rd Friday ofthe month is reached (it may beTuesday if for exampleMonday is a bank holiday)
$ ./rollover.py --rollover --checkdate --plotLen, Name, RollName, Datetime, WeekDay, Open, High, Low, Close, Volume, OpenInterest0001, FESX, 199FESXM4, 2013-09-26, Thu, 2829.0, 2843.0, 2829.0, 2843.0, 3.0, 1000.00002, FESX, 199FESXM4, 2013-09-27, Fri, 2842.0, 2842.0, 2832.0, 2841.0, 16.0, 1101.0...0171, FESX, 199FESXM4, 2014-06-13, Fri, 3283.0, 3292.0, 3253.0, 3276.0, 734907.0, 2715357.00172, FESX, 199FESXU4, 2014-06-16, Mon, 3261.0, 3275.0, 3252.0, 3262.0, 180608.0, 844486.0...0236, FESX, 199FESXU4, 2014-09-12, Fri, 3245.0, 3247.0, 3220.0, 3232.0, 650314.0, 2726874.00237, FESX, 199FESXZ4, 2014-09-15, Mon, 3209.0, 3224.0, 3203.0, 3221.0, 153448.0, 983793.0...0301, FESX, 199FESXZ4, 2014-12-12, Fri, 3127.0, 3143.0, 3038.0, 3042.0, 1409834.0, 2934179.00302, FESX, 199FESXH5, 2014-12-15, Mon, 3041.0, 3089.0, 2963.0, 2980.0, 329896.0, 904053.0...0361, FESX, 199FESXH5, 2015-03-13, Fri, 3657.0, 3680.0, 3627.0, 3670.0, 867678.0, 3499116.00362, FESX, 199FESXM5, 2015-03-16, Mon, 3594.0, 3641.0, 3588.0, 3629.0, 250445.0, 1056099.0...0426, FESX, 199FESXM5, 2015-06-18, Thu, 3398.0, 3540.0, 3373.0, 3465.0, 1173246.0, 811805.00427, FESX, 199FESXM5, 2015-06-19, Fri, 3443.0, 3499.0, 3440.0, 3488.0, 104096.0, 516792.0
Much better. The roll over is now happening5 days before. A quick visualinspection of theLen indices show it. For example:
199FESXM4 to199FESXU4 happens atlen171-172. Withoutcheckdate it happened at176-177
The roll over is happening on the Monday before the 3rd Friday of theexpiration month.
Adding a volume condition
Even with the improvement, the situation can be further improved in that notonly the date but also de negotiatedvolume will be taken into account. Doswitch when the new contract trades more volume than the currently active one.
Let’s add acheckcondition to the mix and run.
$ ./rollover.py --rollover --checkdate --checkcondition --plotLen, Name, RollName, Datetime, WeekDay, Open, High, Low, Close, Volume, OpenInterest0001, FESX, 199FESXM4, 2013-09-26, Thu, 2829.0, 2843.0, 2829.0, 2843.0, 3.0, 1000.00002, FESX, 199FESXM4, 2013-09-27, Fri, 2842.0, 2842.0, 2832.0, 2841.0, 16.0, 1101.0...0175, FESX, 199FESXM4, 2014-06-19, Thu, 3307.0, 3330.0, 3300.0, 3321.0, 717979.0, 759122.00176, FESX, 199FESXU4, 2014-06-20, Fri, 3309.0, 3318.0, 3290.0, 3298.0, 711627.0, 2957641.0...0240, FESX, 199FESXU4, 2014-09-18, Thu, 3249.0, 3275.0, 3243.0, 3270.0, 846600.0, 803202.00241, FESX, 199FESXZ4, 2014-09-19, Fri, 3273.0, 3293.0, 3250.0, 3252.0, 1042294.0, 3021305.0...0305, FESX, 199FESXZ4, 2014-12-18, Thu, 3095.0, 3175.0, 3085.0, 3172.0, 1309574.0, 889112.00306, FESX, 199FESXH5, 2014-12-19, Fri, 3195.0, 3200.0, 3106.0, 3147.0, 1329040.0, 2964538.0...0365, FESX, 199FESXH5, 2015-03-19, Thu, 3661.0, 3691.0, 3646.0, 3668.0, 1271122.0, 1054639.00366, FESX, 199FESXM5, 2015-03-20, Fri, 3607.0, 3664.0, 3595.0, 3646.0, 1182235.0, 3407004.0...0426, FESX, 199FESXM5, 2015-06-18, Thu, 3398.0, 3540.0, 3373.0, 3465.0, 1173246.0, 811805.00427, FESX, 199FESXM5, 2015-06-19, Fri, 3443.0, 3499.0, 3440.0, 3488.0, 104096.0, 516792.0
Even better. We have moved the switch date to theThursday before the wellknown3rd Friday of the expiration month
This should come to no surprise because the expiring future trades a lot lesshours on thatFriday and the volume must be small.
Note
The roll over date could have also been set to thatThursday by thecheckdate callable. But that isn’t the point of the sample.
Concluding
backtrader includes now a flexible mechanism to allow rolling over futures tocreate a continuous stream.
Sample Usage
$ ./rollover.py --helpusage: rollover.py [-h] [--no-cerebro] [--rollover] [--checkdate] [--checkcondition] [--plot [kwargs]]Sample for Roll Over of Futuresoptional arguments: -h, --help show this help message and exit --no-cerebro Use RollOver Directly (default: False) --rollover --checkdate Change during expiration week (default: False) --checkcondition Change when a given condition is met (default: False) --plot [kwargs], -p [kwargs] Plot the read data applying any kwargs passed For example: --plot style="candle" (to plot candles) (default: None)
Sample Code
from__future__import(absolute_import,division,print_function,unicode_literals)importargparseimportbisectimportcalendarimportdatetimeimportbacktraderasbtclassTheStrategy(bt.Strategy):defstart(self):header=['Len','Name','RollName','Datetime','WeekDay','Open','High','Low','Close','Volume','OpenInterest']print(', '.join(header))defnext(self):txt=list()txt.append('%04d'%len(self.data0))txt.append('{}'.format(self.data0._dataname))# Internal knowledge ... current expiration in use is in _dtxt.append('{}'.format(self.data0._d._dataname))txt.append('{}'.format(self.data.datetime.date()))txt.append('{}'.format(self.data.datetime.date().strftime('%a')))txt.append('{}'.format(self.data.open[0]))txt.append('{}'.format(self.data.high[0]))txt.append('{}'.format(self.data.low[0]))txt.append('{}'.format(self.data.close[0]))txt.append('{}'.format(self.data.volume[0]))txt.append('{}'.format(self.data.openinterest[0]))print(', '.join(txt))defcheckdate(dt,d):# Check if the date is in the week where the 3rd friday of Mar/Jun/Sep/Dec# EuroStoxx50 expiry codes: MY# M -> H, M, U, Z (Mar, Jun, Sep, Dec)# Y -> 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 -> year code. 5 -> 2015MONTHS=dict(H=3,M=6,U=9,Z=12)M=MONTHS[d._dataname[-2]]centuria,year=divmod(dt.year,10)decade=centuria*10YCode=int(d._dataname[-1])Y=decade+YCodeifY<dt.year:# Example: year 2019 ... YCode is 0 for 2020Y+=10exp_day=21-(calendar.weekday(Y,M,1)+2)%7exp_dt=datetime.datetime(Y,M,exp_day)# Get the year, week numbersexp_year,exp_week,_=exp_dt.isocalendar()dt_year,dt_week,_=dt.isocalendar()# print('dt {} vs {} exp_dt'.format(dt, exp_dt))# print('dt_week {} vs {} exp_week'.format(dt_week, exp_week))# can switch if in same weekreturn(dt_year,dt_week)==(exp_year,exp_week)defcheckvolume(d0,d1):returnd0.volume[0]<d1.volume[0]# Switch if volume from d0 < d1defrunstrat(args=None):args=parse_args(args)cerebro=bt.Cerebro()fcodes=['199FESXM4','199FESXU4','199FESXZ4','199FESXH5','199FESXM5']store=bt.stores.VChartFile()ffeeds=[store.getdata(dataname=x)forxinfcodes]rollkwargs=dict()ifargs.checkdate:rollkwargs['checkdate']=checkdateifargs.checkcondition:rollkwargs['checkcondition']=checkvolumeifnotargs.no_cerebro:ifargs.rollover:cerebro.rolloverdata(name='FESX',*ffeeds,**rollkwargs)else:cerebro.chaindata(name='FESX',*ffeeds)else:drollover=bt.feeds.RollOver(*ffeeds,dataname='FESX',**rollkwargs)cerebro.adddata(drollover)cerebro.addstrategy(TheStrategy)cerebro.run(stdstats=False)ifargs.plot:pkwargs=dict(style='bar')ifargs.plotisnotTrue:# evals to True but is not Truenpkwargs=eval('dict('+args.plot+')')# args were passedpkwargs.update(npkwargs)cerebro.plot(**pkwargs)defparse_args(pargs=None):parser=argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,description='Sample for Roll Over of Futures')parser.add_argument('--no-cerebro',required=False,action='store_true',help='Use RollOver Directly')parser.add_argument('--rollover',required=False,action='store_true')parser.add_argument('--checkdate',required=False,action='store_true',help='Change during expiration week')parser.add_argument('--checkcondition',required=False,action='store_true',help='Change when a given condition is met')# Plot optionsparser.add_argument('--plot','-p',nargs='?',required=False,metavar='kwargs',const=True,help=('Plot the read data applying any kwargs passed\n''\n''For example:\n''\n'' --plot style="candle" (to plot candles)\n'))ifpargsisnotNone:returnparser.parse_args(pargs)returnparser.parse_args()if__name__=='__main__':runstrat()