Operating the platform

Line Iterators

To engage into operations, the plaftorm uses the notion of lineiterators. They have been loosely modeled after Python’s iterators but haveactually nothing to do with them.

Strategies and Indicators are line iterators.

The line iterator concept tries to describe the following:

• A Line Iterator kicks slave line iterators telling them to iterate

• A Line Iterator then iterates over its own declared named lines setting values

The key to iteration, just like with regular Python iterators, is:

• Thenext method

  It will be called for each iteration. Thedatas array which thelineiterator has and serve as basis for logic/calculations will have alreadybeen moved to the next index by the platform (barring data replay)

  Called when theminimum period for the line iterator has been met. Abit more on this below.

But because they are not regular iterators, two additional methods exist:

• prenext

  Called before theminimum period for the line iterator` has been met.

• nextstart

  Called exactlyONCE when theminimum period for the line iterator`has been met.

  The default behavior is to forward the call tonext, but can of coursebe overriden if needed.

Extra methods forIndicators

To speed up operations, Indicators support a batch operation mode which hasbeen termed as runonce. It is not strictly needed (anext method suffices)but it greatly reduces time.

The runonce methods rules void the get/set point with index 0 and relies ondirect access to the underlying arrays holding the data and being passed theright indices for each state.

The defined methods follow the naming of the next family:

• once(self, start, end)

  Called when the minimum period has been met. The internal array must beprocessed between start and end which are zero based from the start of theinternal array

• preonce(self, start, end)

  Called before the minimum period has been met.

• oncestart(self, start, end)

  Called exactlyONCE when the minimum period has been met.

  The default behavior is to forward the call toonce, but can of coursebe overriden if needed.

Minimum Period

A picture is worth a thousand words and in this case possibly an exampletoo. A SimpleMovingAverage is capable of explaining it:

classSimpleMovingAverage(Indicator):lines=('sma',)params=dict(period=20)def__init__(self):...# Not relevant for the explanationdefprenext(self):print('prenext:: current period:',len(self))defnextstart(self):print('nextstart:: current period:',len(self))# emulate default behavior ... call nextself.next()defnext(self):print('next:: current period:',len(self))

And the instantiation could look like:

sma=btind.SimpleMovingAverage(self.data,period=25)

Briefly explained:

• Assuming the data passed to the moving average is a standard data feed its default period is1 that is: the data feed produces a bar with no initial delay.

• Then the“period=25” instantiated moving average would have its methods called as follows:

  • prenext 24 times

  • nextstart 1 time (in turn callingnext)

  • next n additional times until thedata feed has been exhausted

Let’s go for the killer indicator:a SimpleMovingAverage over anotherSimpleMovingAverage. The instantiation could look like:

sma1=btind.SimpleMovingAverage(self.data,period=25)sma2=btind.SimpleMovingAverage(sma1,period=20)

What now goes on:

• The same as above forsma1

• sma2 is receiving adata feed which has aminimum period of 25 which is oursma1 and therefore

• Thesma2 methods are called as indicated:

  • prenext the first 25 + 18 times for a total of 43 times

  • 25 times to letsma1 produce its 1^st sensible value

  • 18 times to accumulate extrasma1 values

  • For a total of 19 values (1 after 25 calls and then 18 more)

  • nextstart then 1 time (in turn callingnext)

  • next the n additional times until thedata feed has been exhausted

The platform is callingnext when the system has already processed 44 bars.

Theminimum period has been automatically adjusted to the incoming data.

Strategies and Indicators adhere to this behavior:

• Only when the automatically calculated minimum period has been reached willnext be called (barring the initial hook call tonextstart)

Up and Running

Getting up and running involves at least 3Lines objects:

• A Data feed

• A Strategy (actually a class derived from Strategy)

• A Cerebro (brain in Spanish)

Data Feeds

These objects, obviously, provide the data which will be backtested by applyingcalculations (direct and/or with Indicators)

The platform provides several data feeds:

• Several CSV Format and a Generic CSV reader

• Yahoo online fetcher

• Support for receivingPandas DataFrames andblaze objects

• Live Data Feeds withInteracive Brokers,Visual Chart andOanda

The platform makes no assumption about the content of the data feed such astimeframe and compression. Those values, together with a name, can be suppliedfor informational purposes and advance operations like Data Feed Resampling(turning a for example a 5 minute Data Feed into a Daily Data Feed)

Example of setting up a Yahoo Finance Data Feed:

importbacktraderasbtimportbacktrader.feedsasbtfeeds...datapath='path/to/your/yahoo/data.csv'data=btfeeds.YahooFinanceCSVData(dataname=datapath,reversed=True)

The optionalreversed parameter for Yahoo is shown, because the CSV filesdirectly downloaded from Yahoo start with the latest date, rather than with theoldest.

If your data spans a large time range, the actual loaded data can be limited as follows:

data=btfeeds.YahooFinanceCSVData(dataname=datapath,reversed=Truefromdate=datetime.datetime(2014,1,1),todate=datetime.datetime(2014,12,31))

Both thefromdate and thetodate will be included if present in the datafeed.

As already mentioned timeframe, compression and name can be added:

data=btfeeds.YahooFinanceCSVData(dataname=datapath,reversed=Truefromdate=datetime.datetime(2014,1,1),todate=datetime.datetime(2014,12,31)timeframe=bt.TimeFrame.Days,compression=1,name='Yahoo')

If the data is plotted, those values will be used.

A Strategy (derived) class

The goal of anyone using the platform is backtesting the data and this is doneinside a Strategy (derived class).

There are 2 methods which at least need customization:

• __init__

• next

During initialization indicators on data and other calculations are createdprepared to later apply the logic.

The next method is later called to apply the logic for each and every bar of thedata.

A basic Strategy derived class:

classMyStrategy(bt.Strategy):def__init__(self):self.sma=btind.SimpleMovingAverage(self.data,period=20)defnext(self):ifself.sma>self.data.close:self.buy()elifself.sma<self.data.close:self.sell()

Strategies have other methods (or hook points) which can be overriden:

classMyStrategy(bt.Strategy):def__init__(self):self.sma=btind.SimpleMovingAverage(self.data,period=20)defnext(self):ifself.sma>self.data.close:submitted_order=self.buy()elifself.sma<self.data.close:submitted_order=self.sell()defstart(self):print('Backtesting is about to start')defstop(self):print('Backtesting is finished')defnotify_order(self,order):print('An order new/changed/executed/canceled has been received')

Thestart andstop methods should be self-explanatory. As expected andfollowing the text in the print function, thenotify_order method will becalled when the strategy needs a notification. Use case:

• A buy or sell is requested (as seen in next)

  buy/sell will return anorder which is submitted to the broker. Keeping areference to this submitted order is up to the caller.

  It can for example be used to ensure that no new orders are submitted if anorder is still pending.

• If the order is Accepted/Executed/Canceled/Changed the broker will notify the status change (and for example execution size) back to the strategy via the notify method

The QuickStart guide has a complete and functional example of order managementin thenotify_order method.

More can be done with other Strategy classes:

• buy /sell /close

  Use the underlyingbroker andsizer to send the broker a buy/sellorder

  The same could be done by manually creating an Order and passing it over tothe broker. But the platform is about making it easy for those using it.

  close will get the current market position and close it immediately.

• getposition (or the property “position”)

  Returns the current market position

• setsizer/getsizer (or the property “sizer”)

  These allow setting/getting the underlying stake Sizer. The same logic canbe checked against Sizers which provide different stakes for the samesituation (fixed size, proportional to capital, exponential)

  There is plenty of literature but Van K. Tharp has excellent books on thesubject.

A Strategy is aLines object and these support parameters, which are collectedusing the standard Python kwargs argument:

classMyStrategy(bt.Strategy):params=(('period',20),)def__init__(self):self.sma=btind.SimpleMovingAverage(self.data,period=self.params.period)......

Notice how theSimpleMovingAverage is no longer instantiated with a fixedvalue of 20, but rather with the parameter “period” which has been defined forthe strategy.

A Cerebro

Once Data Feeds are available and the Strategy has been defined, a Cerebroinstance is what brings everything together and execute theactions. Instantiating one is easy:

cerebro=bt.Cerebro()

Defaults are taking care of if nothing special is wished.

• A default broker is created

• No commission for the operations

• Data Feeds will be preloaded

• The default execution mode will be runonce (batch operation) which is the faster

  All indicators must support therunonce mode for full speed. The onesincluded in the platform do.

  Custom indicators do not need to implement the runoncefunctionality.Cerebro will simulate it, which means those non-runoncecompatible indicators will run slower. But still most of the system willrun in batch mode.

Since a Data feed is already available and a Strategy too (created earlier) thestandard way to put it all together and get it up and running is:

cerebro.adddata(data)cerebro.addstrategy(MyStrategy,period=25)cerebro.run()

Notice the following:

• The Data Feed “instance” is added

• The MyStrategy “class” is added along with parameters (kwargs) that will be passed to it.

  The instantiation of MyStrategy will be done by cerebro in the backgroundand any kwargs in “addstrategy” will be passed to it

The user may add as many Strategies and Data Feeds as wished. How Strategiescommunicate with each other to achieve coordination (if wished be) is notenforced/restricted by the platform.

Of course a Cerebro offers additional possibilities:

• Decide about preloading and operation mode:

  cerebro=bt.Cerebro(runonce=True,preload=True)

  There is a constraint here:runonce needs preloading (if not, a batchoperation cannot be run) Of course preloading Data Feeds does not enforcerunonce

• setbroker /getbroker (and thebroker property)

  A custom broker can be set if wished. The actual broker instance can also beaccesed

• Plotting. In a regular case as easy as:

  cerebro.run()cerebro.plot()

  plot takes some arguments for the customization

  • numfigs=1

    If the plot is too dense it may be broken down into several plots

  • plotter=None

    A customer plotter instance can be passed and cerebro will notinstantiate a default one

  • **kwargs - standard keyword arguments

    Which will get passed to the plotter.

  Please see the plotting section for more information.

• Optimization of strategies.

  As mentioned above, Cerebro gets a Strategy derived class (not an instance)and the keyword arguments that will be passed to it upon instantiation,which will happen when “run” is called.

  This is so to enable optimization. The same Strategy class will beinstantiated as many times as needed with new parameters. If an instance hadbeen passed to cerebro … this would not be possible.

  Optimization is requested as follows:

  cerebro.optstrategy(MyStrategy,period=xrange(10,20))

  The methodoptstrategy has the same signature asaddstrategy butdoes extra housekeeping to ensure optimization runs as expected. A strategycould be expecting arange as a normal parameter for a strategy andaddstrategy will make no assumptions about the passed parameter.

  On the other hand,optstrategy will understand that an iterable is a setof values that has to be passed in sequence to each instantiation of theStrategy class.

  Notice that instead of a single value arange of values is passed. In thissimple case 10 values 10 -> 19 (20 is the upper limit) will be tried forthis strategy.

  If a more complex strategy is developed with extra parameters they can allbe passed tooptstrategy. Parameters which must not undergo optimizationcan be passed directly without the end user having to create a dummyiterable of just one value. Example:

  cerebro.optstrategy(MyStrategy,period=xrange(10,20),factor=3.5)

  Theoptstrategy method sees factor and creates (a needed) dummy iterablein the background for factor which has a single element (in the example 3.5)

  Note

  Interactive Python shells and some types of frozen executablesunderWindows have problems with the Pythonmultiprocessing module

  Please read the Python documentation aboutmultiprocessing.