Given the expressed intention of buying and selling assets in a market where one has to place orders (seeMotivation), the fundamental question, as already stated by Kearns et. al. [5, 6], is as follows:

"How should one buy (respectively sell) V shares of a given asset over a time horizon H while spending the least (respectively, receiving the most) of the counter asset (e.g. USD)."

Taking into consideration the previously mentionedorder types, a trader has a variety of options in order to fullfill his duties and buy (respectively sell) the demanded shares.In favour of simplicity three options are available [5]:

• “Submit a market order at time 0 for V shares.”
• “Submit a market order at time T for V shares.”
• “Submit a limit order at time 0 for V shares and a market order for the unexecuted shares (if any) at time T.”

To determine the most effective strategy is a non-trivial task; even though only two order types available.As limit orders can be placed arbitrarily deep in the order book, difficulties such as the risk of non-execution and unfavourable price movements have to be taken into consideration.

“(Approximately) Markovian nature of trade execution: if our state space is properly defined, the optimal action at any given point in time is (approximately) independent of any previous actions.” [Kearns et. al.]

E.g. executions do not affect the market for future executions.

Many useful definitions which highlight the difficulties related to the order execution domain were stated by Lim et. al. [8].

Moving the price up by executing large buy orders (respectively down for sell orders) at once. By splitting up a big order (e.g. V shares) into smaller pieces and spreading the execution over a time horizon H, the impact cost can be lessened.

Arises when the price moves against our favour while splitting a big order into pieces and delaying execution. Therefore the opportunity to execute at a better price.

Optimizes trade-off between impact cost and opportunity cost and therefore trying to find best execution.

Kearns et. al. [5] determine which limit order price results in the most advantageousexecution price.Furthermore, they introducerisk defined as thestandard deviation of those returns.Lastly, they combine the two techniques and derive theefficient pricing frontier (based on Markowitz efficient frontier [7]) which shows the trade-off between the risk and return.

An attempt was made to demonstrate the behaviour of limit orders on Bitcoin order book from the past as well as artificially generated order books:https://github.com/backender/ctc-executioner/blob/master/notebooks/order_execution_behaviour.ipynb

According to [6], the factors which have an impact on order execution can be categorized into:

• Market Variables: Reflect various features of the trading activity in the stock.
• Private Variables: the amount of time and shares remaining to be executed.

In this project, we aim to build a pipeline which optimizes order execution considering both of the mentioned factors.The pipeline contains the following steps:

1. Given an inventory and market variables, recommend a time horizon:f(i, M) -> t
2. Given market variables, recommend a risk factor for the placement of orders on a specific side (buy/sell):f(side, M) -> r
3. Learn a policy (using value iteration) for the execution process given a time horizon, inventory and risk aversion:f(t, i, r) -> p
4. Given a policy and market variables, apply deep learning in order to find patterns in the market which lead to optimize the policy (using policy iteration):f(p, M) -> p

[5] Electronic Trading in Order-Driven Markets: Efficient Execution:https://www.cis.upenn.edu/~mkearns/papers/optexec.pdf

[6] Reinforcement Learning for Optimized Trade Execution:https://www.cis.upenn.edu/~mkearns/papers/rlexec.pdf

[7] Markowitz efficient frontier

[8] Optimal Trade Execution: An Evolutionary Approach:http://ieeexplore.ieee.org/document/1554806/