Repository files navigation

Approximate Inference for Stochastic Optimal Control

Input Inference for Control (i2c) is an inference-based optimal control algorithm.The current implementation, Gaussiani2c, can perform trajectory optimization, model predictive control and covariance control via a Gaussian approximation of the optimal state-action distribution. This yields time-varying linear (Gaussian) controllers, and is approximately equivalent to quadratic optimal control methods like differential dynamic programming, iterative-/sequential LQR.

For more information, see the following papers:

[1] J. Watson and H. Abdulsamad and R. Findeisen and J. Peters.Stochastic Control through Approximate Bayesian Input Inference. Submitted to IEEE Transactions on Automatic Control Special Issue, Learning and Control 2021. (arXiv)

[2] J. Watson and J. Peters.Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk. American Control Conference (ACC) 2021 (arXiv)

[3] J. Watson and H. Abdulsamad and J. Peters.Stochastic Optimal Control as Approximate Input Inference. Conference on Robot Learning (CoRL) 2019. (arXiv)

Create environmenti2c and install

cd input-inference-for-control&& conda create -y -n i2c pip python=3.7&& conda activate i2c&& pip3 install -r requirements.txt&& pip install -e.

To optimize pendulum swing-up with cubature quadrature, run

    python scripts/i2c_run.py pendulum_known_quad

the output directory results should look like this

Prior experiments are preserved here. All results are stored in/_results.

Section 3.1 of [3]

    python scripts/LQR_compare.py

Section 3.2 of [3], Section IV.A of [1]

    python scripts/i2c_run.py -h

results are in _results/

Linear Gaussian covariance control.Section IV.C of [2]

    python scripts/linear_covariance_control.py

Pendulum swing-up with covaraince control.Section IV.C of [2]

    python scripts/nonlinear_covariance_control.py

Runsi2c and iLQR MPC with a cubature kalman filter for an acrobatic quadropter task.Section IV.C of [1]

    python scripts/mpc_state_est/mpc_quad.py 0 --plot

For iLQR usehttps://github.com/hanyas/trajopt

To citei2c, please reference the appropriate paper

@misc{watson2021stochastic,title={Stochastic Control through Approximate Bayesian Input Inference},author={Watson, Joe and Abdulsamad, Hany and Findeisen, Rolf and Peters, Jan},year={2021},eprint={2105.07693},archivePrefix={arXiv},primaryClass={cs.LG}}

@inproceedings{i2cacc,author    ={Watson, Joe and Peters, Jan},title     ={Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk},booktitle ={American Control Conference},year      ={2021},}

@inproceedings{i2ccorl,author    ={Watson, Joe and  Abdulsamad, Hany and Peters, Jan},title     ={Stochastic Optimal Control as Approximate Input Inference},booktitle ={Conference on Robot Learning},year      ={2019},}