Repository files navigation

Setting|Documentation|Installation|Main functionality|Cite us|References|Ask a question

Repository for theDeterministic Particle Flow Control framework

• Link topublished article: 📰Phys. Rev. Research 4, 043035
• Link toposter: 📊poster
• Link todocumendation: 📜readthedocs

[ This is work together with Manfred Opper funded by theSFB 1294. ]

[PyPI package includes onlynumpy implementation, the github repository here provides bothnumpy andpytorch functionalities. ]

Obtaining optimal interventions for stochastic nonlinear systems is a computationally demanding process requiring the solution of nonlinear partial differential equations. Here we build on the Path Integral control formalism to derive anoniterative framework that represents the solutions of the underlying partial differential equations in terms ofdeterministic particle flows.

We consider a stochastic system described by a stochastic differential equation (SDE)

with driftf(x,t) and diffusionσ(x,t).

We want to constrain the dynamics of this system for a time interval

• either to reach some target state at time T (terminal constraint),
• and/or to visit/avoid specific regions of the state space (path constraints).

We implement the constraints in terms a time- and state-dependent perturbation of the deterministic part of the dynamics, i.e. we apply interventionsu(x,t) and the controlled system dynamics become

.

Following the assumptions of the Path Integral control formalism, i.e. assuming that control costs are inversely proportional of noise variance (seepaper for more details), we can show that the interventionsu(x,t) can be obtained from thelogarithmic gradient of the solution of a backward linear partial differential equation rescaled by the noise variance.

Here, instead of solving the backward PDE to obtain the optimal drift adjustment that implements the constraints, we express the optimal interventions as thedifference of the logarithmic gradient of two probability flows, and. The probability flow or density satisfies the forward filtering equation, a forward PDE that in the absence of path constraints is the Fokker--Planck equation of the uncontrolled dynamics, while is the marginal constrained density that in turn satisfies the Fokker--Planck equation of the optimally controlled dynamics.

To install the install the software throughPypi run the following commands

# Create and activate a new virtualenvironmentpython-mvenvvenvsourcevenv/bin/activatepipinstallDeterministicParticleFlowControl

or to install from the downloaded repository run

# Create and activate a new virtualenvironmentpython-mvenvvenvsourcevenv/bin/activate# install the depndenciespipinstall-rrequirements.txtpipinstall-e ./

To obtain the time dependent control functions create an instance ofDPFC

fromDeterministicParticleFlowControlimportDPFCcontrol_flows=DPFC(t1,t2,y1,y2,f,g,N,M,reweight,U,dens_est,reject,kern='RBF',f_true=None,brown_bridge=False)

where

• t1,t2 : are define the timeinterval [t1,t2] within which the constraints are imposed onto the system,

• y1,y2 : are the initial and target state,

• f : is the drift function of the uncontrolled system,

• g : is the diffusion coefficient (can be scalar or matrix),

• N : the number of particles that will be employed for sampling the flows,

• M : inducing point number for the sparse kernel estimation of the logarithmic gradients,

• reweight: boolean indicating whether reweighting of forward trajectories is necessary, i.e. when:

  • path constraints are relevant for the problem,
  • the target is a non typical system state and the forward flow will be sampled as a reweighted Brownian bridge,

• U : function handle representing the path constraints, i.e. , for pushing the controlled trajectories towards the sin(t) line. Expects two arguments, so even if path constraint is non timedependent add an extra dummy variable in the argument list,

• dens_est : idicator determining the method to be employed for the logarithmic gradient (score) estimation. Currently supported:

  • nonparametric : for nonparametric sparse kernel estimation,

• reject : boolean variable determining whether rejection of backward trajectories failing to reach the initial condition will take place(this is mostly used as an indicator of numerical instabilities - if everything runs smoothly 1 or 2 trajectories need to be deleted, but often none of themif more trajectories get deleted, then rerun the computation with more particles N)

• kern : the kernel that will be employed for the nonparametric logarithmic gradient estimation of the sampled densities. Currently supported:

  • RBF : radial basis function kernel with lengthscale estimated at every time step from the samples,

• f_true : function handler of system drift function when the reweighted Brownian bridge functionality for forward sampling is used. In that case,f should be supplied with the drift function of the brownian bridge,U should be set to the necessary path constraint (seepaper), andreweight andbrown_bridge should be turned to True,

• brown_bridge: boolean variable indicating whether the sampling of the forward flow will happen with reweighted Brownian bridge dynamics.

This returns an object that contains the sampled flows and control functions. To compute the controls for the timestepti when the system is at statex call

u=control_flows.calculate_u(x,ti)

The method is introduced in thispaper. To cite it:

@article{maoutsa2022deterministic,  title = {Deterministic particle flows for constraining stochastic nonlinear systems},  author = {Maoutsa, Dimitra and Opper, Manfred},  journal = {Phys. Rev. Research},  volume = {4},  issue = {4},  pages = {043035},  numpages = {17},  year = {2022},  month = {Oct},  publisher = {American Physical Society},  doi = {10.1103/PhysRevResearch.4.043035},  url = {https://link.aps.org/doi/10.1103/PhysRevResearch.4.043035}}

[1] Dimitra Maoutsa, Manfred Opper. "Deterministic particle flows for constraining stochastic nonlinear systems". 2021.[Physical review Research]

[2] Dimitra Maoutsa, Manfred Opper. "Deterministic particle flows for constraining SDEs". 2021.[Machine Learning and the Physical Sciences, Workshop at the 35th Conference on Neural Information Processing Systems (NeurIPS)]

[3] Dimitra Maoutsa, Sebastian Reich, Manfred Opper. "Interacting particle solutions of Fokker–Planck equations through gradient–log–density estimation". 2021.[Entropy]