The free energy principle which underlies active inference attempts to explain the emergence of Bayesian inference in stochastic processes under the assumption of (non-equilibrium) steady state distributions. We contribute a study of the dynamics of an exact Bayesian inference hyperparameter embedded in a Markov chain that infers the dynamics of an observed process. This system does not have a steady-state but still contains exact Bayesian inference. Our study may contribute to future generalizations of the free energy principle to non-steady state systems.

Our treatment uses well-known constructions in Bayesian inference. The main contribution is that we take a different perspective than that of standard treatments. We are interested in how the dynamics of Bayesian inference look from the outside.

1. 1.

   For example, the pseudo-counts that are accumulated as the parameters of a Dirichlet posterior over the categorical states of a generative process.

2. 2.

   We add a hat to variables that the beliefs encoded by\(\xi _t\) range over. This is to highlight that the hatted variables can take different values from the actual ones e.g. when we have a fixed\(\phi \) that defines the IID process then in general the encoded belief\(q({\hat{\phi }}|\xi _t)\) still ranges over\({\hat{\phi }}\ne \phi \). A more technical reason is that the hatted variables are in some sense virtual. This should become clearer in the following. A rigorous definition of what “virtual” means is beyond the scope of this paper.

3. 3.

   This is the solution because it leads to the KL divergence being zero which means\(q({\hat{\varPhi }}|f(\xi _t,x_t))=q({\hat{\varPhi }}|x_t,\xi _t)\). See e.g. [2] for properties of Dirichlet priors for categorical distributions.

The work by MB and RK on this publication was made possible through the support of a grant from Templeton World Charity Foundation, Inc. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of Templeton World Charity Foundation, Inc.

Authors and Affiliations

1. Araya Inc., Tokyo, Japan

   Martin Biehl & Ryota Kanai

Corresponding author

Correspondence toMartin Biehl.

Editors and Affiliations

1. Ghent University, Ghent, Belgium

   Tim Verbelen

2. Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands

   Pablo Lanillos

3. University of Sussex, Brighton, UK

   Christopher L. Buckley

4. Ghent University, Ghent, Belgium

   Cedric De Boom

Reprints and permissions

© 2020 Springer Nature Switzerland AG

Policies and ethics