A geometric visualisation of Bayes' theorem. In the table, the values 2, 3, 6 and 9 give the relative weights of each corresponding condition and case. The figures denote the cells of the table involved in each metric, the probability being the fraction of each figure that is shaded. This shows that i.e.. Similar reasoning can be used to show that etc.
H stands for anyhypothesis whose probability may be affected bydata (calledevidence below). Often there are competing hypotheses, and the task is to determine which is the most probable.
, theprior probability, is the estimate of the probability of the hypothesisHbefore the dataE, the current evidence, is observed.
E, theevidence, corresponds to new data that were not used in computing the prior probability.
, theposterior probability, is the probability ofHgivenE, i.e.,afterE is observed. This is what we want to know: the probability of a hypothesisgiven the observed evidence.
is the probability of observingEgivenH and is called thelikelihood. As a function ofE withH fixed, it indicates the compatibility of the evidence with the given hypothesis. The likelihood function is a function of the evidence,E, while the posterior probability is a function of the hypothesis,H.
is sometimes termed themarginal likelihood or "model evidence". This factor is the same for all possible hypotheses being considered (as is evident from the fact that the hypothesisH does not appear anywhere in the symbol, unlike for all the other factors) and hence does not factor into determining the relative probabilities of different hypotheses.
(Else one has.)
For different values ofH, only the factors and, both in the numerator, affect the value of – the posterior probability of a hypothesis is proportional to its prior probability (its inherent likeliness) and the newly acquired likelihood (its compatibility with the new observed evidence).
In cases where ("notH"), thelogical negation ofH, is a valid likelihood, Bayes' rule can be rewritten as follows:
because
and
This focuses attention on the term
If that term is approximately 1, then the probability of the hypothesis given the evidence,, is about, about 50% likely - equally likely or not likely. If that term is very small, close to zero, then the probability of the hypothesis, given the evidence, is close to 1 or the conditional hypothesis is quite likely. If that term is very large, much larger than 1, then the hypothesis, given the evidence, is quite unlikely. If the hypothesis (without consideration of evidence) is unlikely, then is small (but not necessarily astronomically small) and is much larger than 1 and this term can be approximated as and relevant probabilities can be compared directly to each other.
Bayesian updating is widely used and computationally convenient. However, it is not the only updating rule that might be considered rational.
Ian Hacking noted that traditional "Dutch book" arguments did not specify Bayesian updating: they left open the possibility that non-Bayesian updating rules could avoid Dutch books. Hacking wrote:[2] "And neither the Dutch book argument nor any other in the personalist arsenal of proofs of the probability axioms entails the dynamic assumption. Not one entails Bayesianism. So the personalist requires the dynamic assumption to be Bayesian. It is true that in consistency a personalist could abandon the Bayesian model of learning from experience. Salt could lose its savour."
Indeed, there are non-Bayesian updating rules that also avoid Dutch books (as discussed in the literature on "probability kinematics") following the publication ofRichard C. Jeffrey's rule, which applies Bayes' rule to the case where the evidence itself is assigned a probability.[3] The additional hypotheses needed to uniquely require Bayesian updating have been deemed to be substantial, complicated, and unsatisfactory.[4]
Inference over exclusive and exhaustive possibilities
If evidence is simultaneously used to update belief over a set of exclusive and exhaustive propositions, Bayesian inference may be thought of as acting on this belief distribution as a whole.
Diagram illustrating event space in general formulation of Bayesian inference. Although this diagram shows discrete models and events, the continuous case may be visualized similarly using probability densities.
Suppose a process is generating independent and identically distributed events, but theprobability distribution is unknown. Let the event space represent the current state of belief for this process. Each model is represented by event. The conditional probabilities are specified to define the models. is thedegree of belief in. Before the first inference step, is a set ofinitial prior probabilities. These must sum to 1, but are otherwise arbitrary.
Suppose that the process is observed to generate. For each, the prior is updated to the posterior. FromBayes' theorem:[5]
Upon observation of further evidence, this procedure may be repeated.
For a sequence ofindependent and identically distributed observations, it can be shown by induction that repeated application of the above is equivalent towhere
Parametric formulation: motivating the formal description
By parameterizing the space of models, the belief in all models may be updated in a single step. The distribution of belief over the model space may then be thought of as a distribution of belief over the parameter space. The distributions in this section are expressed as continuous, represented by probability densities, as this is the usual situation. The technique is, however, equally applicable to discrete distributions.
Theprior distribution is the distribution of the parameter(s) before any data is observed, i.e. . The prior distribution might not be easily determined; in such a case, one possibility may be to use theJeffreys prior to obtain a prior distribution before updating it with newer observations.
Thesampling distribution is the distribution of the observed data conditional on its parameters, i.e.. This is also termed thelikelihood, especially when viewed as a function of the parameter(s), sometimes written.
Themarginal likelihood (sometimes also termed theevidence) is the distribution of the observed datamarginalized over the parameter(s), i.e. It quantifies the agreement between data and expert opinion, in a geometric sense that can be made precise.[6] If the marginal likelihood is 0 then there is no agreement between the data and expert opinion and Bayes' rule cannot be applied.
Theposterior distribution is the distribution of the parameter(s) after taking into account the observed data. This is determined byBayes' rule, which forms the heart of Bayesian inference: This is expressed in words as "posterior is proportional to likelihood times prior", or sometimes as "posterior = likelihood times prior, over evidence".
In practice, for almost all complex Bayesian models used in machine learning, the posterior distribution is not obtained in a closed form distribution, mainly because the parameter space for can be very high, or the Bayesian model retains certain hierarchical structure formulated from the observations and parameter. In such situations, we need to resort to approximation techniques.[7]
General case: Let be the conditional distribution of given and let be the distribution of. The joint distribution is then. The conditional distribution of given is then determined by
Existence and uniqueness of the neededconditional expectation is a consequence of theRadon–Nikodym theorem. This was formulated byKolmogorov in his famous book from 1933. Kolmogorov underlines the importance of conditional probability by writing "I wish to call attention to ... and especially the theory of conditional probabilities and conditional expectations ..." in the Preface.[8] The Bayes theorem determines the posterior distribution from the prior distribution. Uniqueness requires continuity assumptions.[9] Bayes' theorem can be generalized to include improper prior distributions such as the uniform distribution on the real line.[10] ModernMarkov chain Monte Carlo methods have boosted the importance of Bayes' theorem including cases with improper priors.[11]
Bayesian theory calls for the use of the posterior predictive distribution to dopredictive inference, i.e., topredict the distribution of a new, unobserved data point. That is, instead of a fixed point as a prediction, a distribution over possible points is returned. Only this way is the entire posterior distribution of the parameter(s) used. By comparison, prediction infrequentist statistics often involves finding an optimum point estimate of the parameter(s)—e.g., bymaximum likelihood ormaximum a posteriori estimation (MAP)—and then plugging this estimate into the formula for the distribution of a data point. This has the disadvantage that it does not account for any uncertainty in the value of the parameter, and hence will underestimate thevariance of the predictive distribution.
In some instances, frequentist statistics can work around this problem. For example,confidence intervals andprediction intervals in frequentist statistics when constructed from anormal distribution with unknownmean andvariance are constructed using aStudent's t-distribution. This correctly estimates the variance, due to the facts that (1) the average of normally distributed random variables is also normally distributed, and (2) the predictive distribution of a normally distributed data point with unknown mean and variance, using conjugate or uninformative priors, has a Student's t-distribution. In Bayesian statistics, however, the posterior predictive distribution can always be determined exactly—or at least to an arbitrary level of precision when numerical methods are used.
Both types of predictive distributions have the form of acompound probability distribution (as does themarginal likelihood). In fact, if the prior distribution is aconjugate prior, such that the prior and posterior distributions come from the same family, it can be seen that both prior and posterior predictive distributions also come from the same family of compound distributions. The only difference is that the posterior predictive distribution uses the updated values of the hyperparameters (applying the Bayesian update rules given in theconjugate prior article), while the prior predictive distribution uses the values of the hyperparameters that appear in the prior distribution.
. That is, if the model were true, the evidence would be more likely than is predicted by the current state of belief. The reverse applies for a decrease in belief. If the belief does not change,. That is, the evidence is independent of the model. If the model were true, the evidence would be exactly as likely as predicted by the current state of belief.
If then. If and, then. This can be interpreted to mean that hard convictions are insensitive to counter-evidence.
The former follows directly from Bayes' theorem. The latter can be derived by applying the first rule to the event "not" in place of "", yielding "if, then", from which the result immediately follows.
Consider the behaviour of a belief distribution as it is updated a large number of times withindependent and identically distributed trials. For sufficiently nice prior probabilities, theBernstein-von Mises theorem gives that in the limit of infinite trials, the posterior converges to aGaussian distribution independent of the initial prior under some conditions firstly outlined and rigorously proven byJoseph L. Doob in 1948, namely if the random variable in consideration has a finiteprobability space. The more general results were obtained later by the statisticianDavid A. Freedman who published in two seminal research papers in 1963[12] and 1965[13] when and under what circumstances the asymptotic behaviour of posterior is guaranteed. His 1963 paper treats, like Doob (1949), the finite case and comes to a satisfactory conclusion. However, if the random variable has an infinite but countableprobability space (i.e., corresponding to a die with infinite many faces) the 1965 paper demonstrates that for a dense subset of priors theBernstein-von Mises theorem is not applicable. In this case there isalmost surely no asymptotic convergence. Later in the 1980s and 1990sFreedman andPersi Diaconis continued to work on the case of infinite countable probability spaces.[14] To summarise, there may be insufficient trials to suppress the effects of the initial choice, and especially for large (but finite) systems the convergence might be very slow.
In parameterized form, the prior distribution is often assumed to come from a family of distributions calledconjugate priors. The usefulness of a conjugate prior is that the corresponding posterior distribution will be in the same family, and the calculation may be expressed inclosed form.
It is often desired to use a posterior distribution to estimate a parameter or variable. Several methods of Bayesian estimation selectmeasurements of central tendency from the posterior distribution.
For one-dimensional problems, a unique median exists for practical continuous problems. The posterior median is attractive as arobust estimator.[15]
If there exists a finite mean for the posterior distribution, then the posterior mean is a method of estimation.[16]
There are examples where no maximum is attained, in which case the set of MAP estimates isempty.
There are other methods of estimation that minimize the posteriorrisk (expected-posterior loss) with respect to aloss function, and these are of interest tostatistical decision theory using the sampling distribution ("frequentist statistics").[18]
Suppose there are two full bowls of cookies. Bowl #1 has 10 chocolate chip and 30 plain cookies, while bowl #2 has 20 of each. Our friend Fred picks a bowl at random, and then picks a cookie at random. We may assume there is no reason to believe Fred treats one bowl differently from another, likewise for the cookies. The cookie turns out to be a plain one. How probable is it that Fred picked it out of bowl #1?
Intuitively, it seems clear that the answer should be more than a half, since there are more plain cookies in bowl #1. The precise answer is given by Bayes' theorem. Let correspond to bowl #1, and to bowl #2.It is given that the bowls are identical from Fred's point of view, thus, and the two must add up to 1, so both are equal to 0.5.The event is the observation of a plain cookie. From the contents of the bowls, we know that and Bayes' formula then yields
Before we observed the cookie, the probability we assigned for Fred having chosen bowl #1 was the prior probability,, which was 0.5. After observing the cookie, we must revise the probability to, which is 0.6.
Example results for archaeology example. This simulation was generated using c=15.2.
An archaeologist is working at a site thought to be from the medieval period, between the 11th century to the 16th century. However, it is uncertain exactly when in this period the site was inhabited. Fragments of pottery are found, some of which are glazed and some of which are decorated. It is expected that if the site were inhabited during the early medieval period, then 1% of the pottery would be glazed and 50% of its area decorated, whereas if it had been inhabited in the late medieval period then 81% would be glazed and 5% of its area decorated. How confident can the archaeologist be in the date of inhabitation as fragments are unearthed?
The degree of belief in the continuous variable (century) is to be calculated, with the discrete set of events as evidence. Assuming linear variation of glaze and decoration with time, and that these variables are independent,
Assume a uniform prior of, and that trials areindependent and identically distributed. When a new fragment of type is discovered, Bayes' theorem is applied to update the degree of belief for each:
A computer simulation of the changing belief as 50 fragments are unearthed is shown on the graph. In the simulation, the site was inhabited around 1420, or. By calculating the area under the relevant portion of the graph for 50 trials, the archaeologist can say that there is practically no chance the site was inhabited in the 11th and 12th centuries, about 1% chance that it was inhabited during the 13th century, 63% chance during the 14th century and 36% during the 15th century. TheBernstein-von Mises theorem asserts here the asymptotic convergence to the "true" distribution because theprobability space corresponding to the discrete set of events is finite (see above section on asymptotic behaviour of the posterior).
Adecision-theoretic justification of the use of Bayesian inference was given byAbraham Wald, who proved that every unique Bayesian procedure isadmissible. Conversely, everyadmissible statistical procedure is either a Bayesian procedure or a limit of Bayesian procedures.[20]
"Under some conditions, all admissible procedures are either Bayes procedures or limits of Bayes procedures (in various senses). These remarkable results, at least in their original form, are due essentially to Wald. They are useful because the property of being Bayes is easier to analyze than admissibility."[20]
"In decision theory, a quite general method for proving admissibility consists in exhibiting a procedure as a unique Bayes solution."[24]
"In the first chapters of this work, prior distributions with finite support and the corresponding Bayes procedures were used to establish some of the main theorems relating to the comparison of experiments. Bayes procedures with respect to more general prior distributions have played a very important role in the development of statistics, including its asymptotic theory." "There are many problems where a glance at posterior distributions, for suitable priors, yields immediately interesting information. Also, this technique can hardly be avoided in sequential analysis."[25]
"A useful fact is that any Bayes decision rule obtained by taking a proper prior over the whole parameter space must be admissible"[26]
"An important area of investigation in the development of admissibility ideas has been that of conventional sampling-theory procedures, and many interesting results have been obtained."[27]
Bayesian methodology also plays a role inmodel selection where the aim is to select one model from a set of competing models that represents most closely the underlying process that generated the observed data. In Bayesian model comparison, the model with the highestposterior probability given the data is selected. The posterior probability of a model depends on the evidence, ormarginal likelihood, which reflects the probability that the data is generated by the model, and on theprior belief of the model. When two competing models are a priori considered to be equiprobable, the ratio of their posterior probabilities corresponds to theBayes factor. Since Bayesian model comparison is aimed on selecting the model with the highest posterior probability, this methodology is also referred to as the maximum a posteriori (MAP) selection rule[28] or the MAP probability rule.[29]
While conceptually simple, Bayesian methods can be mathematically and numerically challenging. Probabilistic programming languages (PPLs) implement functions to easily build Bayesian models together with efficient automatic inference methods. This helps separate the model building from the inference, allowing practitioners to focus on their specific problems and leaving PPLs to handle the computational details for them.[30][31][32]
Bayesian inference has applications inartificial intelligence andexpert systems. Bayesian inference techniques have been a fundamental part of computerizedpattern recognition techniques since the late 1950s.[33] There is also an ever-growing connection between Bayesian methods and simulation-basedMonte Carlo techniques since complex models cannot be processed in closed form by a Bayesian analysis, while agraphical model structuremay allow for efficient simulation algorithms like theGibbs sampling and otherMetropolis–Hastings algorithm schemes.[34] Recently[when?] Bayesian inference has gained popularity among thephylogenetics community for these reasons; a number of applications allow many demographic and evolutionary parameters to be estimated simultaneously.
Solomonoff's Inductive inference is the theory of prediction based on observations; for example, predicting the next symbol based upon a given series of symbols. The only assumption is that the environment follows some unknown but computableprobability distribution. It is a formal inductive framework that combines two well-studied principles of inductive inference: Bayesian statistics andOccam's Razor.[35][unreliable source?] Solomonoff's universal prior probability of any prefixp of a computable sequencex is the sum of the probabilities of all programs (for a universal computer) that compute something starting withp. Given somep and any computable but unknown probability distribution from whichx is sampled, the universal prior and Bayes' theorem can be used to predict the yet unseen parts ofx in optimal fashion.[36][37]
Bayesian inference has been applied in differentbioinformatics applications, including differential gene expression analysis.[38] Bayesian inference is also used in a general cancer risk model, calledCIRI (Continuous Individualized Risk Index), where serial measurements are incorporated to update a Bayesian model which is primarily built from prior knowledge.[39][40]
The Bayesian approach has been central to recent progress in cosmology and astrophysical applications,[41][42] and extends to a wide range of astrophysical problems, including the characterisation of exoplanet (such as the fitting of atmosphere fork2-18b[43]), parameter constraints with cosmological data,[44] and calibration in astrophysical experiments.[45]
In cosmology, it is often employed with computational techniques such asMarkov chain Monte Carlo(MCMC) andNested sampling algorithm to analyse complex datasets and navigate high-dimensional parameter space. A notable application is to the Planck 2018 CMB data for parameter inference.[44]The six base cosmological parameters inLambda-CDM model are not predicted by a theory, but rather fitted from Cosmic microwave background (CMB) data to a chosen model of cosmology (the Lambda-CDM model).[46] The bayesian code for cosmology `cobaya`[47] sets up cosmological runs and interfaces cosmological likelihoods, Boltzmann code,[48][49] which computes the predicted CMB anisotropies for any given set of cosmological parameters, with MCMC or nested sampler.
This computational framework is not limited to the standard model, it is also essential for testing alternative or extended theories of cosmology, such as theories with early dark energy,[50] or modified gravity theories introducing additional parameters beyond Lambda-CDM.Bayesian model comparison can then be employed to calculate the evidence for competing models, providing a statistical basis to assess whether the data support them over the standard Lambda-CDM.[51]
Bayesian inference can be used by jurors to coherently accumulate the evidence for and against a defendant, and to see whether, in totality, it meets their personal threshold for "beyond a reasonable doubt".[52][53][54] Bayes' theorem is applied successively to all evidence presented, with the posterior from one stage becoming the prior for the next. The benefit of a Bayesian approach is that it gives the juror an unbiased, rational mechanism for combining evidence. It may be appropriate to explain Bayes' theorem to jurors inodds form, asbetting odds are more widely understood than probabilities. Alternatively, alogarithmic approach, replacing multiplication with addition, might be easier for a jury to handle.
Adding up evidence
If the existence of the crime is not in doubt, only the identity of the culprit, it has been suggested that the prior should be uniform over the qualifying population.[55] For example, if 1,000 people could have committed the crime, the prior probability of guilt would be 1/1000.
The use of Bayes' theorem by jurors is controversial. In the United Kingdom, a defenceexpert witness explained Bayes' theorem to the jury inR v Adams. The jury convicted, but the case went to appeal on the basis that no means of accumulating evidence had been provided for jurors who did not wish to use Bayes' theorem. The Court of Appeal upheld the conviction, but it also gave the opinion that "To introduce Bayes' Theorem, or any similar method, into a criminal trial plunges the jury into inappropriate and unnecessary realms of theory and complexity, deflecting them from their proper task."
Gardner-Medwin[56] argues that the criterion on which a verdict in a criminal trial should be based isnot the probability of guilt, but rather theprobability of the evidence, given that the defendant is innocent (akin to afrequentistp-value). He argues that if the posterior probability of guilt is to be computed by Bayes' theorem, the prior probability of guilt must be known. This will depend on the incidence of the crime, which is an unusual piece of evidence to consider in a criminal trial. Consider the following three propositions:
A – the known facts and testimony could have arisen if the defendant is guilty.
B – the known facts and testimony could have arisen if the defendant is innocent.
C – the defendant is guilty.
Gardner-Medwin argues that the jury should believe bothA and not-B in order to convict.A and not-B implies the truth ofC, but the reverse is not true. It is possible thatB andC are both true, but in this case he argues that a jury should acquit, even though they know that they will be letting some guilty people go free. See alsoLindley's paradox.
Bayesian epistemology is a movement that advocates for Bayesian inference as a means of justifying the rules of inductive logic.
Karl Popper andDavid Miller have rejected the idea of Bayesian rationalism, i.e. using Bayes rule to make epistemological inferences:[57] It is prone to the samevicious circle as any otherjustificationist epistemology, because it presupposes what it attempts to justify. According to this view, a rational interpretation of Bayesian inference would see it merely as a probabilistic version offalsification, rejecting the belief, commonly held by Bayesians, that high likelihood achieved by a series of Bayesian updates would prove the hypothesis beyond any reasonable doubt, or even with likelihood greater than 0.
Thescientific method is sometimes interpreted as an application of Bayesian inference. In this view, Bayes' rule guides (or should guide) the updating of probabilities abouthypotheses conditional on new observations orexperiments.[58] The Bayesian inference has also been applied to treatstochastic scheduling problems with incomplete information by Cai et al. (2009).[59]
In the 20th century, the ideas of Laplace were further developed in two different directions, giving rise toobjective andsubjective currents in Bayesian practice. In the objective or "non-informative" current, the statistical analysis depends on only the model assumed, the data analyzed,[67] and the method assigning the prior, which differs from one objective Bayesian practitioner to another. In the subjective or "informative" current, the specification of the prior depends on the belief (that is, propositions on which the analysis is prepared to act), which can summarize information from experts, previous studies, etc.
In the 1980s, there was a dramatic growth in research and applications of Bayesian methods, mostly attributed to the discovery ofMarkov chain Monte Carlo methods, which removed many of the computational problems, and an increasing interest in nonstandard, complex applications.[68] Despite growth of Bayesian research, most undergraduate teaching is still based on frequentist statistics.[69] Nonetheless, Bayesian methods are widely accepted and used, such as for example in the field ofmachine learning.[70]
^Gelman, Andrew; Carlin, John B.; Stern, Hal S.; Dunson, David B.; Vehtari, Aki; Rubin, Donald B. (2013).Bayesian Data Analysis, Third Edition. Chapman and Hall/CRC.ISBN978-1-4398-4095-5.
^Robins, James; Wasserman, Larry (2000). "Conditioning, likelihood, and coherence: A review of some foundational concepts".Journal of the American Statistical Association.95 (452):1340–1346.doi:10.1080/01621459.2000.10474344.S2CID120767108.
^Sen, Pranab K.; Keating, J. P.; Mason, R. L. (1993).Pitman's measure of closeness: A comparison of statistical estimators. Philadelphia: SIAM.
^Lehmann, Erich (1986).Testing Statistical Hypotheses (Second ed.). (see p. 309 of Chapter 6.7 "Admissibility", and pp. 17–18 of Chapter 1.8 "Complete Classes"
^Le Cam, Lucien (1986).Asymptotic Methods in Statistical Decision Theory. Springer-Verlag.ISBN978-0-387-96307-5. (From "Chapter 12 Posterior Distributions and Bayes Solutions", p. 324)
^Gács, Peter; Vitányi, Paul M. B. (2 December 2010). "Raymond J. Solomonoff 1926-2009".CiteSeerX10.1.1.186.8268.
^Robinson, Mark D & McCarthy, Davis J & Smyth, Gordon K edgeR: a Bioconductor package for differential expression analysis of digital gene expression data, Bioinformatics.
^Foreman, L. A.; Smith, A. F. M., and Evett, I. W. (1997). "Bayesian analysis of deoxyribonucleic acid profiling data in forensic identification applications (with discussion)".Journal of the Royal Statistical Society, Series A, 160, 429–469.
^Robertson, B. and Vignaux, G. A. (1995)Interpreting Evidence: Evaluating Forensic Science in the Courtroom. John Wiley and Sons. Chichester.ISBN978-0-471-96026-3.
^Evaristo, Jaivime; McDonnell, Jeffrey J.; Scholl, Martha A.; Bruijnzeel, L. Adrian; Chun, Kwok P. (2016-01-01). "Insights into plant water uptake from xylem-water isotope measurements in two tropical catchments with contrasting moisture conditions".Hydrological Processes.30 (18):3210–3227.Bibcode:2016HyPr...30.3210E.doi:10.1002/hyp.10841.ISSN1099-1085.S2CID131588159.
^Stigler, Stephen (1982). "Thomas Bayes's Bayesian Inference".Journal of the Royal Statistical Society.145 (2):250–58.doi:10.2307/2981538.JSTOR2981538.
Aster, Richard; Borchers, Brian, and Thurber, Clifford (2012).Parameter Estimation and Inverse Problems, Second Edition, Elsevier.ISBN0123850487,ISBN978-0123850485
Bickel, Peter J. & Doksum, Kjell A. (2001).Mathematical Statistics, Volume 1: Basic and Selected Topics (Second (updated printing 2007) ed.). Pearson Prentice–Hall.ISBN978-0-13-850363-5.
Phillips, L. D.; Edwards, Ward (October 2008). "Chapter 6: Conservatism in a Simple Probability Inference Task (Journal of Experimental Psychology (1966) 72: 346-354)". In Jie W. Weiss; David J. Weiss (eds.).A Science of Decision Making:The Legacy of Ward Edwards. Oxford University Press. p. 536.ISBN978-0-19-532298-9.
For a full report on the history of Bayesian statistics and the debates with frequentists approaches, readVallverdu, Jordi (2016).Bayesians Versus Frequentists A Philosophical Debate on Statistical Reasoning. New York: Springer.ISBN978-3-662-48638-2.
Bolstad, William M. (2007)Introduction to Bayesian Statistics: Second Edition, John WileyISBN0-471-27020-2
Winkler, Robert L (2003).Introduction to Bayesian Inference and Decision (2nd ed.). Probabilistic.ISBN978-0-9647938-4-2. Updated classic textbook. Bayesian theory clearly presented.
Lee, Peter M.Bayesian Statistics: An Introduction. Fourth Edition (2012), John WileyISBN978-1-1183-3257-3
Carlin, Bradley P. & Louis, Thomas A. (2008).Bayesian Methods for Data Analysis, Third Edition. Boca Raton, FL: Chapman and Hall/CRC.ISBN978-1-58488-697-6.
O'Hagan, A. and Forster, J. (2003).Kendall's Advanced Theory of Statistics, Volume 2B:Bayesian Inference. Arnold, New York.ISBN0-340-52922-9.
Robert, Christian P (2007).The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation (paperback ed.). Springer.ISBN978-0-387-71598-8.
Pearl, Judea. (1988).Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA: Morgan Kaufmann.
A. Hajek and S. Hartmann:Bayesian Epistemology, in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
S. Hartmann and J. Sprenger:Bayesian Epistemology, in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.
