An Essay Towards Solving a Problem in the Doctrine of Chances

"An Essay Towards Solving a Problem in the Doctrine of Chances" is a work in the mathematical theory of probability by Thomas Bayes, published in 1763, two years after its author's death. The essay was prepared for publication, and supplemented with amendments and additions, by Bayes's friend Richard Price.

The title reflects the contemporary use of "doctrine of chances" to mean probability theory, a usage introduced by the title of a book by Abraham de Moivre. Contemporary reprints of the essay instead carry the more descriptive title A Method of Calculating the Exact Probability of All Conclusions Founded on Induction.

The essay contains theorems of conditional probability that form the basis of what is now known as Bayes' theorem, together with a detailed treatment of the problem of setting a prior probability.

Overview

Bayes considered a sequence of independent trials, each resulting in either success or failure, with the probability of success on any given trial equal to some fixed but unknown quantity p between 0 and 1. He treated p itself as uncertain, assigning it a probability of falling in any subinterval of [0, 1] equal to the length of that subinterval — in modern language, a uniform distribution on [0, 1].

Conditional on the value of p, the outcomes of the trials are independent; unconditionally, they are not, since observing many successes makes a larger value of p more probable, which in turn makes success on the next trial more probable. Bayes's question was: given the number of successes and failures observed so far, what is the conditional probability distribution of p?

The answer is that the probability density function of p, given k successes in n trials, is

$f(p)={\frac {(n+1)!}{k!(n-k)!}},p^{k}(1-p)^{n-k}\qquad {\text{for }}0\leq p\leq 1$

(and f(p) = 0 outside that interval). This is the Beta distribution with parameters k + 1 and n − k + 1.

Outline

Bayes's preliminary results on conditional probability — particularly Propositions 3, 4, and 5 — imply the theorem now named for him. In his own words:

If there be two subsequent events, the probability of the second b/N and the probability of both together P/N, and it being first discovered that the second event has also happened, from hence I guess that the first event has also happened, the probability I am right is P/b.

Symbolically (see Stigler 1982), this is:

$P(B\mid A)={\frac {P(A\cap B)}{P(A)}},\quad {\text{if }}P(A)\neq 0$

which leads to Bayes's theorem for conditional probabilities:

$P(A\mid B)={\frac {P(B\mid A),P(A)}{P(B)}},\quad {\text{if }}P(B)\neq 0$

Bayes does not appear to have emphasized this identity itself. His real target was a broader inferential problem:

Given the number of times in which an unknown event has happened and failed [...] Find the chance that the probability of its happening in a single trial lies somewhere between any two degrees of probability that can be named.

The essay illustrates this with a man trying to estimate the ratio of "blanks" to "prizes" in a lottery. Having watched the lottery produce ten blanks and one prize, Bayes shows how to compute the probability that the true ratio of blanks to prizes lies between 9:1 and 11:1 — about 7.7%. He then extends the calculation as more draws accumulate: twenty blanks and two prizes, forty blanks and four prizes, and so on, until, after 10,000 blanks and 1,000 prizes, the probability rises to about 97%.

Bayes's central result, Proposition 9, is, in modern notation:

Assuming a uniform prior distribution on the binomial parameter p, after observing m successes and n failures,

$P(a<p<b\mid m,n)={\frac {\displaystyle \int _{a}^{b}{\binom {n+m}{m}}p^{m}(1-p)^{n},dp}{\displaystyle \int _{0}^{1}{\binom {n+m}{m}}p^{m}(1-p)^{n},dp}}$

Whether Bayes himself was "Bayesian" in the modern sense — that is, whether his interest lay in Bayesian inference specifically, rather than probability more generally — is unclear. Proposition 9 reads as "Bayesian" in framing a probability statement about the parameter p itself. Yet Bayes poses the underlying problem in frequentist-sounding physical terms: a ball is thrown at random onto a square table, and further balls are then thrown, landing to the left or right of the first with probabilities p and 1 − p respectively. (The table is often described in retellings as a billiard table and the ball as a billiard ball, though Bayes himself never calls them that.) The algebra is identical regardless of which interpretation is taken.

Richard Price and the existence of God

Richard Price discovered the essay and its theorem among Bayes's papers after Bayes's death, and believed it offered support for the existence of God. In his introduction to the essay, Price wrote:

The purpose I mean is, to show what reason we have for believing that there are in the constitution of things fixt laws according to which things happen, and that, therefore, the frame of the world must be the effect of the wisdom and power of an intelligent cause; and thus to confirm the argument taken from final causes for the existence of the Deity. It will be easy to see that the converse problem solved in this essay is more directly applicable to this purpose; for it shews us, with distinctness and precision, in every case of any particular order or recurrency of events, what reason there is to think that such recurrency or order is derived from stable causes or regulations in nature, and not from any irregularities of chance.

In modern terms, this is an instance of the teleological argument.

Versions of the essay

• Bayes, Thomas; Price, Richard (1763). "An Essay towards Solving a Problem in the Doctrine of Chances". Philosophical Transactions of the Royal Society of London. 53: 370–418. doi:10.1098/rstl.1763.0053.
• Barnard, G. A. (1958). "Studies in the History of Probability and Statistics: IX. Thomas Bayes's Essay Towards Solving a Problem in the Doctrine of Chances". Biometrika. 45 (3–4): 293–295. doi:10.1093/biomet/45.3-4.293.

References

Further reading

• Barnard, G. A. (1958). "Studies in the History of Probability and Statistics: IX. Thomas Bayes's Essay Towards Solving a Problem in the Doctrine of Chances". Biometrika. 45: 293–295.
• Stigler, Stephen M. (1982). "Thomas Bayes's Bayesian Inference". Journal of the Royal Statistical Society, Series A. 145: 250–258.
• Todhunter, Isaac (1865). A History of the Mathematical Theory of Probability from the Time of Pascal to that of Laplace. Macmillan.