Bayes' Theorem
Bayes' Theorem describes how to update the probability of a hypothesis given new evidence, relating the conditional probability P(A|B) to P(B|A) through prior probabilities. It is the mathematical foundation of Bayesian inference, where beliefs are updated as new data arrives. The theorem has transformative applications in medical diagnostics, spam filtering, machine learning, scientific hypothesis testing, and artificial intelligence.
Key Formula
P(A|B) = [P(B|A) × P(A)] / P(B)
LaTeX: P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}
| Symbol | Meaning | Unit |
|---|---|---|
| P(A|B) | Posterior probability: probability of A given evidence B | dimensionless (0 to 1) |
| P(B|A) | Likelihood: probability of observing B if A is true | dimensionless (0 to 1) |
| P(A) | Prior probability: initial probability of A before evidence | dimensionless (0 to 1) |
| P(B) | Marginal probability: total probability of evidence B | dimensionless (0 to 1) |
Worked Example
Problem
A disease affects 1% of a population. A diagnostic test is 95% sensitive (correctly identifies diseased) and 90% specific (correctly identifies healthy). If a person tests positive, what is the probability they actually have the disease?
Solution
Step 1: Define events: D = has disease, T = tests positive. Step 2: Priors: P(D) = 0.01, P(D') = 0.99. Step 3: Likelihoods: P(T|D) = 0.95 (sensitivity), P(T|D') = 0.10 (false positive rate = 1 − specificity). Step 4: Total probability: P(T) = P(T|D)·P(D) + P(T|D')·P(D') = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085. Step 5: Bayes' theorem: P(D|T) = P(T|D)·P(D)/P(T) = (0.95 × 0.01)/0.1085 ≈ 0.0875.
Answer
P(disease | positive test) ≈ 8.75% — despite a 95% sensitive test, most positives are false due to low disease prevalence
Bayes' Theorem: Components and Interpretation
| Component | Symbol | Meaning | Example (Medical Test) |
|---|---|---|---|
| Prior | P(A) | Belief before seeing evidence | P(disease) = 1% base rate |
| Likelihood | P(B|A) | How probable is evidence if A is true | P(positive | disease) = 95% |
| Marginal | P(B) | Overall probability of evidence | P(positive test) = 10.85% |
| Posterior | P(A|B) | Updated belief after evidence | P(disease | positive) = 8.75% |
| False positive impact | P(B|A')·P(A') | Evidence from non-cases | P(pos | healthy)×P(healthy) = 9.9% |
Interactive Tools
Wikimedia Commons, CC BY-SA
Related Terms
Conditional Probability
Conditional probability is the probability of an event A occurring given that another event B has already occurred, denoted P(A|B) and read "probability of A given B." It updates the original probability by restricting the sample space to the outcomes where B has occurred. Conditional probability is the cornerstone of Bayesian reasoning, decision trees, diagnostic testing, and machine learning classifiers.
Probability
Probability is a numerical measure of the likelihood that a specific event will occur, expressed as a value between 0 (impossible) and 1 (certain). It quantifies uncertainty by assigning weights to outcomes in a sample space, and forms the mathematical foundation for statistics, stochastic processes, and decision theory. Probability theory underpins fields as diverse as quantum mechanics, financial modeling, machine learning, and epidemiology.
Random Variable
A random variable is a function that assigns a numerical value to each outcome in a sample space of a random experiment, providing a bridge between probability theory and real-valued analysis. Discrete random variables take countable values (like the number of heads in coin flips), while continuous random variables take values over an interval (like height or temperature). Random variables are characterized by their probability distributions, which describe the likelihood of each possible value, and are fundamental to statistics, signal processing, and stochastic modeling.
Named after Reverend Thomas Bayes (1701–1761), an English statistician and Presbyterian minister. His essay "An Essay towards solving a Problem in the Doctrine of Chances" was published posthumously in 1763 by Richard Price. Pierre-Simon Laplace independently rediscovered and extended the theorem in 1774.