Comparing Exact and Normal Probabilities

Why Compare an Exact Probability with a Normal Approximation?

The central limit theorem gives a way to approximate probabilities for sums when an exact calculation is inconvenient. But an approximation is not the same as the probability under the original discrete model. Comparing the two helps show whether discreteness or skewness is materially affecting the answer, and whether an approximation is adequate for the question at hand.

For a binomial or Poisson random variable, an exact probability can be calculated from its probability mass function. A normal approximation instead uses a continuous distribution with the same mean and variance. For an integer-valued variable, a continuity correction places the normal boundary halfway between adjacent integers. The comparison is informative: it does not turn the normal approximation into an exact result.

Definition: If \(X\) is integer-valued and \(Y\) is its normal approximation, the exact probability is calculated using the probability mass function of \(X\). The normal approximation is the corresponding probability calculated under \(Y\), usually with a continuity correction. The absolute approximation error is \(|P(X\in A)-P(Y\in A_{\mathrm{cc}})|\), where \(A\) is the event of interest and \(A_{\mathrm{cc}}\) is its continuity-corrected interval.

Exact and Normal Calculations

For \(X\sim\operatorname{Bin}(n,p)\), the exact probability of a range of integer values is a sum of binomial probabilities:

P(a\le X\le b) = \sum_{k=a}^{b}\binom{n}{k}p^k(1-p)^{n-k}, \qquad E[X]=np,\quad \operatorname{Var}(X)=np(1-p).

For \(X\sim\operatorname{Poisson}(\lambda)\), the corresponding exact calculation is a sum of Poisson probabilities, with \(E[X]=\operatorname{Var}(X)=\lambda\). In either case, the normal approximation uses \(Y\sim N(\mu,\sigma^2)\), where \(\mu\) and \(\sigma^2\) match the mean and variance of \(X\). The second parameter in \(N(\mu,\sigma^2)\) is the variance, so the standard deviation used in standardizing is \(\sigma=\sqrt{\sigma^2}\).

For an integer-valued \(X\), the event \(X\le k\) corresponds to the real interval ending at \(k+\tfrac12\); the event \(X\ge k\) begins at \(k-\tfrac12\). For a range \(a\le X\le b\), use boundaries \(a-\tfrac12\) and \(b+\tfrac12\). These adjustments are called continuity corrections.

Result: If \(X\) is approximately normal with mean \(\mu\) and variance \(\sigma^2\), then a continuity-corrected approximation to \(P(X\le k)\) is \(\Phi((k+0.5-\mu)/\sigma)\), where \(\Phi\) is the standard normal cumulative distribution function. This is an approximation, not an identity. It may be inaccurate when the distribution is strongly skewed, the probability of interest is in a tail, or the distribution has substantial discreteness relative to its spread.

Why the Continuity Correction Uses Half-Units

The correction follows from assigning each integer its interval of real values closer to it than to any other integer. The values closer to \(k\) than to its neighbors lie between \(k-\tfrac12\) and \(k+\tfrac12\). Thus, the integer event \(X\le k\) corresponds to the continuous region extending up to \(k+\tfrac12\), not merely up to \(k\). A normal curve has no point masses, so integrating up to the integer boundary can systematically omit or include part of the interval that represents a discrete outcome.

This explains the correction, but does not guarantee high accuracy. A normal curve has a symmetric shape, while binomial and Poisson distributions can be skewed, especially when their means are small or a binomial probability is far from one half. Exact calculation provides a useful benchmark whenever it is feasible.

Worked Example: A Binomial Lower Tail

A hypothetical quality check examines 12 independently selected components. Each component is defective with probability \(0.25\), independently of the others. Let \(X\) be the number of defectives. Compare the exact probability of at most two defectives with a continuity-corrected normal approximation.

Method and model. The assumptions give \(X\sim\operatorname{Bin}(12,0.25)\). Its mean and variance are \(12(0.25)=3\) and \(12(0.25)(0.75)=2.25\), so the approximating variable is \(Y\sim N(3,2.25)\), with standard deviation \(1.5\).

Exact probability. Sum the probabilities for zero, one, and two defectives:

\begin{aligned} P(X\le 2) &=\sum_{k=0}^{2}\binom{12}{k}(0.25)^k(0.75)^{12-k}\\ &=\frac{3^{12}+12(3^{11})+66(3^{10})}{4^{12}}\\ &=\frac{531441+2125764+3897234}{16777216}\\ &=\frac{6554439}{16777216}\\ &\approx 0.3907. \end{aligned}

Normal approximation. Apply the upper boundary \(2.5\), then standardize:

P(Y\le 2.5) = \Phi\!\left(\frac{2.5-3}{1.5}\right) = \Phi(-0.3333) \approx 0.3694.

The exact probability is approximately \(0.3907\), while the corrected normal result is approximately \(0.3694\). Using unrounded values, the absolute difference is about \(0.0212\). The normal approximation understates this lower-tail probability; the continuity correction does not remove all effects of the binomial distribution's shape.

Common mistake avoided: Using \(\Phi((2-3)/1.5)\) places the boundary at \(2\), not \(2.5\), and gives a different approximation. For the event \(X\le2\), the correction is \(2+0.5\).

Comparing Errors in a Tail Probability

An absolute difference describes the error in probability units. A relative difference describes that error in relation to the exact probability. When the exact probability is small, a modest absolute error can be a substantial fraction of the probability being estimated. Neither measure alone determines whether an approximation is acceptable: the needed accuracy depends on how the result will be used.

Worked Example: A Poisson Upper Tail

Suppose the number of reported incidents in a hypothetical one-week period is modeled as \(X\sim\operatorname{Poisson}(3)\). Compare the exact probability of at least five incidents with its continuity-corrected normal approximation, and quantify their difference.

Exact calculation. The model has mean and variance \(3\). Using the complement,

\begin{aligned} P(X\ge5) &=1-P(X\le4)\\ &=1-e^{-3}\left(1+3+\frac{3^2}{2!}+\frac{3^3}{3!}+\frac{3^4}{4!}\right)\\ &=1-e^{-3}(1+3+4.5+4.5+3.375)\\ &=1-16.375e^{-3}\\ &\approx 1-16.375(0.0497871)\\ &\approx 0.1847. \end{aligned}

Normal calculation. Match the mean and variance with \(Y\sim N(3,3)\), whose standard deviation is \(\sqrt{3}\). The corrected lower boundary for \(X\ge5\) is \(4.5\):

P(Y\ge4.5) = 1-\Phi\!\left(\frac{4.5-3}{\sqrt{3}}\right) = 1-\Phi(0.8660) \approx 0.1932.

The absolute difference is approximately \(|0.1847-0.1932|=0.0085\). Using unrounded probabilities, the relative difference is approximately \(0.008502/0.184737=0.0460\), or \(4.60\%\). The approximation is somewhat higher than the exact tail probability.

Common mistake avoided: For an upper tail beginning at five, use the boundary \(4.5\), not \(5.5\). The correction moves the boundary halfway toward the excluded integer, four.

Discreteness and Skewness Can Matter Together

A continuity correction accounts for the gap between integer values, but not for asymmetry. A binomial distribution with success probability well below \(0.5\), for example, may have a longer right tail than a symmetric normal curve with the same mean and variance. An approximation can therefore be noticeably different even after its boundary has been corrected.

Worked Example: A Skewed Binomial Distribution

A hypothetical screening process flags each item independently with probability \(0.20\). For a batch of 10 items, let \(X\) be the number flagged. Compare the exact probability of no more than one flagged item with the corrected normal approximation.

Method and model. Here \(X\sim\operatorname{Bin}(10,0.20)\). The mean is \(10(0.20)=2\), the variance is \(10(0.20)(0.80)=1.6\), and the normal approximation is \(Y\sim N(2,1.6)\), with standard deviation \(\sqrt{1.6}\).

Exact calculation. The event includes zero and one flagged item:

\begin{aligned} P(X\le1) &=(0.80)^{10}+\binom{10}{1}(0.20)(0.80)^9\\ &=0.1073741824+10(0.20)(0.134217728)\\ &=0.1073741824+0.268435456\\ &=0.3758096384\\ &\approx 0.3758. \end{aligned}

Normal approximation. The corrected upper boundary is \(1.5\):

P(Y\le1.5) = \Phi\!\left(\frac{1.5-2}{\sqrt{1.6}}\right) = \Phi(-0.3953) \approx 0.3463.

Using unrounded values, the absolute difference is approximately \(0.02949\). The normal approximation is lower. The difference is not eliminated by the continuity correction, in part because this binomial distribution is right-skewed: its success probability is substantially below one half and its mean is small.

Common mistake avoided: The variance is \(1.6\), but standardization divides by the standard deviation \(\sqrt{1.6}\), not by \(1.6\). Also, the corrected cutoff for “no more than one” is \(1.5\), not \(0.5\).

How to Judge a Comparison

When both calculations are available, report them as distinct quantities and state their difference. Rounding should not obscure the comparison: calculate differences from unrounded values when possible, and round only the displayed final results. If values are rounded before subtraction, the displayed subtraction may not reproduce the reported difference.

For an exam-style calculation, a useful sequence is to identify the exact model and event, calculate the exact probability, match the normal mean and variance, apply the correct half-unit boundary, and then compare. The comparison should also mention the direction of the error. For example, saying that the approximation is high or low is more informative than reporting only an error magnitude.

Specify the discrete model and event.
State the distribution, parameters, and integer values included. Check whether the event is a lower tail, upper tail, point probability, or interval.

Calculate the exact probability.
Use the probability mass function or a complement, keeping sufficient precision for the eventual comparison.

Match a normal distribution and correct the boundary.
Use the same mean and variance, divide by the standard deviation, and place each boundary halfway between integers.

Compare and interpret.
Report the direction and size of the discrepancy. Consider whether discreteness, skewness, or the tail location may explain it.

A normal approximation is more convincing when the discrete distribution is not strongly skewed and the event is not an unusually remote tail. These are practical warning signs, not guarantees of accuracy. If an exact binomial or Poisson calculation is simple, it is a useful benchmark; when approximation error matters, the numerical comparison is stronger evidence than a rule of thumb alone.

Key takeaway: The exact probability belongs to the stated discrete model; the normal probability is an approximation. Match the mean and variance, use continuity-corrected boundaries, and compare unrounded results before deciding whether the discrepancy is material.

Check Your Understanding

For each problem, compare the exact discrete probability with the continuity-corrected normal approximation. Probabilities are rounded to four decimal places.

Let \(X\sim\operatorname{Bin}(8,0.25)\). Compare \(P(X=0)\) with its normal approximation, using the interval from \(-0.5\) to \(0.5\). (Answer: Exact \(0.1001\); normal approximation \(\Phi(-1.2247)-\Phi(-2.0412)\approx0.0897\).)
Let \(X\sim\operatorname{Poisson}(2)\). Compare the exact and corrected normal probabilities of at least three events. (Answer: Exact \(1-5e^{-2}\approx0.3233\); normal approximation \(1-\Phi(0.3536)\approx0.3618\).)
Let \(X\sim\operatorname{Bin}(6,0.5)\). Compare the exact probability of exactly three successes with the normal approximation over the corrected interval \((2.5,3.5)\). (Answer: Exact \(0.3125\); normal approximation \(\Phi(0.4082)-\Phi(-0.4082)\approx0.3169\).)
Let \(X\sim\operatorname{Bin}(10,0.5)\). Compare the exact and corrected normal probabilities of at least five successes, and state whether the approximation is high or low. (Answer: Exact \(0.6230\); normal approximation \(1-\Phi(-0.3162)\approx0.6241\), which is high.)
Let \(X\sim\operatorname{Poisson}(4)\). Compare the exact and corrected normal probabilities of at most two events. (Answer: Exact \(13e^{-4}\approx0.2381\); normal approximation \(\Phi(-0.7500)\approx0.2266\), which is low.)

What You'll Learn