Central limit theorem: what becomes Gaussian?

The central limit theorem approximates standardized averages without changing the distribution of individual observations. Which distribution narrows when we average fresh samples of skewed waiting times?

Summary

Suppose $X_{1}, \dots, X_{n}$ are independent, identically distributed observations with mean $μ$ and finite variance $0 < σ^{2} < \infty$ . Each repetition draws a fresh sample and computes $\overset{ˉ}{X}_{n} = n^{- 1} \sum_{i} X_{i}$ . The population law stays unchanged across repetitions.

Linearity and independence give exact identities:

E [\overset{ˉ}{X}_{n}] = μ, Var (\overset{ˉ}{X}_{n}) = \frac{1}{n ^{2}} i \sum σ^{2} = \frac{σ ^{2}}{n} .

The standard error, $σ / n$ , describes variation across repeated means, not individual observations. The theorem separately supplies a limiting shape:

Z_{n} = \frac{n ( X ˉ _{n} - μ )}{σ} d N (0, 1) .

This is convergence of distribution functions at continuity points, not exact finite-sample normality.

What becomes Gaussian?

Averaging changes the mean's distribution, not the original observations

Derivation

Read it this way: compare rows at the dashed mean. The top row describes unchanged individual observations. Each lower curve describes means across fresh independent samples. Their standard error is σ/√n. These are schematic curves, not exact finite-sample densities. The reference is OpenStax Introductory Statistics; the graphic is original.

Check: Does averaging make the individual observations Gaussian, or only change the distribution of their mean?

Read it this way: The original exponential population stays unchanged. Ten independent observations produce one mean. Here, its exact Gamma density is narrower. Its population variance is one tenth.
Does collecting means alter the distribution of an individual future observation?

What “enough samples” means

Use hypothetical waiting times with an exponential distribution of rate $0.5$ per minute. Then $μ = 2$ minutes and $σ^{2} = 4$ square minutes.

Independent observations	Mean of repeated averages	Variance	Standard error
$4$	$2$	$1$	$1$
$25$	$2$	$0.16$	$0.4$

At $n = 25$ , the threshold $\overset{ˉ}{X}_{n} > 2.8$ standardizes to $(2.8 - 2) /0.4 = 2$ . The Gaussian upper tail is about $0.02275$ . The exact sum has a Gamma distribution with shape $25$ and scale $2$ :

Pr (\overset{ˉ}{X}_{25} > 2.8) = e^{- 35} j = 0 \sum 24 \frac{3 5 ^{j}}{j !} \approx 0.032374.

The exact variance calculation did not require a Gaussian approximation; the tail calculation did. No universal sample count makes every tail approximation accurate.

Change the precision target

To halve standard error from $0.4$ to $0.2$ , solve $2/ n = 0.2$ . The answer is $n = 100$ , four times the observations. Doubling observations only divides standard error by $2$ . Use this calculation when precision, rather than sample count, is the design target.

Change the independent unit

Repeat each of five independent waiting times five times, producing 25 rows. Their average equals the average of the original five times. Its variance is $4/5 = 0.8$ , with standard error $0.8 \approx 0.894427$ , not $0.4$ . Covariances between repeated rows prevent the independent-row variance calculation. Use the actual independent units when designing clustered or repeated-measurement inference.

When CLT fails

Averages of independent standard Cauchy observations remain standard Cauchy. Their variance is undefined, so this theorem’s premises fail.

Finite variance alone does not make $n = 30$ sufficient. For Bernoulli probability $0.001$ , all 30 observations are zero with probability $0.99 9^{30} \approx 0.970431$ . That discrete mass defeats a universal bell-shape rule.

Reconstruct: independent observations have variance $9$ . Which sample count gives standard error $0.3$ ? Does it change the observation law?

Answer

Solve $3/ n = 0.3$ to obtain $n = 100$ . The observation law stays unchanged; only the standardized mean has the stated Gaussian limit.

Variants

For iid observations with finite third absolute centered moment $ρ$ , Berry-Esseen bounds approximation error:

x sup ∣ Pr (Z_{n} \leq x) - Φ (x) ∣ \leq C \frac{ρ}{σ ^{3} n} .

Here $Φ$ is the standard normal distribution function and $C$ is a universal constant. For this iid bound, $C = 0.4748$ is valid by Shevtsova’s upper estimate. The third-moment condition is stronger than finite variance. Large standardized third moments make this bound less informative.

Multivariate CLTs replace scalar variance with covariance for iid vectors with finite second moments. Lindeberg conditions handle suitable independent nonidentical triangular arrays. Martingale CLTs require conditional-variance convergence and controls on large increments; dependence alone does not establish them. Estimator-specific CLTs need their own expansions and regularity conditions. Neither arbitrary estimators nor degenerate U-statistics automatically have this mean’s normalization or Gaussian limit.

The law of large numbers concerns concentration toward $μ$ , not the standardized fluctuation’s shape. These are different conclusions even when both theorems apply.

Common pitfalls

Student’s exact small-sample $t$ law requires iid Gaussian observations and their sample variance. Consistent variance estimation instead permits asymptotic studentization under appropriate conditions. It is not a generic small-sample repair.

A bootstrap histogram describes resampling from observed data. It cannot establish coverage when the sample missed rare outcomes or used the wrong sampling unit. Metric uncertainty requires an estimand, independent units, a suitable scale estimate, and a justified approximation. Writing a metric plus or minus an unspecified standard deviation supplies none of those guarantees.