The Universality Phenomenon in Probability

The Universality Phenomenon

Many systems in nature are built from a large number of microscopic "degrees of freedom." In probabilistic and statistical models, one often represents those degrees of freedom by random variables \(X_1,X_2,\dots\), and for each system size \(n\) studies an observable \[ F_n(X_1,\dots,X_n). \] We are interested in the "macroscopic" behavior, so we “zoom out” by sending \(n\to\infty\). Our interest is often in whether we can find parameters \(a_n\) and \(b_n\) so that when we center and scale appropriately, \[ \frac{F_n(X_1,\dots,X_n)-a_n}{b_n}, \] we see an interesting limit. If this feels abstract at first, don’t worry. We will turn to concrete examples that illustrate what this all means in a moment.

Universality is the observation that very frequently different microscopic models can exhibit the same large-scale behavior after the correct normalization. In many examples, the precise constants in these terms depend on the model, while their orders of growth are shared across a wide class of models and physical phenomena with the same bhavior. This collection is called a "universality class." The shared growth rates are often expressed in terms of "scaling exponents." The universality phenomenon is ubiquitous and is responsible for much of the power of statistical methods. To illustrate the idea, we now turn to some concrete examples.

The Gaussian universality class

The most familiar universality class is the Gaussian one. It appears when a quantity is built from many random contributions that are being added together and are roughly independent and similarly distributed, where no single term dominates the rest. In the classical setting one works with i.i.d. random variables, but exact independence and exact identical distribution are not absolutely necessary. Here we stay in the finite-variance setting. If the variance is infinite, different stable-law behavior can appear.

Start with observations \(X_1,\dots,X_n\), where \(X_i\) is the \(i\)-th sample. The basic statistic is the sample mean The sample mean x bar sub n equals x one plus x two plus through x n, all divided by n. \[ \overline{X}_n=\frac{X_1+\cdots+X_n}{n}. \] We expect \(\overline{X}_n\) to get close to the true mean \(\mu=\mathbb{E}[X_1]\) as \(n\) grows. The more refined question is how \(\overline{X}_n\) fluctuates around \(\mu\). Using the variance identity for uncorrelated random variables, if \(\sigma^2=\mathrm{Var}(X_1)\), then The variance of the sample mean equals sigma squared divided by n. \[ \mathrm{Var}(\overline{X}_n)=\mathrm{Var}\left(\frac{X_1+\cdots+X_n}{n}\right)=\frac{\sigma^2}{n}. \] This tells us the fluctuations of \(\overline{X}_n\) are of order \(n^{-1/2}\). To see a non-trivial limit, it is natural to standardize: The normalized sample mean is square root of n times the sample mean minus mu, divided by sigma. This quantity has mean zero and variance one. \[ \frac{\sqrt{n}(\overline{X}_n-\mu)}{\sigma}, \] which has mean \(0\) and variance \(1\). In the finite-variance setting, the Gaussian law describes the fluctuations of this normalized sample mean.

Here \(\Longrightarrow\) means convergence in distribution, and \(N(0,1)\) denotes the standard Gaussian distribution.

Figure 1. Gaussian density

Convergence in distribution here means that if we repeatedly draw samples of size \(n\) and compute the normalized sample mean each time, then for large \(n\) the resulting values have a distribution that is close to \(N(0,1)\). Figure 2 illustrates this with simulations: averages of coin flips, uniform random variables, and exponential random variables begin with very different distributions, but after centering and scaling their fluctuations are described increasingly well by the same Gaussian law.

The same phenomenon appears in practical averages built from observed data. In that setting the population mean and standard deviation are not known, so we estimate them from the full dataset and then standardize the averages we want to study.

The classical extreme-value universality classes

The Gaussian universality class is ubiquitous because sums and averages appear throughout statistics, but it is not the only recurring universal large-scale phenomenon. Another classical example appears when the statistic of interest is the maximum rather than the average. In the classical setting, one studies maxima of large i.i.d. samples and asks how those maxima behave after the right centering and scaling.

Start with the maximum M sub n is the maximum of x one through x n. \[ M_n=\max\{X_1,\dots,X_n\}. \] As in the Gaussian case, the question is not just whether \(M_n\) grows, but how it fluctuates after the right normalization.

The underlying distribution can vary enormously, but after centering and scaling the maxima, the limiting law must fall into this one generalized extreme-value family. Figure 5 shows the three classical parameter regimes first in simulated data.

Classical extreme-value ideas also appear in real data, but now one begins with an observed series and extracts maxima from blocks of time rather than from repeated i.i.d. samples generated by a model.

For an empirical illustration, we can take a long observed series, divide it into blocks of time, compute one maximum from each block, and compare the resulting histogram with a fitted extreme-value density.

Toward the KPZ universality class

The classical extreme-value theorems describe maxima of independent samples, or of block maxima treated as approximately independent. A natural next question is what happens when the quantity of interest is still extremal, but the competing random quantities are tied together by geometry and strong dependence.

Some models of this kind fall into the KPZ universality class. Last-passage percolation is a basic example: one maximizes over path weights, but overlapping paths create strong correlations.