The Covariance Ellipse

A bivariate Gaussian with mean μ = 0. Everything on this page is driven by the three numbers σ₁, σ₂, ρ — drag them and watch the cloud, its eigenvectors, and the conditional slice move together. Murphy, Probabilistic Machine Learning, §3.2.

Joint distribution  p(y₁, y₂)

Drag anywhere in the plot to move the conditioning line y₂ = c. Points inside the band are the ones you are conditioning on.

Conditional  p(y₁ | y₂ = c)  vs  marginal  p(y₁)

conditional marginal same x-axis as the plot above

Controls

Covariance matrix Σ

Eigendecomposition — shape of the ellipse
λ₁ (major)
λ₂ (minor)
axis tilt
elongation √(λ₁/λ₂)
|Σ|
Conditioning on y₂ =
E[y₁]
E[y₁ | y₂]
V[y₁]
V[y₁ | y₂] = σ₁²(1−ρ²)
variance explained ρ²

What to try

  1. Drag the conditioning line up and down. The orange curve slides, but its width never changes. That is Σ1|2 = Σ11 − Σ12Σ22−1Σ21 not depending on the observed value — you know how much you will learn before you look.
  2. Set ρ = 0. The conditional lands exactly on top of the marginal. Observing y₂ tells you nothing. For jointly Gaussian variables, uncorrelated really does mean independent.
  3. Push ρ toward 1. The ellipse collapses onto a line, λ₂ → 0, |Σ| → 0, and the conditional spikes. That is why the 1/√(1−ρ²) in the pdf blows up: Σ is going singular.
  4. Compare the presets. Full tilts at any angle (3 free parameters), diagonal is stuck axis-aligned (2), spherical must be a circle (1). That is Figure 3.5, and it is the flexibility-vs-parameters tradeoff in every Gaussian model.
  5. Watch the eigenvectors, not the axes. The pink arrow is the direction of greatest variance, length √λ₁. Finding that arrow is all PCA does.