Chapter 3 — Stochastic Processes
Companion material for Chapter 3. Covers definitions, wide-sense stationarity, ergodicity, power spectral density, and LTI systems.
§ 3.1 Definition
Ensemble vs. Time Averaging
Compare temporal averaging (along one realization) to ensemble averaging (across many realizations). Shows when they agree (ergodic) and when they differ.
§ 3.2–3.3 Expected Values
§ 3.4 Properties of Random Processes
§ 3.5 Stationarity
Wide-Sense Stationary (WSS)
AR(1) process autocorrelation with adjustable pole coefficient. Displays sample vs. theoretical ACF and compares ensemble sizes. Shows WSS conditions in action.
Vary the AR(1) pole coefficient and watch the autocorrelation function and power spectral density change together as a Wiener–Khinchin pair.
§ 3.6 Ergodicity
When Does Time Average = Ensemble Average?
Side-by-side: ensemble average at fixed time vs. time average over a single realization. Shows convergence for ergodic processes and divergence for non-ergodic ones.
§ 3.8 Power Spectral Density
Wiener–Khinchin Theorem
Observe how the ACF shape determines the PSD shape via the Fourier transform relationship (Wiener–Khinchin theorem).
Visualize the Fourier transform: build a signal from sinusoids and observe the frequency content.
See how finite observation windows affect ACF and PSD estimation — spectral leakage and bias vs. variance trade-off.
§ 3.9 LTI Systems and WSS Processes
LTI Output Statistics
Step-by-step animation of the convolution integral — the operation that defines an LTI system’s response.
Apply LTI systems to audio signals and hear the effect — connects system theory to the stochastic processing setting.
Full IIR/FIR filter analysis: impulse response, pole-zero plot, magnitude and phase response. Apply to audio or images.