Chapter 1 — Introduction
Companion material for Chapter 1. The lecture motivates why randomness and stochastic signals matter and how they arise in communication systems.
§ 1.1 Why This Course?
Goals
What Is Randomness?
We use probability in everyday language. But what do we mean?
Even the canonical coin flip is less obvious as it might first seem:
“How random is a coin toss?” — Veritasium. The lecture’s opening question made concrete.
The nature of randomness is a subject of significant philosophical debate, which we will largely ignore in this course. However, this article may provide some useful insights if you are interested: What is randomness?.
Pseudo-random number generators (PRNGs) are deterministic algorithms that generate sequences intended to mimic randomness. A famous method is the Mersenne Twister.
Uncertainty
Rather than randomness, we consider (epistemic) uncertainty. Edwin T. Jaynes advocated viewing probability theory as the logic of science, i.e., as a method for reasoning under uncertainty.
Why Stochastic Signals?
All real-world signals are partly stochastic, including those from sensors, images, audio, stock prices, and weather.
Transmission Scheme
Only stochastic signals carry information. If there is no uncertainty about a signal before observing it, then observing it provides no new information.