learn.ifs.name

A Mathematical Theory of Communication

Shannon, 1948, A Mathematical Theory of Communication (Bell System Technical Journal)

Subscriber deck — free during early access

3 cards in “5 Ergodic and mixed sources” · whole deck

5 Ergodic and mixed sources

  1. What is an ergodic source, and why assume one?
    One where every sequence has the same statistical properties (except for a set of probability zero): letter and \(n\)-gram frequencies in a long sequence converge to the same limits. It lets averages along one sequence stand for averages over the ensemble.
  2. What graph conditions make a Markov source ergodic?
    The graph doesn’t fall into two parts that can’t reach each other, and the greatest common divisor of the lengths of all its circuits is 1 (otherwise sequences have a periodic structure).
  3. What is a mixed source?
    A weighted mixture \(L = p_1 L_1 + p_2 L_2 + \ldots\) of ergodic components: one component is chosen at the start with probability \(p_i\), and the sequence then continues according to it indefinitely.

Source