learn.ifs.name

A Mathematical Theory of Communication

Shannon, 1948, A Mathematical Theory of Communication (Bell System Technical Journal)

Subscriber deck — free during early access

2 cards in “8 Representation of the encoding and decoding operations” · whole deck

8 Representation of the encoding and decoding operations

  1. How does Shannon model transmitters and receivers in the discrete case?
    As finite-state transducers: output \(y_n = f(x_n, \alpha_n)\) and next state \(\alpha_{n+1} = g(x_n, \alpha_n)\). A transducer is non-singular if an inverse transducer can recover its input.
  2. What happens to entropy when a source passes through a finite-state transducer (Theorem 7)?
    The output is again a finite-state source with entropy per unit time no greater than the input’s; equal if the transducer is non-singular. Encoding cannot create information.

Source