learn.ifs.name

A Mathematical Theory of Communication

Shannon, 1948, A Mathematical Theory of Communication (Bell System Technical Journal)

Subscriber deck — free during early access

3 cards in “14 Discussion” · whole deck

14 Discussion

  1. Why is the noisy coding theorem hard to use directly in practice?
    Its proof is close to an existence proof: following it to build codes is impractical, and apart from trivial cases Shannon knew no explicit sequence of codes approaching the ideal, which he linked to the difficulty of explicitly constructing good approximations to random sequences.
  2. Why does the redundancy of English help when transmitting over a noisy channel?
    Redundancy left in the source helps combat noise if the receiver uses it: telegraphy doesn’t compress away English’s ~50% redundancy, so a sizable fraction of letters can be garbled and still be reconstructed from context.
  3. What extra role does coding delay play in the noisy case?
    Besides matching probabilities to signal lengths, it lets a large sample of noise affect the signal before the receiver judges what was sent; larger samples sharpen the statistical decision.

Source