learn.ifs.name

A Mathematical Theory of Communication

Shannon, 1948, A Mathematical Theory of Communication (Bell System Technical Journal)

Subscriber deck — free during early access

3 cards in “1 The discrete noiseless channel” · whole deck

1 The discrete noiseless channel

  1. How does Shannon define the capacity of a discrete noiseless channel?
    \(C = \lim_{T \to \infty} \frac{\log N(T)}{T}\), where \(N(T)\) is the number of allowed signals of duration \(T\): the growth rate of the log of the number of possible signals.
  2. What is the capacity of a teletype channel with 32 equal-length symbols sent at \(n\) symbols per second, any sequence allowed?
    Each symbol carries \(\log_2 32 = 5\) bits, so \(C = 5n\) bits per second: the maximum rate, reached only if the source feeding the channel makes full use of it.
  3. How did Shannon model a channel whose allowed symbol sequences are constrained, as in telegraphy?
    As a set of states with a graph: each state permits certain symbols, and sending one moves to a new state (e.g. after a space only a dot or dash may follow). Theorem 1 gives \(C = \log W\), with \(W\) the largest real root of a determinant equation built from the symbol durations.

Source