Shannon's channel capacity: C = B log₂(1 + S/N) bits per second
Image: de:Benutzer:Hejkal, CC BY-SA 3.0, via Wikimedia Commons
Channel capacity
Shannon's channel capacity: C = B log₂(1 + S/N) bits per second
Channel capacity is the theoretical maximum rate for reliable information transmission over a communication channel. Shannon's theorem states that this capacity is the highest information rate achievable with arbitrarily small error probability. Information theory, developed by Claude E. Shannon, provides a mathematical model to compute this capacity.
Example
Consider a channel with a bandwidth (B) of 3000 Hz and a signal-to-noise ratio (S/N) of 1000. Using Shannon's formula, the channel capacity (C) can be calculated as C = 3000 log₂(1 + 1000) ≈ 30,000 bits per second.
Understanding Shannon's channel capacity is crucial for designing efficient communication systems that approach theoretical limits of data transmission.
Related concepts
Shannon's source coding theorem: you can't compress below entropy
Shannon's theorem: Data compression can't exceed entropy limit
Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits
Entropy H = -Σ p(x) log₂ p(x) quantifies uncertainty in a system
Rate-distortion theory: minimum bits to represent data within distortion D
Rate-distortion theory: minimum bits to represent data within distortion D = R(D)
A fair die has entropy of log₂(6) ≈ 2.58 bits
A fair die's entropy: log₂(6) ≈ 2.58 bits
Binary search
Time complexity of binary search: O(log n) — halves search space each step
arithmetic intensity is
Arithmetic intensity = FLOPs / Bytes accessed
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews