5. Information Theory (The Deep Root) · Concept 6 of 6
Redundancy
It is the extra, repeated, or predictable stuff in a signal that is not strictly needed to carry the message.
Same redundancy seesaw: trim it on the left to shrink files, add it on the right so signals survive a rough link.
What it is
Redundancy is the extra, repeated, or predictable part of a signal that the message does not strictly need.
Key facts
Redundancy R = 1 - (H / Hmax), where H = actual entropy (average bits of new info per symbol) and Hmax = max entropy if every symbol were equally likely and independent.
Printed English is roughly 75 percent redundant: real info is about 1-1.5 bits per letter vs Hmax of log2(26) = 4.7 bits per letter (Shannon, 1951).
CD-quality PCM = 44,100 samples/sec x 16 bits x 2 channels = 1,411,200 bits/sec (1411 kbps), zero compression.
Lossy codecs strip perceptual redundancy: MP3 at 320 kbps vs 1411 kbps CD = about 4.4 to 1 shrink; AAC 256 kbps is near-transparent for most ears.
FLAC and ALAC are LOSSLESS: they remove only statistical redundancy and shrink files about 40-60 percent with bit-perfect playback.
Shannon source coding theorem: you cannot losslessly compress below the entropy H; that is the hard floor.
Shannon channel coding theorem: ADDING redundancy lets you transmit near error-free up to the channel capacity C (bits/sec).
Hamming(7,4) error-correcting code sends 7 bits to carry 4 (3 parity bits) and fixes any single-bit error.
CD audio uses CIRC (Cross-Interleaved Reed-Solomon Code) and can correct a burst up to about 4,000 bits, roughly a 2.5 mm scratch.
Parity bit = 1 added bit making total 1s even or odd; catches any single-bit flip but cannot fix it.
How it works
Encoder studies the signal and finds patterns: silence, repeats, predictable next sample, near-identical L/R channels.
To SHRINK: it predicts each sample, stores only the small error, and entropy-codes that (lossless) or discards masked detail (lossy).
Lossy codecs also drop perceptually redundant data using a psychoacoustic masking model, not just statistical repeats.
To PROTECT: a transmitter ADDS structured redundancy (parity, FEC, packet repeats) before sending.
Receiver uses that added redundancy to detect and rebuild bits lost or flipped in transit.
The seesaw: less redundancy = smaller but fragile; more redundancy = bigger but survives a rough link.
Real examples
A long held organ note: thousands of near-identical samples an encoder stores as one pattern plus tiny corrections.
Stereo music where L and R are 95 percent the same: codec sends mid/side and skips the duplicated part.
A scratched CD that still plays perfectly because CIRC error-correction rebuilds the missing bits.
A digital wireless mic that walks behind a wall and does not drop out because FEC fills the lost packet.
A WAV (1411 kbps) vs the same track as a transparent 256 kbps AAC: same perceived sound, about 5x smaller.
How it helps in live sound
For playback to FOH, use lossless WAV/FLAC or 320 kbps minimum; never run a gig off 128 kbps MP3 (artefacts on big PA).
Pick DIGITAL wireless (Shure ULX-D, QLX-D, Sennheiser EW-DX) for shows: built-in FEC/redundancy resists dropouts vs analogue.
Run wireless in high-density mode when channel count matters more than range; it trades RF efficiency for solid links.
On Dante/AES67, use a redundant secondary network (Dante redundancy port) so one cable fault does not kill audio.
Keep masters lossless; only export lossy for client preview or web, so you never bake in artefacts you cannot undo.
More RF redundancy/FEC costs spectrum: coordinate frequencies and leave guard band so robust mics still fit your channel count.
Everyday analogy
Like reading 'u cn stll rd ths txt wth th vwls mssng' because language repeats itself so much the missing letters were never carrying real information.
Watch out
Myth: a 320 kbps MP3 is 'CD quality' because it sounds close; truth: it is lossy and permanently throws data away, so it is never bit-perfect like FLAC or WAV.
Fun fact
Claude Shannon measured English redundancy by having people GUESS the next letter of a sentence; how predictable it was revealed the language is about 75 percent redundant.
Key takeaways
Redundancy = the predictable, repeated, removable-or-addable part of a signal.
Trim it to SHRINK files (MP3, AAC, FLAC); add it to PROTECT signals (parity, FEC, CIRC).
Lossless removes only statistical redundancy (reversible); lossy also drops perceptually redundant data (permanent).
Entropy H is the hard floor: you cannot losslessly compress below it.
Same seesaw runs your file sizes AND your dropout-free wireless and digital audio.