Topic Two: Shannon’s Theory
n Shannon defines a bit as the amount of
information required to decide between two
equally probable outcomes
n Example: a sequence of tosses of a fair coin
can be encode 1 bit per toss, such that heads
are 1 and tails 0.