Topic Two: Shannon’s Theory
n
Shannon defines a bit as
the amount of
information required to decide between two
equally probable outcomes
n
Example
: a sequence of tosses of a fair coin
can be encode 1 bit per toss, such that heads
are 1 and tails 0.