CIS 725
Homework 1
Due 9/24 at the beginning of class. I'm not terribly picky about
formatting -- handwritten, typed, whatever. As long as it's legible
and reasonably well organized.
Problem 1
In our various texts you'll find 3 discussions of why Shannon's formula
for H is the only possible one. (One is in Shannon's paper, one is Ch.2
of Sayood, and one is in the online "primer"). Read over these and, in
about a page, explain in your own words why this formula is the one we
must use. (I expect your work will bear quite a bit of resemblance to the
sources, but I want to see that you understand it well enough to explain
it).
Problem 2
Consider a source which emits symbols with the following probabilities:
A 1/8
B 1/8
C 1/2
D 1/6
E 1/12
(a) What is the entropy/uncertainty of this source?
(b) How would these symbols be encoded using Shannon's technique?
(c) What is the average number of bits per symbol in this encoding?
(d) Your answers to parts a and c should be different. What accounts
for the difference?