Frank E. Ritter
19 Jan 2001
Explain H= - sum ( Pi Log(Pi) )
Where Pi is the probability of each event, and log is computed base 2.
Homework
1. Given 6 different characters (e.g. a-f), compute how many bits of information there are. (Assume an equal distribution of characters).
2. Find the distribution of letters in the english language (from the net, from a book, by counting them even in this document). How much information is there in a single letter typically? How does this compare to the 7 and 8 bit ascii characters?
3. Find the Morse code. Examine its design using Shannon's H (if you can) and the distribution of the English language. Is it well designed? How ould it be improved?
[Hint: compute the information of each character and compare it to its length. Perhaps draw a tree of how the codes are related.]