Shannon's Entropy

This miniMOOC teaches the math behind Shannon's entropy. It was created by Dr. Rivki Gadot (Open University , Lev Academic Center) & Dvir Lanzberg (the lecturer). It was made for undergraduate students, but we believe that anyone with a basic knowledge in calculus and algebra can get over the math in this miniMOOC.Shannon started by asking what is "information" and how it can be measured. Instead of giving a definition, he claimed that any function that measures information must have three properties, and proved that there is a single function that meets these three properties. This function become to be known as, Shannon's entropy.There are six short clips in this miniMOOC. Each clip is accompanied by exercises that let you deepen your understanding. The topics of the six clips are:

1. The problem - states what was the problem that Shannon tried to solve.
2. The solution - introduces Shannon's entropy.
3. The three properties - explains what are the three properties that every function for the amount of information, must have.
4. Equal probabilities - proves that Shannon's entropy is the only function that has the three properties, if the events' probabilities were equal.
5. Rational probabilities - proves that Shannon's entropy is the only function that has the three properties, if the events' probabilities were rational numbers.
6. Real probabilities - proves that Shannon's entropy is the only function that has the three properties, if the events' probabilities were real numbers.

The clips' presentation in PPTX format, and in PDF format (including lecturer's note).