Entropy

← Back to Information Theory

A measure of the uncertainty or information content of a random variable. Shannon entropy quantifies the minimum number of bits needed to encode messages from a source, forming the foundation of data compression.

Key Properties

• Shannon Entropy
• Information Content

---

mathematics-for-cs information-theory entropy