Shannon Entropy
A measure of average information uncertainty in a probability distribution, widely used in coding, compression and analysis of binary data.
Shannon entropy measures the average information uncertainty of a probability distribution, commonly expressed in bits when logarithms are base two.
Definition
For a discrete source, Shannon entropy is:
H(X) = - Σ p(x) log2 p(x)Uncertainty increases as the distribution becomes more balanced and approaches zero as one outcome becomes certain.
Binary and Forensic Analysis
High byte-level entropy can appear in compressed, encrypted or naturally irregular data. It is therefore not proof of encryption or malicious content by itself.
In forensic analysis, entropy can be useful for locating candidate regions when combined with structural evidence.
Compression
Entropy provides a theoretical view of average information content. Actual compression ratio also depends on the source model, dependencies, coding overhead and implementation.