ENTROPY
Scrabble letter values: E1N1T1R1O1P3Y4 = 12
Definition
- noun (communication theory) a numerical measure of the uncertainty of an outcome. — synonyms: information, selective_information
- noun (thermodynamics) a thermodynamic quantity representing the amount of energy in a system that is no longer available for doing mechanical work. — synonyms: randomness, S
Examples
- “the signal contained thousands of bits of information”
- “entropy increases as matter and energy in the universe degrade to an ultimate state of inert uniformity”
- “Ludwig Boltzmann defined entropy as being directly proportional to the natural logarithm of the number of microstates yielding an equivalent thermodynamic macrostate (with the eponymous constant of proportionality). Assuming (by the fundamental postulate of statistical mechanics), that all microstates are equally probable, this means, on the one hand, that macrostates with higher entropy are more probable, and on the other hand, that for such macrostates, the quantity of information required to describe a particular one of its microstates will be higher. That is, the Shannon entropy of a macrostate would be directly proportional to the logarithm of the number of equivalent microstates (making it up). In other words, thermodynamic and informational entropies are rather compatible, which shouldn't be surprising since Claude Shannon derived the notation 'H' for information entropy from Boltzmann's H-theorem.”