Entropy Is the Only Finitely Observable Invariant

Citation:

Weiss, D. O., & Benjamin, Nathans, . (2006). Entropy Is the Only Finitely Observable Invariant. Discussion Papers. presented at the 5. Retrieved from /files/dp420.pdf

Abstract:

Our main purpose is to present a very surprising new characterization of the Shannon entropy of stationary ergodic processes. We will use two basic concepts: isomorphism of stationary processes and a notion of finite observability, and we will see how one is led, inevitably, to Shannon's entropy. A function J with values in some metric space, defined on all finite-valued, stationary, ergodic processes is said to be finitely observable (FO) if there is a sequence of functions Sn(x1,x2,...,xn) that for all processes \S converges to J(\S) for almost every realization x1ˆ\v z of \S. It is called an invariant if it returns the same value for isomorphic processes. We show that any finitely observable invariant is necessarily a continuous function of the entropy. Several extensions of this result will also be given.

Website