A Caltech Library Service

Relating Thermodynamics to Information Theory: The Equality of Free Energy and Mutual Information


Feinstein, David I. (1986) Relating Thermodynamics to Information Theory: The Equality of Free Energy and Mutual Information. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/XVQB-7902.


In this thesis we uncover a new relation which links thermodynamics and information theory. We consider time as a channel and the detailed state of a physical system as a message. As the system evolves with time, ever present noise insures that the "message" is corrupted. Thermodynamic free energy measures the approach of the system toward equilibrium. Information theoretical mutual information measures the loss of memory of initial state. We regard the free energy and the mutual information as operators which map probability distributions over state space to real numbers. In the limit of long times, we show how the free energy operator and the mutual information operator asymptotically attain a very simple relationship to one another. This relationship is founded on the common appearance of entropy in the two operators and on an identity between internal energy and conditional entropy. The use of conditional entropy is what distinguishes our approach from previous efforts to relate thermodynamics and information theory.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Information theory, thermodynamcs, mutual information, free energy
Degree Grantor:California Institute of Technology
Division:Physics, Mathematics and Astronomy
Major Option:Applied Mathematics
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Cohen, Donald S.
Thesis Committee:
  • Mead, Carver (chair)
  • Cohen, Donald S.
  • Keller, Herbert Bishop
  • Hopfield, John J.
  • Franklin, Joel N.
Defense Date:21 May 1986
Non-Caltech Author Email:dif (AT)
Funding AgencyGrant Number
System Development FoundationUNSPECIFIED
Record Number:CaltechTHESIS:03212013-084718067
Persistent URL:
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:7537
Deposited On:21 Mar 2013 16:03
Last Modified:20 Dec 2019 19:38

Thesis Files

PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page