Bateson on several occasions described information as a “difference that makes a difference” (see particularly Form, Substance, and Difference), connecting information theory to metaphysical questions of identity.