信息

Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.
-- Vaclav Havel
  1. 信息
  2. 信息熵
    1. 联合分布下的信息熵
  3. 条件熵
    1. 互信息
  4. 相对熵