Preprint Article Version 2 This version is not peer-reviewed

A Measure of Information Available for Inference

Version 1 : Received: 2 November 2017 / Approved: 2 November 2017 / Online: 2 November 2017 (15:14:40 CET)
Version 2 : Received: 11 May 2018 / Approved: 11 May 2018 / Online: 11 May 2018 (06:24:06 CEST)

How to cite: Isomura, T. A Measure of Information Available for Inference. Preprints 2017, 2017110020 (doi: 10.20944/preprints201711.0020.v2). Isomura, T. A Measure of Information Available for Inference. Preprints 2017, 2017110020 (doi: 10.20944/preprints201711.0020.v2).

Abstract

The mutual information between the state of a neural network and the state of the external world represents the amount of information stored in the neural network that is associated with the external world. In contrast, the surprise of the sensory input indicates the unpredictability of the current input. In other words, this is a measure of inference ability, and an upper bound of the surprise is known as the variational free energy. According to the free-energy principle (FEP), a neural network continuously minimizes the free energy to perceive the external world. For the survival of animals, inference ability is considered to be more important than simply memorized information. In this study, the free energy is shown to represent the gap between the amount of information stored in the neural network and that available for inference. This concept involves both the FEP and the infomax principle, and will be a useful measure for quantifying the amount of information available for inference.

Subject Areas

the free-energy principle; internal model hypothesis; unconscious inference; infomax principle; predictive information; independent component analysis; principal component analysis

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.