Markov Blankets and Mirror Symmetries. Free Energy Minimization and Mesocortical Anatomy

1. Introduction

This paper argues that the free energy principle can be used to derive a model of neocortical self-organization accounting for anatomical structure and function at millimetric (mesoanatomical) scale.

Embryologically the neocortex develops in accord with the “structural model” [1,2,3,4]. Neuron precursors migrate and differentiate under genetic control along radial pathways from archi- and paleocortical precursors, the pathways of migration partly determining later functional connections between neocortex and subcortical systems [5,6]. As cellular differentiations proceed, cell connectivity also undergoes modification under the distance rule, [7,8] which describes inter-areal connectivity as an approximation to shortest and locally dense pathways – an arrangement facilitating metabolic efficiency and rapid interactions in a “small world”. Actual anatomy is a compromise between the radial versus small world arrangements [9]. Ultimately the neocortex becomes a closed extended sheet, embracing the subcortical systems from which it has arisen, and organised in depth as a six-layered structure of mixed excitatory and inhibitory cells [10]; essentially two layers of cell bodies with other layers in which inputs are received and lateral axons – largely excitatory - spread over greater distances. Among these laterally spreading axonal connections the superficial patch system is prominent [11,12,13].This is made up of patches of cells that make connections, skipping from patch to neighbouring patches in several steps, and thus forming gridworks apparently organized to distribute information between cortical locales.

Lateral organization is highly variable at mesoanatomical scale, although much effort has been made to systematize the appearances [14]. Some parts of the cortex - notably the primary visual cortex of large animals – are organised in a columnar fashion, in which zones of short axon neurons are surrounded by groups of superficial patch cells, creating macrocolumns, but elsewhere this organisation is minimal to apparently absent. Yet the superficial patch system is ubiquitous, and is overlapping where columnar organization is absent [12]. Within macrocolumns individual cells exhibit organisation according to the stimulus preferences of cells. Neurons that respond to straight line visual stimuli are organised about the centre of macrocolumns, with orientation preference from 0-180 degrees circling the centre from 0-360 degrees, creating a singularity [15]. Superficial patch cells have a tendency to link cells with common orientation preference in adjacent columns – “like to like” [16]. Cells also show orderly, but more complicated structured preferences for stimulus spatial and temporal frequencies [17]. In noncolumnar cortex neurons also show the same types of preference, but with minimal or no apparent order [18].

Puzzles surround the significance of columnar structure versus its absence, as well as the functional significance of the neuron preferences. In animals with columnar visual areas, organised orientation preferences are present at birth [19] without having required exposure to visual stimuli, yet structure is not sustained in later development if the animal is deprived of stimuli of any given orientation – cells with that preference being lost [20]. Particular difficulties surround the nature of ocular dominance columns. These are organised in stripes in parts of the visual cortex that receive inputs from both eyes in some animals with binocular vision. Orientation preferences surrounding their singularities form mirror reflections between side-by-side columns, and between singularities above and below in a single column [21] – exaggerating a tendency much less apparent in the monocular cortex. Inputs from both eyes are organised topographically, as a map of each eye’s visual field, with input from each eye alternating, column by column, with that of the other. This hints strongly of a locus in which comparisons can be made between each eye’s input, as required for binocular vision. However, not all animals with binocular vision have ocular dominance columns. New world owl moneys provide a particularly difficult instance [22]. They have either very poorly ordered, or absent, ocular dominance columns – yet it can be shown that they too receive alternating inputs from their eyes as if dominance columns were present. These and other findings have led to suggestions that columns are “spandrels” - viz: geometric forms without necessary function [23]. The view taken in this paper is that the columns represent one end of a spectrum of orderliness, but that a single organizational order underlies all.

Many theoretical formulations have been devised to account for limited aspects of mesocortical organization, without any wide-ranging explanation having achieved universal acceptance. In contrast, the free energy principle and its relationship to the concept of prediction error minimization, as advanced by Friston and colleagues [24,25,26,27,28,29,30,31,32,33,34] proposes an overarching explanation for self-organizing systems, including brain function as a specific instance. It draws parallels between laws of nature from the principle of least action to the organization of artificial and real intelligence. A central concept is provided by Jaynes’ linking of the maximum entropy principle of optimum statistical information to the laws of thermodynamics, and, in a further step of unification, to Bayesian inference [26,27,35], so that these concepts are duals, each to the others. As an example of such a duality, the interactions of coupled modes in a dynamical system are equivalent to iterative models of grammar [36].This means that a correct explanation of neuronal growth and dynamics is de facto an explanation of information processing – a relationship Friston terms “self-evidencing” – that neuronal and synaptic growth must tend inevitably toward a stable state in which perturbations created by inputs are predicted by the consequences of earlier learning, so that signals generated internally minimize the impact of uninformative current inputs on ongoing activity, both within the organism, and in its interactions with the environment. At an asymptotic limit (although never reached in life) a state of perfect adaptation to the environment is hypothetically attained. Another way to express this process is that within any system with a boundary via which it must interact with a surrounding environment, an open steady state must be reached in which equal and opposite signals are continuously exchanged via the boundary, so as to cancel each other. This boundary is termed by Friston a Markov blanket [37]. In neurophysiological terms this means at the asymptotic limit information exchanges between brain and environment would correspond exactly to their mutual information, and the variational free energy – effectively the uncertainty of the system – would be zero. A mapping of all sensory and motor interactions with the environment onto the structure of developed synaptic connectivity would have emerged, as pathways for neural signals replicating all the ways the organism has learned to interact with the world.

Prediction error minimization and free energy minimization within a canonical model of the cortical macrocolumn have been introduced within the structural model of the cortex [38,39]. Here we provide a broader account, showing how growth processes and the maintenance of excitatory/inhibitory balance result in a structure conforming to minimum free energy and prediction error minimization, and suggesting a functional unity underlies the paradoxical appearances in the anatomical findings outlined above.

To do this, first we present a theoretic account utilizing the free energy principle and allied neural field theory, to show that a particular anatomical order – one involving meso-scale mirror-symmetric systems of synaptic connection, and Markov blankets – ought to emerge during early development. This is followed by review of a biologically based explanation of (non)columnarity and feature preferences, leading to agreement of both accounts.

2. Basic Considerations

2.1. Cells

Consistent with the structural model we assume that genetic determinants govern the pathways of cell migration in embryogenesis, ultimately leading to characteristic six layered neocortex structure of excitatory and inhibitory neurons. Developing neurons operate close to metabolic limits imposed by their large surface area and high demands for ion pumping [40]. Synapses are few compared to the number of contacts made between axons and dendrites, so neurons form a sparse one-to-many network, with weak connectivity per synapse [41,42]. Neurons make synaptic contacts preferentially with neighbours, since dendritic and axonal trees are denser near their somas, but synaptic sparsity forces connection to jump intervening cells, so that closely placed neurons form densely interwoven and interpenetrating networks.

2.2. Pre-Synaptic Flux and Learning Rules

All neurons in the neural field exchange synaptic flux via all polysynaptic and monosynaptic routes to all other neurons. Peak synaptic flux delivered along all pathways of flow is given by

$${\phi }_{ij}\left(t+\frac{\left|i-j\right|}{\nu }\right)={\epsilon }_{ij}{g}_{ij}{\rho }_{ij}{Q}_j\left(t\right)$$

(1)

where${\phi }_{ij}$ is the pre-synaptic flux received at the $i-th$ neuron from the $j-th$ neuron, $Q_j$ is the pulse rate of the $j-th$ neuron, $\upsilon$ is the speed of signal spread, and $\frac{\left|i-j\right|}{\nu }$ is the delay from pulse generation to arrival of peak pulse density at pre-synapses, averaged over all routes. Synaptic gains are separated into three time-scales, so that ${\epsilon }_{ij},g_{ij},{\rho }_{ij}$ are the transient synaptic efficacy, the slow dynamic synaptic gain, and the structural synapse gains along the polysynaptic pathways, respectively – and are competitive on all three time-scales. The Hebbian gain terms in equation (1) follow the unification of fast and slow synaptic learning rules proposed by Izhikevich and Desai [43].combining short-term plasticity (STP) and short-term depression (STD), with the slower and more permanent Bienenstock-Cooper-Monro (BCM) rule, including slow “floating hook” negative feedback. Following dendritic summation of pre-synaptic pulses, pulse generation follows a sigmoid curve.

2.3. Excitatory/Inhibitory Balance

Homeostatic mechanisms keep the firing rates of cortical neurons and the balance of excitatory and inhibitory synaptic impulses within stable limits. Anti-Hebbian synaptic plasticity, not yet fully understood, acts to normalise excitatory and inhibitory synaptic gains, while leaving the relative strengths of Hebbian influences unchanged [44].

As observed experimentally, the time-course of anti-Hebbian plasticity is between hours and days. Yet rapid excitatory/inhibitory stabilization is essential as attested by the ease with which epilepsy can be triggered by local cortical irritation, or photic stimulation. The means by which ongoing rapid stabilization is maintained is key to our following arguments.

3. A minimum Free Energy Organizational Unit

3.1. Rationale

Equation (1) can be re-written as a state equation for the neural field, in matrix/vector form

$$\mathsf{\Phi }\left(t\right)=GDQ\left(t\right)$$

(1a)

$\mathsf{\Phi }\left(t\right)$ is a matrix of unidirectional pre-synaptic flows, $G\left(t\right)$ is a matrix of aggregate presynaptic gains associated with each flow, $D\left(t\right)$ is a delay matrix of axo-dendritic conduction times, and $Q\left(t\right)$ is a vector of action potential pulse rates.

$\mathsf{\Phi }\left(t\right)$ can be decomposed into spatial eigenmodes (patterns of synchronous flow) and fluxes coupling the spatial eigenmodes into time-varying patterns. $G\left(t\right)$ and $D\left(t\right)$ are a description of synaptic strengths and cell positions - with $\mathsf{\Phi }\left(t\right)$ leading the development of the growth of synapses and cells. By tracking development of $\mathsf{\Phi }\left(t\right)$ we should arrive at descriptions of both neural system dynamics, and the associated mesoanatomical order created by the consolidation of synaptic connections.

3.2. Constraints

Development is subject to three constraints:

Firstly, minimization of free energy:

$$\mathrm{F}=\mathrm{A}-\mathrm{C}\to 0$$

(2)

where $F$ is variational free energy, $A$ is total presynaptic flux autocorrelation, and $C$ is total presynaptic flux cross-correlation.

Secondly, minimization of perturbation from steady state, equivalent to prediction error minimization:

$$\mathsf{\Delta }{\mathsf{\Phi }}^{+}\left(t\right)-\mathsf{\Delta }{\mathsf{\Phi }}^{-}\left(t\right)\to 0$$

(3)

where $\mathsf{\Delta }{\mathsf{\Phi }}^{+}\left(t\right)$ and $\mathsf{\Delta }{\mathsf{\Phi }}^{-}\left(t\right)$ are each other’s negative vector sums. At asymptote, each acts to predict and minimize errors in the other. This constraint is here applied not only for sensory and motor exchanges, but to exchanges in the neural field at all levels.

Thirdly, maintenance of excitatory/inhibitory balance:

$$\sum {\mathsf{\phi }}_{\mathrm{e}}\to \sum {\mathsf{\phi }}_{\mathrm{i}}\to \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{y}$$

(4)

where $\sum {\phi }_e$ is the total excitatory pre-synaptic flux, and $\sum {\phi }_i$ is the total inhibitory pre-synaptic flux. This supplies a steady-state constraint that cell pulse rates remain stationary in the main, as development proceeds.

3.3. Minimization of Free Energy, $F\to 0$

At each stage of growth, although there are an increasing number, $n$, of unidirectional flows of presynaptic flux as synaptic and cell numbers are increasing, the total autocorrelation, $A$, during a relatively short epoch, $T$, at all lags, $\tau$, is

$$A=\frac{1}{2T}\sum _{ij}^n{\int }_0^T{\int }_{-\infty }^{+\infty }{\phi }_{ij}\left(t\right){\phi }_{ij}\left(t-\tau \right)d\tau dt$$

(5)

and for the $\frac{n}{2}$ pairs of bidirectional flows, total cross-correlation, $C$, is

$$C=\frac{1}{T}\sum _{ij,ji}^{\frac{n}{2}}{\int }_0^T{\int }_{-\infty }^{+\infty }{\phi }_{ij}\left(t\right){\phi }_{ji}\left(t-\tau \right)d\tau dt$$

(6)

Therefore free energy is zero when for all $i,j,t,\tau$,

$${\phi }_{ij}\left(t\right){\phi }_{ij}\left(t-\tau \right)+{\phi }_{ji}\left(t\right){\phi }_{ji}\left(t-\tau \right)={\phi }_{ij}\left(t\right){\phi }_{ji}\left(t-\tau \right)+{\phi }_{ji}\left(t\right){\phi }_{ij}\left(t-\tau \right)$$

(7)

On arithmetical grounds this requires that at least one term ${\phi }_{ij}$ is equal to a term ${\phi }_{ji}$ at each time-step, defining trajectories of perturbation about an equilibrium in which all four terms are equal, so that excitatory or inhibitory fluxes between all pairs of cells are symmetrically equal and opposite, and the system is time-stationary and periodic. This equilibrium condition is zero-lag synchronous oscillation [45]. For exchanges between pairs of excitatory cells and between pairs of inhibitory cells, $\tau =0$. Excitatory and inhibitory populations of neurons each fire synchronously and exchange equal flux. For exchanges between excitatory and inhibitory cells, $\tau =\left|i-j\right|/\nu$, equal to half the period of oscillation, so that excitatory and inhibitory cells fire in anti-phase. Perturbations result in feedback responses from opposite cells, so the oscillating equilibrium is stable. Fields of synchrony are spatial eigenmodes of $\mathsf{\Phi }(t$).

Conversely asymmetric exchanges of flux can mediate time-varying eigenmode couplings – that is, the control of perturbations about synchronous oscillation. As synaptic consolidation advances and learning approaches asymptote, then state transitions become noise-free and deterministic – implying the controlled perturbations are taking place near or at zero free energy.

There will be a tendency over time toward global synchrony, since persistence near equilibrium will continue to strengthen synaptic connections favouring synchronous equilibrium. The perturbations associated with eigenmode coupling necessarily become both minimal and efficient, analogous to the minimization of pdV work in thermodynamic systems as pressure and temperature differentials settle to stability. Leading to:

3.4. Minimization of Prediction Error, $\Delta {\Phi }^{+}\left(t\right)-\Delta {\Phi }^{-}\left(t\right)\to 0$

Symmetric exchanges of flux at equilibrium meet the condition by definition. Asymmetric exchanges must evolve to become paired so that each one of a pair generates a flux oppositely directed to the other as closely as possible – yet these cannot be between the same cells. This requires dual systems of connection, one system the mirror image of the other, and since these connections must mediate coupling between spatial eigenmodes rather than simply blocking eigenmode interactions, the eigenmodes themselves must occur as duplicates with mirror-symmetry.

3.5. Excitatory/Inhibitory Balance, $\sum {\phi }_e\to \sum {\phi }_i\to constancy$

Synchronous equilibrium itself requires equal exchange between excitatory and inhibitory neurons. If the entire field is to remain in balance, parts of the field in excess excitation must interact with parts of the field in excess inhibition, and vice-versa. Therefore each of the systems of coupled eigenmodes in a mirrored pair must interact with its partner to establish joint excitatory/inhibitory balance – necessarily requiring collision of travelling waves in the neural field – as follows.

3.6. Interactions within and between Spatial Eigenmodes

Figure 1 shows the four possible ways that adjacent areas in a neural field, each area part of a spatial eigenmode organization, can interact. In the middle column blocks of interacting excitatory and inhibitory cells constituting spatial eigenmodes are shown cross connected by excitatory links. The cross-connections shown are those of medium or long range connections, and are excitatory only, but short-range inhibitory cross-links follow exactly analogous forms.

In the top row, interaction is symmetrical and excitatory, maximizing co-synchrony while increasing total excitation. In the second row interaction is symmetrical and inhibitory. The fields cancel equal and opposite components in the colliding waves, permitting co-synchrony with reduction of total excitation. Asymmetric interactions shown in the remaining two rows mediate eigenmode cross-coupling, with an increase or reduction in total excitation respectively.

3.7. Mirror Symmetric Fields and Markov Blankets

Applying the mechanisms in Figure 1, Figure 2 illustrates a system composed of a pair of mirror-symmetric coupled spatial eigenmodes, each of the pair generating oppositely directed, colliding, travelling waves. The diagram shows the topology of the connections and flux exchanges – not a specific topography in the form shown. It is the synaptic connectivity that is essential, so the twin eigenmode systems might be separated by some distance, or their cell soma positions might be interdigitated.

Excitatory/inhibitory stabilization can take place at the line of wave collision, since excess of excitation or inhibition in waves from either side can be compensated at a fast time scale by shift between the symmetrical excitation and symmetrical inhibition modes of coupling - shifts modulated by the negative feedback “floating hook” property of the BCM rule, which diminishes synaptic gain in the more driven synapses. Adaptation may then be mediated more slowly by other cellular mechanisms of anti-Hebbian plasticity. As junctional exchange manages excitatory/inhibitory balance, prediction error minimization proceeds within each of the mirror duals, and free energy approaches zero. The signals arriving at the junction progressively maximize their mutual information. The mirror-like junction is therefore a Markov blanket, in Friston’s sense.

Interaction of dual systems can be generalised to multi-way interactions throughout the cortex, as continuously changing synaptic efficacies (equation (1)) modulate and segregate the patterns of pulses and synaptic flux present at any one instant. At a whole-brain scale, such a system is suited to minimize prediction error in interaction with subcortical systems and the external milieu – thus forming a large-scale Markov blanket between cortex and subcortex.

3.8. Redundancy and Information Storage

The hypothetical mirror-symmetric connection systems require a 2:1 redundancy of the information storage in their synapses. Using the Nyquist and Shannon-Hartley theorems, and considering $n$ directed synaptic couplings as unit-valued and composed of $S$ synapses that have been shaped by learning, with the remainder essentially random, thus

$$\frac{S}{n}=\frac{C}{A}\le 1$$

(8)

is the signal/noise ratio, and

$$D=n{log}_2\left(1+\frac{S}{n}\right)$$

(9)

is the number of bits needed to specify the information stored in the synaptic couplings, providing an upper limit of $2^D/2$ possible variants within similar mirror symmetric systems.

The approach to $S/n=1$ as learning becomes complete means that system determinacy is increasing, and is therefore an alternative way of expressing reduction in variational free energy.

4. Emergence of Mirrored Synaptic Maps in Actual Anatomy

It must now be shown whether dual mirror-and-blanket systems can be identified in cortical mesoanatomy, and explained how their development takes place.

The preceding argument showed that if free energy is minimized, maximization of synchrony is a consequence. In biological terms the converse argument is more easily made. Synchronous firing appears early in neuronal development along with the development of small world connectivity [46]. A substantial fraction of developing neurons succumb to apoptosis [47], and developing neurons prevented from entering into synchrony succumb to apoptosis [48,49]). The surviving cells thus form a matrix maximizing synchronous oscillation. A second factor in cell selection, minimization of total axonal length, lowering metabolic demand in the surviving cells, will assist their survival [40], and favour the evolution of a small-world configuration [7,50].

4.1. Columnar versus Noncolumnar Cortex

It is useful to first explain how, in this model, the difference between columnar versus noncolumnar cortex comes about. Simulation of cortical development [51] shows small world selection and selection for maximum synchrony can be in conflict. It is the relative length of long and short axon neurons included in the simulation that determines whether clearly columnar, or apparently diffuse, non-columnar organisation results. Suppose two populations of cortical neurons, with axonal tree distributions

$${\rho }_{\alpha }=N_{\alpha }{\lambda }_{\alpha }\exp \left[-{\lambda }_{\alpha }x\right]$$

(2)

$${\rho }_{\beta }=N_{\beta }{\lambda }_{\beta }\exp \left[-{\lambda }_{\beta }x\right]$$

(10b)

where ${\rho }_{\alpha }\left(x\right),{\rho }_{\beta }\left(x\right)$ are respective normalized densities of the axonal trees of long-axon, $\alpha$ cells, and short-axon, $\beta$ cells, as a function of distance, $x$, from their cell somas. The fraction of pre-synapses generated by the two cell types are $N_{\alpha },N_{\beta }$, and ${\lambda }_{\alpha },{\lambda }_{\beta }$, are their axonal inverse length constants.

Bidirectional connection density, ${\rho }_{\alpha +\beta }$, for all cells would be a maximum if

$${\rho }_{\alpha +\beta }\left(x\right)=N_{\alpha }{\lambda }_{\alpha }\exp \left[-{\lambda }_{\alpha }x\right]+N_{\beta }{\lambda }_{\beta }\exp \left[-{\lambda }_{\beta }x\right]$$

(3)

whereas density of connection in an ultra-small world network [52] where inter-soma distance is surrogate for increasing order of neighbour separation, is given by ${{\rho }_{\alpha +\beta }\left(x+k\right)}^{-2}$. Thus, disparity of connection density, $\mathsf{\Delta }\left(x\right)$, of an ultra-small world system and that of the axonal trees of $\alpha$ and $\beta$ cells is at best

$$\mathsf{\Delta }\left(x\right)={\left(x+k\right)}^{-2}-\left(N_{\alpha }{\lambda }_{\alpha }\exp \left[-{\lambda }_{\alpha }x\right]+N_{\beta }{\lambda }_{\beta }\exp \left[-{\lambda }_{\beta }x\right]\right)$$

(4)

and competitive processes maximizing synchrony (see below) force further departures in separation of cell bodies from the ultra-small optimum.

Simulations of cortical growth presumed that the axonal tree lengths are genetically determined and that the numbers of cells in the two populations are selected so as to optimise both synchrony and small world connectivity. For higher and more equal values of ${\lambda }_{\alpha }$ and ${\lambda }_{\beta }$ ultra-small world order is most closely approximated, therefore predominates, and columnar definition is not apparent. Where ${\lambda }_{\alpha }\ll {\lambda }_{\beta }$, maximization of synchrony among the numerous short-axon neurons is the predominant influence, and clearly columnar organization results. The loss of definition in the noncolumnar instances arises from the merging and inter-weaving of cell networks, made possible by the sparsity of synaptic connectivity. Whether apparently columnar or diffuse, simulations show that the same patterns of synaptic connections best maximizing synchrony are present, but are organized in interdigitated overlapping systems where small-world organization has predominated. Figure 3, bottom right, illustrates the way this merging takes place.

It is emphasised that In following account, the description of emerging patterns of synaptic connection is to considered general throughout cortex, although comparison with the clearly columnar visual cortex (V1), for many years the focus of experimental study, makes comparison between theory and experiment direct.

4.2. Early Embryonic Development

At the earliest stage developing synaptic connections are initially random, and polysynaptic pathways between any two neurons develop as cells and synapses proliferate, bringing about polysynaptic flows that are roughly bidirectionally symmetrical between all cells, so synchrony is early apparent in developing cells, as they begin to associate into small world systems. Bidirectional monosynaptic connections begin to develop, preferentially selected out of the polysynaptic flow between neurons, further increasing magnitude of synchrony.

As previously described, for simplicity we treat distribution of axonal lengths in the developing cells as two populations – one of excitatory cells with long axons, and a short axon population of mixed excitatory and inhibitory cells [53].

At a distance, $X$, from their cell bodies, the population density of the axonal trees of the short-axon and long axon cell populations are equal.

$$X=\frac{-ln\left(\frac{N_{\alpha }{\lambda }_{\alpha }}{N_{\beta }{\lambda }_{\beta }}\right)}{{\lambda }_{\beta }{-\lambda }_{\alpha }}$$

(5)

The short-axon, $\beta$, cells, whose axonal density is greatest at short range, preferentially form densest connections with each other at distances less than $X$, clustering into columnar-like systems. The long axon, $\alpha$, cells form preferential connections in patches where their cell bodies are closely situated, and because of competitive exclusion by $\beta$ cell synapse formation, form other preferential long-range connections at distances greater than $X$ - so that patches of $\alpha$cells form with skipping connections at lengths that are multiples of $X$, in a grid with edges of length $X$, enclosing clusters of short-axon cells. This reproduces the superficial patch cell network.

The long axon cells and short axon cells exchange bidirectional monosynaptic connections at distance $X.$ The upshot is that within each cluster the short-axon cells and their surrounding patches of long-axon cells project synapses to each other 1:1, maximizing synchrony by creating swaths of connection in arcs of a circle (in two dimensions) or segments of a spherical surface (in three dimensions) of radius $X$. Again because of synaptic sparsity, the formation of 1:1 maps is not confined to a simple Euclidean projection, but can project from the clusters of $\alpha$ cells to separate, interpenetrating, parts of the enclosed $\beta$ networks as the Rheimann projection that will best maximize joint synchrony. Positions in the $\alpha$-cell network can be considered as global positions in the cortical area, and designated complex number positions, $P$, while positions in any of the local $\beta$-cell clusters are designated $p$. (The complex plane positions may be further generalised to positions in three dimensions, as required.) As bidirectional monosynaptic connections emerge, they result in global-to-local maps of the form

$$P\leftrightarrow pwherep=\pm \sqrt{-1}k\frac{{\left(P-p_0\right)}^n}{{\left|P-p_0\right|}^{n-1}}+p_0$$

(6)

${(P-p_0)}^n/{\left|P-p_0\right|}^{n-1}$ describes angular multiplication by $n$ in the projection from $P$ to $p$. The factor $\sqrt{-1}k$ defines the rotation by 90 degrees and scale of the projection created by the arcs of synapses, chirality is shown $+$ or $-$, and $p_0$ is the centre of a short-axon $\beta$ cell cluster. This is a mirror-mapping in a topological sense – the global field being reflected in each local map.

Figure 3 left shows a reconstruction of these synaptic projections in the upper, and in the lower, layers of a developing column. The value of $n$ also represents the number of turns about the $\beta$ cell cluster centre made by sparse and interpenetrating $\beta$ cell networks before they form a closed self-exciting system, and the global-to-local projection must match the closed loop conformation in the form best maximizing synchrony.

The projection of $\alpha$ cells to $\beta$ cells from diametrically opposite sides of a local map, each at range $X$, forces their synapses to be deployed in arcs radiating from the local maps centre – either deployed on opposite sides of the map centre – in which case $n=1$ – or both radiating from the centre on the same side – in which case $n=2$. The $n=1$ case is a simple Euclidean mapping, whereas $n=2$ is a mapping analogous to the mapping of a plane onto a Mobius strip. The latter arrangement permits greater total synchrony by dint of the longer chains of connection among the sparse, but cross-connected, $\beta$ cell networks. Angles in the global field from $0-\pi$ are mapped locally from $0-2\pi$ in the plane view of the column, while global angles from $\pi -2\pi$ are also mapped (on a separate mesh of cells) from $0-2\pi$ in the same view, creating the form of an orientation preference singularity. Figure 3 top right shows how connections in the interpenetrating nets of sparsely connected cells can be construed in this way.

By forming mirror symmetry arrangement of adjacent local maps, homologous positions in the projections from the global map are brought into highest contiguity – thus enabling them to form connections further maximizing their joint synchrony. That is

$$\begin{array}{c}{p}_A\leftrightarrow p_{B}where{p}_A=+\sqrt{-1}k\frac{{\left(P-p_{0A}\right)}^2}{\left|P-p_{0A}\right|}+p_{0A}and{p}_B\\ =-\sqrt{-1}k\frac{{\left(P-p_{0B}\right)}^2}{\left|P-p_{0B}\right|}+p_{0B}\end{array}$$

(7)

$A$ and $B$ indicate adjacent local maps (columns). The arrangement may be discrete and columnar, or the adjacent maps may themselves be interpenetrating to variable degree in noncolumnar cortex, as shown bottom right in Figure 3 – synchrony will still be maximized.

Similarly, maps can form at different depths on the five-layered cortex. As these form in layers, each similarly oriented with regard to the surrounding global map, they are arranged in mirror symmetry in the axis of cortical depth.

Experimental findings explained by this model include patch cell clustering and interpatch order, the organisation of orientation preference (OP) in monocular areas of V1 including OP singularities, linear zones, and saddle points, and in binocular ocular dominance (OD) columns – also explaining the “like-to-like” connections made by patch cells to short-axon cells with common OP preference in separate local maps.

A critical test of this explanation of the organization of OP maps [53] was passed in the simulation of variation of OP when measured using moving visual lines with differing angle of attack, line length, and stimulus speed [54] – a finding explained by lag times of conduction in lateral intracortical connections.

A separate consideration applies to formation of mirror assemblies maximizing joint synchrony as cortico-cortical connections develop, creating inter-area linkage. Cortico-cortical projections form U-shaped loops in cortical white matter, projecting from one cortical area to its neighbours with mirror symmetry, and with subsequent onward projections to further cortical areas creating recurrent reversals of chirality [55]. This can be accounted for as a simple consequence of the form of the fibre projections [56] although the complexity of interareal connections and hierarchies makes the correspondence inexact.

Thus a multitude of mirror systems can tile the cortex, as adjacent columns, as interpenetrating sparse systems equivalent to columns, or as systems separated but interconnected by cortico-cortical connections. They can be mirrored in layers of cortical depth, with each layer laterally mirrored. They form mirrors between scales, as the patch system projects to each column or its non-columnar equivalent, and as mirrors between entire cortical areas. (Figure 4.) These alternative ways in which mirrors can be arranged are the set of topographies corresponding to the topology of the theoretical unit in Figure 2.

(a). 

Cortico-cortical and Inter-areal connections. Their U-shaped form projects each cortical area to its neighbours with mirror symmetry.

(b). 

Each local map interacts with the global map with (topological) mirror symmetry, as the local short-axon neurons exchange flux with the surrounding cortex via the patch cell system.

(c). 

Local cell groups interact with adjacent groups of opposite chirality - whether the groups interpenetrate, abut, or are further separated.

(d). 

Within every column mirror symmetry is generated between layers, while also able to interact laterally with other mirrored systems.

4.3. Later Embryonic and Early Antenatal Development

Early in antenatal life sensory afferents reach the cortex [57] and eventually impose complicated temporal structure on the inputs to the cortex, replacing the earlier stochastic exchanges. The radially symmetric mirror structures are now able to act as a scaffold upon which spatiotemporal images can be stored and read out.

4.4. Spatiotemporal Images

As an external stimulus is imposed upon the cortex, signals relayed intracortically from the global to the local scales, arrive in neighbouring macrocolumns within a short epoch. Concurrent arrivals at closely situated neurons generated from different positions and different times in the global field are able to promote synchrony and secondary formation of synapses between the local cells. It can be shown (ref) that this will result in the storage in the local map of the representation of a moving image $P,(t-\frac{\left|P-p\right|}{\upsilon })\to p\left(t\right)$. This is the basis of variation of OP with stimulus velocity and orientation mentioned above. Representations formed in this manner can differ in the information about the object represented. Information from widespread positions in the global field would better represent movement than shape, and from positions closely situated in the global field, the shape of the object. This may account for representations higher in the cortical hierarchy specializing in differing type of visual information – the dorsal and ventral visual streams [58].

Chains of such images would store more complicated sequences, and in motor cortex, reversal of the processes could be read out as spatio-temporal motor outputs.

4.5. Coupled Spatial Eigenmodes, Spatial and Temporal Frequency Preferences

The process generating spatiotemporal images is equivalent to the generation of coupled spatial eigenmodes, and explains other response preferences of V1 neurons [59]. Signals from positions in the global field circumferentially arrayed with respect to local maps generate a high frequency response in the local cells, in contrast to the lower frequency of responses elicited from radially positioned inputs. Synchronous fields thus generated are preferentially tuned to high frequencies and arrayed circumferentially within local maps, or tuned to low frequencies and arrayed radially. Adjacent circumferential high frequency domains are readily coupled by unidirectional excitatory couplings, as are adjacent radial low frequency domains – but the orthogonal disposition and poor frequency matching of high and low frequency domains leads them to be mutually antagonistic via their inhibitory surrounds. These properties account for the spatial (SFP) and temporal (TFP) frequency preferences of local cells [60,61]. High SFP cells (HSFP) occur most commonly in linear zones near the circumferential perimeter of macrocolumns. Low SFP (LSFP) zones are more scattered and radially located. At OP singularities either an HSFP domain or an LSFP domain is located - interpreted as competitive conflict forcing one or other outcome. Temporal frequency preferences (TFP) are accounted for along with SFP, since it is known that TFP = stimulus velocity x SFP [62] – as expected for intracortical laterally spreading signals. HSFP/HTFP and LSFP/LTFP zones thus appear to reveal the existence of coupled spatial eigenmodes on each macrocolumn.

Adjacent macrocolumns must receive inputs from the global field that are from the same stimulus, translated in space and time. Since the scaffold structure of each microcolumn approximates a mirror reflection of its neighbours, adjacent macrocolumns could interact with each other as envisaged in Figure 2, with the line of junction acting as a Markov blanket. By reaching a co-synchronous stable exchange, they would have abstracted and stored wider general characteristics of the stimulus object’s shape and movement.

5. Conclusions

We have shown that application of the free energy principle to a simple but realistic neural field leads to a theoretical unit of self-organization, constructed of mirrored assemblies of synaptic connections, and separated by a Markov blanket. On the other hand, simulations of neural growth, assuming that neurons that survive apoptosis approach optimum metabolic efficiency, leads to a closely compatible outcome – with provisos. The conflicting demands of maximized synchrony versus small-world organization means the outcome of growth simulations is expressed in a simple topographic relation to the theoretical unit only in columnar cortex. Yet the topology of connections would be identical, whether the cortex were columnar or noncolumnar. The sparsity of all neuronal connections accounts for the way a single unit of synaptic organization can be masked by the interpenetration of separated networks, and in a related way, sparsity of connection accounts for the form of OP singularities and the Mobius strip-like form that the mirrored assemblies must take.

By itself, the growth model has wide explanatory power, greater than any preceding model, for findings in the visual cortex – notably accounting for the topographic organization of OP, SFP, TFP, and like-to-like connections. It explains why OP maps are apparent at birth, since emergence of these structures requires only noise-like driving, and only a radially symmetrical structure appears at that stage. It also accounts for the results of postnatal visual deprivation. When exposed to postnatal visual inputs, the radially organised scaffold gives way to the mature organization of coupled spatial eigenmodes, and when deprived of certain features competitive processes eliminate representation of those features.

The theoretical unit, derived directly from the free energy principle, makes explicit an extra property not obvious from the growth model alone – the development of a Markov blanket between each pair of mirror ordered connections. This formulation is not in contradiction with the canonical model of error minimization, in which interactions in columnar depth mediate the minimization, but it is not identical to that model either. In contrast, opposed signals are not brought directly into matching interaction, but their cumulative effects on eigenmode coupling within each mirror assembly are brought to balance at the line of mirror junction. This has major functional implications, introducing local stabilization and the interplay of extensive co-synchrony with prediction error minimization throughout the neural field, at all scales and in cortical depth, without restricting error minimization to particular special systems. It provides a universal mechanism for the abstraction and storage of common features in cortical interactions at all scales - and incidentally provides an explanation for the enigmatic relationship of OD columns to binocular fusion in different species. Adjacent OD columns each receiving an input from the visual field from opposite eyes and reaching stability via their common Markov blanket can converge to represent information equivalent to a 3D image. But because this interaction does not require that the cortex be columnar, species without cortical columns in V1 can still have 3D vision – and have separated inputs from each eye in OD- like conformation.

If subsequently confirmed, this model seems to provide a reasonably sufficient preliminary model of cortical development and function, although lacking integration with cellular detail in many respects. It is restricted to neocortical self-organization, and begs the question of integration of cortical and subcortical systems not only as pathways of sensory and motor interactions with environment, but in the regulation of cortical arousal and attentional focus, and of reinforcement. These aspects have been given brief attention in regard to the growth model [63]. Integration with the details of the structural model of cerebral embryogenesis and its functional ramifications (5, 6) may be possible. As the cortex emerges by differentiation from cells of limbic origin, during its growth it would be brought, by the same process of predictive error minimization, into harmony with the developing limbic and subcortical systems, while concurrently more direct exchanges of neocortex and environment via major sensory and motor pathways develop.

The model is subject to further testing on a rather grand, but definitive level. In principle connectivity analysis in both columnar and noncolumnar cortex ought to establish that linkages of patch cells and short axon cell clusters are similar in both types - discrete in columnar cortex, and overlapping in noncolumnar cortex. At the same detailed microscopic level it should be possible to demonstrate that within short-axon clusters like-to-like connections terminate in a Mobius-like manner, on the interpenetrating and intertwined short-axon local cell networks.

Finally, it may be remarked that simulations of simplified neurons organised in mirror arrays may be practicable, and might provide an anatomically realistic framework in which to explore unsupervised learning.