Competitive neurocognitive functions can tell us what to learn and when: In schools and beyond

Author Note: Correspondence and requests for material should be addressed to Prof. Dezso Nemeth, CRNL – Lyon Neuroscience Research Center, Université Claude Bernard Lyon 1, Lyon, France. Address: CRNL – Centre Hospitalier Le Vinatier – Bâtiment 462 – 95 Bd Pinel – 69675 Bron Cedex – France. Phone: +33 4 81 10 65 46. Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 5 June 2020 doi:10.20944/preprints202006.0050.v1


3
Our physical, economic and social environment is subject to constant changes. For example, old job types disappear and novel ones emerge within a generation or even within a few years' time. There is also an increasing pressure to replace old routines with new ones and to develop new skills (e.g., learn how to use a new electronic device or software, or speak a second language). The changes to our environment present challenges both for individuals and for society as a whole. Education systems aim at providing skill sets to individuals to help them adapt to these changes more efficiently. National education authorities have increasingly incorporated recommendations into their national curriculum guidelines based on scientific evidence from developmental and cognitive psychology, although with a relatively long lag (Whitebread, 2011;Wood & Hedges, 2016). It is mainly in the last decade that links have been established between current neuroscientific knowledge and educational practices. A growing body of research has focused on how different neurocognitive functions, such as attention, executive functions, working memory and aspects of long-term memory are involved in learning in the classroom and linked to academic outcomes (Anderson, Betts, Ferris, & Fincham, 2013;Kaufman et al., 2010;Pluck et al., 2019;Posner & Rothbart, 2014). However, the accumulating knowledge on the interactions between different neurocognitive functions and how these interactions contribute to learning across development have largely been neglected. Here we aim to rectify this situation by briefly summarizing current neuroscientific knowledge on the interactions between two neural circuitry linked to distinct learning functions, their developmental trajectories, and how they interact with other neurocognitive functions, such as cognitive control or executive functions. We then discuss how this knowledge of interactive neurocognitive functions could inform educational policies and academic practices to improve academic outcomes.

Insights from neuroscience into interactive learning and memory functions
The existence of neuroanatomically distinct learning and memory functions that compute specific mnemonic operations is well established in the scientific community (Henke, 2010;Squire, 2004;Squire & Zola, 1996;Yin & Knowlton, 2006). In the last two decades, ample studies have reported that, depending on the task at hand, these functions interact intimately (Janacsek, Fiser, & Nemeth, 2012;Lee, Shimojo, & O'Doherty, 2014;Nemeth, Janacsek, Polner, & Kovacs, 2013;Otto, Gershman, Markman, & Daw, 2013;Packard & Knowlton, 2002;Poldrack et al., 2001). These interactions can be of a cooperative or competitive nature (Poldrack et al., 2001), and are mediated through direct neuroanatomical projections between the brain structures involved in the specific learning/memory functions, or indirectly through 4 neuromodulatory influences of other brain structures (Poldrack & Packard, 2003). In the following subsections, we review evidence for interactive (in particular, competitive) learning/memory functions from cognitive, developmental and computational neuroscience research.

Competition between neural circuitry underlying learning and memory
Ample evidence from cognitive neuroscience research suggests that, during learning and memory tasks, a competitive relationship exists between the neural circuitry involving the basal ganglia (BG), particularly the dorsal striatum, and the circuitry involving medial temporal lobe (MTL) structures such as the hippocampus (Albouy, King, Maquet, & Doyon, 2013;Moody, Bookheimer, Vanek, & Knowlton, 2004;Packard & Knowlton, 2002;Poldrack et al., 2001;Poldrack & Packard, 2003). For example, neuroimaging studies have shown that the more the learning relies on the circuitry involving the BG, the smaller the reliance on the circuitry involving the MTL, and vice versa (Moody et al., 2004;Poldrack et al., 2001). Similarly, the hippocampus appears to be more active when the BG are 'silenced' during consolidation periods, in which acquired memories are typically stabilized (Logothetis et al., 2012).
Traditionally, the BG and MTL have been associated with different learning/memory functions: the former has been linked to procedural memory that is involved in the acquisition of skills and habits, while the latter has been linked to declarative memory such as memory for events and facts (Squire, 2004;Squire & Zola, 1996). Another line of research has argued for differential involvement of the BG and MTL in habitual vs. goal-directed systems, which show some similarities with procedural and declarative memory, respectively (Yin & Knowlton, 2006), although they refer to somewhat different aspects of learning and memory. Importantly, neither of these distinctions can explain the interactions observed in previous studies, as the competitive relationship between the neural circuitry involving the BG vs. the MTL has been observed within a certain (e.g., procedural memory) task and not across tasks. Instead, the competitive relationship between these neural circuitry suggests that they are linked to more specific neurocognitive processes that operate in parallel and compete for the control of behavior in a given task.
Evidence for competitive neural circuitry involving the PFC comes from intervention studies. For example, studies using non-invasive (transcranial magnetic) brain stimulation to disrupt the dorsolateral PFC in healthy young adults found that the disruption of this region led to improved performance in aspects of procedural memory, presumably due to a greater reliance on the circuitry involving the BG (Ambrus et al., in press;Galea, Albert, Ditye, & Miall, 2010). Another study employed relaxational hypnosis to decrease the reliance on control functions that are linked to PFC regions, which led to improved procedural learning (Nemeth, Janacsek, Polner, et al., 2013).
On the functional level, a negative correlation has been observed between executive functions, that heavily rely on the PFC, and procedural learning, in that, better executive functions were associated with weaker learning (Nemeth, Janacsek, Polner, et al., 2013;Virag et al., 2015). In contrast, executive functions appear to have a positive relationship with other learning/memory functions, such as working memory and declarative memory (Ruchkin, Grafman, Cameron, & Berndt, 2004;Van der Linden, Meulemans, Marczewski, & Collette, 2000).
Overall, competition seems to exist between at least two neural circuitry underlying learning and memory, one that includes the BG and the other that includes the MTL. Regions in the PFC seem to be intimately involved in both circuitry and may arbitrate between them to determine which circuitry gains control over behavior in a certain situation (for more details, see the 'Evidence from computational neuroscience on competitive neurocognitive processes' subsection below). Thus, the relative dominance of the circuitry involving the BG vs. the MTL may affect what is learned and how learning occurs. Since these neural circuitry exhibit different developmental trajectories from childhood to adulthood (Blakemore, 2012;Raznahan et al., 2014), optimal learning strategies may also change during development.

Competitive neurocognitive functions in typical and atypical development
A growing body of research has shown that children (under the age of ~12-13 years) outperform adolescents and adults on learning/memory tasks that require the extraction of predictable patterns from the stimulus stream, which is an important aspect of procedural memory and skill learning (Janacsek et al., 2012;Juhasz, Nemeth, & Janacsek, 2019;Nemeth, Janacsek, & Fiser, 2013;Thompson-Schill, Ramscar, & Chrysikou, 2009). It has been proposed that this behavioral pattern can be explained by a relatively greater reliance on one neural 6 circuitry involved in learning compared to the other due to their different pace of maturation.
Specifically, since the circuitry involving the PFC and MTL/hippocampal regions is not fully developed in childhood (Blakemore, 2012;Shaw et al., 2008), it has been suggested that children rely on the circuitry involving the BG to a greater extent (Janacsek et al., 2012). When the circuitry involving the PFC and MTL/hippocampus becomes more mature, starting around adolescence and reaching its peak in young adulthood, competition emerges between the two circuitry.
An age-dependent shift in reliance on the two circuitry may explain previous findings and theories of reasoning and language learning (Goldowsky & Newport, 1993;Gopnik, Griffiths, & Lucas, 2015;Gopnik et al., 2017;Newport, 1990). For example, Gopnik et al. (2017) showed that during learning and reasoning about the causes of physical events, children perform better than adults despite their weaker executive functions, cognitive control and working memory capacity. They argue that across development from childhood to adulthood, learners become less flexible, in that they are less likely to adopt a novel hypothesis that is consistent with new evidence. Instead, adults learners prefer a familiar hypothesis even if it is less consistent with the evidence, suggesting an age-dependent shift in the relative reliance on exploration vs. exploitation (Gopnik et al., 2017). A childhood advantage was also reported in language learning (Newport, 1990). According to the 'Less is More Hypothesis', fewer cognitive resources (such as weaker executive functions and lower working memory capacity) can be advantageous in learning a complex combinatorial system such as language because these cognitive limitations constrain them to first address smaller units of language that have predictive value and carry meaning, and will acquire more complex constructions only as they mature (Goldowsky & Newport, 1993;Newport, 1990).
A different level of reliance on the circuitry involving the BG vs. the MTL during learning may also explain several findings in neurodevelopmental disorders such as specific language impairment, dyslexia and developmental coordination disorder. Research has shown that the circuitry involving the BG are affected in these neurodevelopmental disorders, resulting in specific neurocognitive patterns, outlined in the Procedural Circuit Deficit Hypothesis (Ullman, Earle, Walenski, & Janacsek, 2020). Acquisition of predictable patterns and sequences appears to be impaired in disorders affecting this circuitry but this may not be seen in behavior because they may compensate with a greater reliance on the neural circuitry involving the MTL (Ullman & Pullman, 2015). Surprisingly, cognitive functions that have been shown to heavily rely on this latter circuitry, such as declarative memory, may even be enhanced in these cases (Lukacs, Kemeny, Lum, & Ullman, 2017). The opposite pattern could 7 also be observed: the hyperactivation of the circuitry involving the BG in Tourette syndrome could even lead to enhanced performance during the acquisition of predictable patterns (Takács et al., 2018), suggesting a disproportionately larger reliance on this circuitry that happens to be beneficial for this type of learning.

Evidence from computational neuroscience on competitive neurocognitive processes
Accumulating evidence from computational neuroscience suggests that the interactions between learning/memory functions are driven by more specific cognitive processes. In particular, a widely recognized computational model has postulated the existence of modelbased and model-free processes underlying learning, and more generally, behavior (Daw et al., 2005). The central distinction between the two processes has been posited to lie in whether or not an internal model/representation of the task at hand is formed during learning, characterizing model-based vs. model-free processes, respectively. Since these processes appear to be relevant in a wide range of learning situations, they could potentially shed further light on the competition previously observed both on the neural and behavioral level (Daw et al., 2005;Decker, Otto, Daw, & Hartley, 2016;Lee et al., 2014;Otto, Gershman, et al., 2013).
Tasks or situations typically involve the evaluation of the available information and some decision making, such as action selection. Model-based processes have been proposed to select actions via a flexible but computationally demanding process of developing and searching a cognitive model of the task at hand. These models include predictions of long-term outcomes by chaining together short-term predictions about the immediate outcomes of each action in a sequence (Daw et al., 2005). Thus, model-based learning involves the exploration of a branching set of possible future situations, which can be resource-expensive in terms of memory capacity and time and can also be prone to errors. In contrast, model-free processes have been proposed to provide a less resourcedemanding way of evaluating information and selecting an appropriate action (Daw et al., 2005). In this view, instead of searching (sets of) possible future outcome sequences of our current decisions/actions, model-free processes rely on more recent outcomes when determining the action value associated with a stimulus, decreasing the burden of memory and time constraints during learning and decision-making. These processes have been posited to recruit trial-and-error feedback to optimize performance. Thus, overall, model-free processes can provide a faster, more parsimonious alternative to model-based processes, which may lead to more efficient learning and decisions depending on the circumstances. Adaptive control of behavior has been proposed to involve a fluid and contextually sensitive balance between 8 model-based and model-free learning processes (Daw et al., 2005). The existence of an arbitrator has been posited that determines whether the model-based or the model-free processes gain control of behavior when they disagree. Thus, this line of computational neuroscience research also posits a competition between parallel neurocognitive processes.
On the neural level, an extensive network of regions, including regions in the PFC and the hippocampus, has been suggested to be involved in model-based processes, while the dorsal striatum in the BG has been linked to model-free processes (Daw et al., 2005). Evidence from studies combining neuroimaging with computational modelling suggests a more complex picture (Doll, Simon, & Daw, 2012). For example, Lee et al. (2014) found that the PFC is associated with both model-based and model-free learning, with more medial/orbital PFC regions being associated with model-based, and more dorsal regions being associated with model-free learning. Activity in the striatum was uniquely associated with model-free learning.
Further, their findings suggested that other PFC regions (in particular, the inferior lateral PFC) implement the arbitration between model-based and model-free control. Remarkably, they found a negative correlation between activity in the inferior lateral PFC and the regions associated with model-free processes, suggesting that the arbitrator may work predominantly by acting on the model-free system, and suppressing its activity if the model-based system is deemed more efficient in a given situation.
Thus, based on extant theoretical and empirical work, model-based and model-free processes are posited to compete for control over behavior in adulthood (Daw et al., 2005;Lee et al., 2014). Reliance on model-based vs. model-free processes appears to be sensitive to the cognitive and affective demands placed on the individual (Otto, Gershman, et al., 2013;Otto, Raio, Chiang, Phelps, & Daw, 2013). For example, based on the hypothesis that model-based processes require more cognitive resources,  showed that having participants perform a demanding secondary task leads to increased reliance on model-free learning.
Moreover, the trade-off between the reliance on the two learning processes changed dynamically as a function of concurrent executive function demands.
Considering the cognitive and neurodevelopmental changes discussed above, the relative reliance on model-based vs. model-free learning might also change markedly with age.
Indeed, Decker et al. (2016) tested children, adolescents and adults on a learning task, and showed that whereas model-free processes were present in participants' choice behavior across all age groups, model-based processes were absent in children, emerged in adolescents, and strengthened in adults. These findings suggest that the model-free processes are already wellfunctioning in childhood and the recruitment of model-based processes emerges during development. This shows striking parallels with the marked structural and functional changes of the neurocircuitry involving the PFC and MTL from adolescence to adulthood (Blakemore, 2012;Shaw et al., 2008) and the concurrent strengthening of control functions such as cognitive control and executive functions around the same time (Casey, Tottenham, Liston, & Durston, 2005;Craik & Bialystok, 2006).

Competitive neurocognitive functions: an integrative account
Linking the evidence from cognitive, developmental and computational neuroscience, here we attempt to provide an integrative account of competitive neurocognitive functions.
Learning linked to the neurocircuitry that involves the BG, particularly the striatum, seems to encompass associative learning processes whereby we extract predictable regularities (e.g., repeating patterns) from the environment automatically, that is, without intention or conscious monitoring . This form of learning appears to be primarily external/stimulus-driven and occurs implicitly, without any awareness or cognitive effort (Janacsek et al., 2012). It may recruit model-free processes that utilize trial-and-error feedback and provide a faster, less resource-expensive form of learning by relying on more recent experience instead of searching internal models of past experiences. The capacity to detect predictable regularities is crucial for aspects of procedural memory, which are involved in the acquisition of cognitive, social and motor skills and habits (Kaufman et al., 2010;Lieberman, 2000;Nemeth & Janacsek, 2011;Romano Bergstrom, Howard, & Howard, 2012;Ullman, 2004). Since many skills are acquired during childhood, this form of learning is a vital aspect of child development.
By contrast, learning linked to the neurocircuitry that involves regions in the PFC and the MTL seems to represent a more controlled form of learning that relies more heavily on cognitive resources such as cognitive control and executive functions. This form of learning may occur incidentally or intentionally. It appears to involve the development of complex representations, internal models of the environment/task, whereby past experiences become more influential determining one's current behavior at the expense of the reliance on more recent experiences (Nemeth, Janacsek, & Fiser, 2013). While this form of learning may be beneficial in multiple situations, it is cognitive resource-expensive and therefore can be prone to errors. In terms of related functions, it appears to be linked to cognitive control, executive functions, and working memory, and may be involved in aspects of declarative as well as procedural memory. In turn, these functions support abilities such as reasoning, planning and goal-directed decision making (see Figure 1). Competition seems to exist between the neurocircuitry involving the BG vs. the PFC and MTL for gaining control of behavior, whereby the greater the reliance on the circuitry involving the BG, the smaller the reliance on the circuitry involving the PFC/MTL, and vice versa (Poldrack & Packard, 2003). Due to the different maturational trajectories of these circuitry, the reliance on the cognitive processes and functions they subserve also shows marked shifts during development. Children appear to outperform adults in tasks that require the extraction of predictable patterns of the environment typically occurring without mental effort such as in procedural learning (Janacsek et al., 2012;Juhasz et al., 2019;Nemeth, Janacsek, & Fiser, 2013). In contrast, other functions requiring more cognitive resources, such as cognitive control and executive functions demonstrate marked development through adolescence and peak in adulthood (Casey et al., 2005;Craik & Bialystok, 2006). These findings from cognitive and developmental neuroscience are consistent with the evidence from computational neuroscience on model-free and model-based learning, respectively.
Specifically, there seems to be a greater reliance on faster, more parsimonious model-free processes until the neurocircuitry subserving the development and utilization of internal models becomes more influential, starting around adolescence (Decker et al., 2016). the reliance on the latter circuitry that is linked to model-based processes is reduced in children (dotted black arrow), 'liberating' its competitive weight (dotted red arrow) on the other circuitry linked to model-free processes (continuous black arrow). Since both circuitry are fully matured in adulthood (continuous black arrows), they compete for gaining control over behavior (continuous red arrow). Regions in the PFC seem to arbitrate between the two circuitry by suppressing the circuitry involving the BG if the other circuitry deems more efficient in a given situation. C) Whereas children rely more on the circuitry involving the BG, the circuitry involving the PFC and the MTL becomes more influential from adolescence to adulthood. This relative reliance is associated with distinct cognitive processes and functions being more influential in different stages of development, which, in turn, support the development of different abilities.

Competitive neurocognitive functions in schools
As described above, the neurocircuitry subserving distinct cognitive processes/functions interact and compete with one another for the control of behavior. Moreover, they undergo different developmental trajectories, resulting in altered reliance on them during development.
Considering the competitive interactions between these circuitry and their effects on learning across development in the academic curriculum could help improve academic outcomes. For example, for pupils under the age of 12-13, it may be beneficial to lower the number of tasks that rely on the circuitry involving PFC and MTL regions, which undergo marked changes through adolescence to adulthood, and instead, to increase the number of tasks that rely more on the circuitry involving the BG, which is fully functional already in childhood.
Although learning situations in schools are typically complex and likely involve multiple circuitry, it may be possible to design tasks that rely more on one circuitry than on the other. Some academic practices already exist that seem to put a heavier weight on the circuitry involving the BG. For example, learning a foreign language through immersion is becoming increasingly popular (Cenoz & Gorter, 2015;Isabelli-García & Isabelli, 2020). For example, if someone wants to learn Spanish, spending 2-3 months in Spain can boost their knowledge of the Spanish language to a greater extent than staying in their home country and studying that foreign language for an equal amount of time. One possible explanation for this is that, by being immersed in the Spanish speaking environment, learning becomes more externally/stimulus-driven, and learners pick up knowledge without even fully realizing it, 12 likely relying more on the circuitry that involves the BG. After all, this is also how toddlers learn their first language.
So what are the main characteristics of learning that relies on the circuitry involving the BG vs. the PFC and MTL? Although there is still intensive research on these two forms of learning, there are some characteristics that might help us distinguish between them in the classroom. As outlined above, learning linked to the former circuitry appears to be more externally/stimulus-driven, incidental and less cognitive resource-expensive, while learning linked to the latter circuitry is more internally-driven, memory-based and resource-expensive (see also Figure 1). Considering these characteristics, together with the pupil's age may help design tasks for the classroom that could boost learning outcomes. It should also be noted that, even in a certain age group, individual differences may exist among pupils due to possible differences in the maturation of the underlying neurocircuitry. Thus, one size does not fit all, and individualized learning approaches should also be considered.

Conclusion
There has been a growing interest in incorporating psychological and neuroscientific knowledge about the development of cognitive functions in educational policies and academic practices. In this paper, we argue that the current knowledge about the interactions that occur between the neural circuitry underlying these neurocognitive functions should also be considered in order to develop practices that could be more suitable to pupils depending on their age. The implementation of this knowledge in the education system could potentially help improve academic outcomes and equip learners with a more flexible skill set that may help them adapt to environmental changes more efficiently even beyond school.