Submitted:
30 August 2026
Posted:
01 September 2026
You are already at the latest version
Abstract
Autonomous agents require not only deliberate reasoning for unfamiliar situations but also the ability to convert repeatedly useful reasoning into efficient automatic behavior. However, existing LLM-based agents and knowledge-distillation approaches generally couple reasoning and learning through predefined training objectives or direct teacher--student supervision, leaving unclear how an agent can autonomously determine what should become learned behavior. This paper proposes Thought-Mediated Learning (TML), a framework that separates LLM-based deliberative thinking from an independent ML/DL background learning process. Thinking cannot directly update the automatic model; instead, it produces thought materials whose content and frequency determine what becomes available for learning. Repeated thinking implicitly strengthens important experiences, attention determines which factors are represented, jointly considered events may create spurious associations, and imagined experiences can provide learning materials before corresponding real events occur. Observed outcomes further generate corrective materials that revise previously learned automatic behavior. Experimental results show that TML achieves 96.67% hidden-pattern acquisition and 99.93% post-learning automatic accuracy, reaches 97.83% OOD (out-of-distribution) accuracy versus 9.50% for raw learning, and improves zero-real rare-case success to 59.38% while corrective learning raises final hidden-case success to 96.82%.
Keywords:
Thought-Mediated Learning
; deliberative reasoning
; automatic behavior
; autonomous learning
1. Introduction
Humans do not rely on deliberate reasoning for every familiar situation. New, uncertain, or difficult situations often require explicit thinking, whereas frequently encountered and repeatedly practiced situations can gradually be handled with little conscious effort. This suggests that deliberative thinking and automatic learning are related but distinct processes: thinking interprets experience and produces meaningful internal representations, while learning gradually turns recurring regularities into automatic responses.
Recent large language models (LLMs) provide strong capabilities for planning, reflection, and reasoning in autonomous and embodied agents [1,2]. Meanwhile, knowledge distillation and continual learning can transfer or preserve useful behaviors in smaller models [3,4]. However, most existing approaches either use the LLM directly to determine actions or treat it as a teacher that explicitly supplies labels, trajectories, or training targets. They therefore provide limited modeling of a more indirect question: how does what an agent thinks about determine what an independent learning process eventually learns?
This question is important for long-term autonomous robots. A robot may observe a very large number of ordinary events, while only a small fraction are novel, risky, conflicting, or otherwise important enough to deserve deliberation. It is impractical to manually specify in advance every behavior that should eventually become automatic. Moreover, simply experiencing an event does not guarantee that all of its relevant factors are represented. An overlooked factor may never enter learning materials, while unrelated events repeatedly considered together may form an undesirable association. Likewise, imagined situations may provide useful experience before a real event occurs, but such imagined experience may be incomplete and require later correction.
To model this relationship, this paper proposes Thought-Mediated Learning (TML). TML separates deliberative thinking and automatic learning into two independent processes. An LLM performs deliberative reasoning and produces thought materials, while an ML/DL model independently learns automatic behavior from the accumulated materials in the background. Importantly, the LLM cannot directly modify the parameters of the automatic model or assign explicit learning weights. Its influence is indirect: attention determines which factors enter a material, repeated thinking increases the frequency of related materials, imagination generates materials beyond direct experience, and observed outcomes can trigger corrective materials that revise previously learned behavior.
This architecture enables a transition from deliberation to automaticity. Unfamiliar situations initially invoke LLM reasoning, but repeated and consistent thought materials can eventually allow the learned model to handle similar situations directly. At the same time, because learning targets emerge from the agent’s own thought process rather than being completely predefined, the framework supports autonomous and open-ended formation of automatic behaviors.
The main contributions of this work are summarized as follows:
- We introduce a dual-process architecture in which LLM-based thinking and ML/DL-based automatic learning are independent, and thinking influences learning only through generated thought materials.
- We model how attention, repeated thinking, imagination, and joint representation shape what can be learned, allowing the same framework to explain useful automaticity as well as missing or spurious learned associations.
- We introduce outcome-driven corrective material generation so that previously learned automatic behavior can be strengthened, weakened, or replaced without direct intervention in the learning process.
The remainder of this paper introduces the proposed TML model, discusses its relationship to existing deliberative, distillation, and continual-learning approaches, presents the experimental verification, and concludes with the main findings and limitations.
2. Related Work
2.1. Deliberative Reasoning for Embodied and Autonomous Agents
Large language models have increasingly been used as high-level reasoning components for embodied agents because they provide broad semantic knowledge and can decompose long-horizon instructions into interpretable intermediate decisions. SayCan combines an LLM with affordance/value estimates from pretrained robotic skills so that language-level plans are constrained by what the robot can physically execute [5]. Inner Monologue further incorporates success detection, scene descriptions, and other environmental feedback into an iterative language reasoning loop, allowing a robot to revise its plan after observing the effects of previous actions [2]. These approaches demonstrate an important benefit of LLM-based deliberation: complex and previously unseen situations can often be handled without retraining the underlying language model, while the reasoning trace remains explicit enough to incorporate semantic context and feedback.
More recent agent architectures extend this idea from single-episode planning toward longer-term self-improvement. Reflexion stores verbal reflections produced from task feedback and reuses them as episodic context in later trials rather than updating the LLM parameters [6]. Voyager similarly uses an LLM-driven automatic curriculum, iterative feedback, and a growing library of executable skills to support open-ended embodied learning in Minecraft [7]. Generative Agents also maintain streams of natural-language memories and synthesize them into higher-level reflections that influence subsequent planning [8]. These systems show that reflection and memory can improve future behavior and reduce repeated reasoning failures. However, their persistent improvement remains primarily within the LLM context, a textual memory, or a symbolic/program skill library. They do not explicitly model an independent background ML/DL process that gradually converts thought-produced experience into a separate automatic policy.
The distinction is important for the present work. Existing LLM-agent methods mainly ask how deliberation can directly produce better actions, plans, programs, or future LLM prompts. In contrast, Thought-Mediated Learning (TML) treats deliberative reasoning and automatic learning as two separate processes. The LLM cannot directly update the automatic model; it can only produce thought materials that later become available to an independent learner. Consequently, the proposed framework is concerned not only with whether reasoning improves an immediate action, but also with which parts of experience become learnable, how repeated thinking changes their frequency, how jointly represented factors may form spurious associations, and how sufficiently repeated thought can eventually reduce the need for LLM reasoning itself.
2.2. Knowledge Distillation and the Transition to Efficient Automatic Models
A second closely related line of work seeks to transfer expensive capabilities into smaller and faster models. Policy distillation showed that a compact student policy can be trained from one or more larger reinforcement-learning policies while preserving strong task performance [9]. In language modeling, Distilling Step-by-Step uses rationales generated by a large model as additional supervision, enabling substantially smaller models to learn from the teacher with reduced data requirements [3]. Symbolic chain-of-thought distillation likewise transfers sampled reasoning traces from a large teacher into much smaller students, showing that step-by-step supervision can improve compact models on reasoning benchmarks [10]. The principal advantage of this family is computational: once the student has absorbed useful behavior from the teacher, inference can be substantially cheaper than repeatedly invoking the large model.
Despite the superficial similarity to the transition from deliberation to automaticity considered here, conventional distillation has a different information pathway. The teacher normally provides labels, logits, rationales, trajectories, or other explicit training targets to the student under a predefined training objective. The teacher–student relation is therefore direct, and the dataset or set of target tasks is typically fixed before training. Distillation is very effective when the objective is to compress a known expert, but it does not by itself explain why some encountered experiences become training data while others do not, why repeated consideration of the same event should increase its eventual influence, or why an incorrect association formed during deliberation might also become automatic.
TML therefore should not be interpreted simply as LLM-to-small-model distillation. In the proposed architecture, the LLM does not assign student loss weights, directly optimize the policy, or necessarily generate a single authoritative target for every environmental observation. Instead, thought determines the content and frequency of an intermediate material stream, and the ML/DL learner independently extracts regularities from that stream. This separation allows the set of eventual learning targets to emerge during autonomous operation rather than being fully enumerated by a human in advance. It also makes the quality of automatic behavior explicitly dependent on thought attention, repetition, imagination, and association, which are outside the usual scope of teacher–student distillation.
2.3. Continual Replay, Selective Memory, and Imagined Experience
Continual learning provides another important perspective because an autonomous agent must accumulate new knowledge without discarding useful old behavior. Experience replay is a widely used solution: CLEAR mixes new and replayed experience and combines off-policy learning with behavioral cloning to reduce catastrophic forgetting in sequential reinforcement-learning tasks [11]. Rather than replaying memories uniformly, Maximal Interfered Retrieval selects examples expected to suffer the greatest loss increase from the current update, thereby allocating replay capacity toward samples that are most useful for preserving previous knowledge [12]. These methods establish two ideas that are also relevant to TML: repeated exposure to stored experience can substantially influence later learning, and limited replay capacity can benefit from selective allocation rather than treating all past samples equally.
However, replay methods generally begin after an experience has already been encoded as a learner-accessible sample. Their main question is which stored samples should be replayed to optimize stability, plasticity, or gradient interference. TML places an additional process before this stage: an environmental event is not automatically equivalent to a thought material. If a factor is not represented during deliberation, the background learner may never receive it; conversely, factors repeatedly represented together may become spuriously associated. Moreover, TML’s selection criterion can be semantic and agent-centered—for example, repeated thought may concentrate on events judged risky, novel, or consequential—rather than being determined only by gradient interference or forgetting. Thus, selective replay and TML are complementary: replay controls reuse of already available samples, whereas TML models how the learnable sample stream itself is shaped by thought.
Related work has also explored learning from internally generated experience. DreamerV3 learns a world model and improves an actor and critic by imagining future latent trajectories, demonstrating that imagined experience can greatly improve data-efficient control across diverse domains [13]. LLM-based reflective systems provide another form of internal experience: Reflexion can derive verbal feedback from externally observed or internally simulated signals [6], while Generative Agents synthesize higher-level reflections from stored events [8]. These methods show the benefit of learning or acting beyond immediate raw observations. Their limitation with respect to the present problem is that imagined trajectories or reflections usually remain inside the world-model optimization loop or the LLM context. TML instead treats imagined thought as one possible source of training material for an independent automatic learner and explicitly allows such material to be incomplete or wrong. Later real outcomes can generate corrective thought materials that weaken or replace the previously learned automatic response. This provides a unified mechanism for useful imagination, erroneous automaticity, and subsequent behavior revision.
3. Thought-Mediated Learning Model
We consider an autonomous agent in which deliberative thinking and automatic learning are implemented as two independent processes. Deliberative reasoning is performed by a large language model (LLM), whereas automatic behavior is learned by a machine-learning or deep-learning model. The central assumption is that the thinking process does not directly update the parameters of the learning model. Instead, thinking produces intermediate materials that are stored and subsequently consumed by an independent background learning process. Consequently, what is eventually learned depends not only on what happens in the environment, but also on what is noticed, interpreted, reconsidered, associated, or imagined during thinking.
3.1. Independent Thinking and Automatic Processing
At time t, the agent receives an observation and contextual information . The automatic model first attempts to handle the current situation,
where denotes the learned automatic model with parameters , and is its predicted action or response.
The automatic model is intended to handle familiar situations without deliberative reasoning. When the current input is unfamiliar, uncertain, conflicting, or otherwise requires additional reasoning, the thinking process is activated. The LLM then produces a thought representation
where denotes the deliberative thinking process and represents relevant historical context.
Importantly, is not a learning command and does not directly modify . Instead, the thinking process converts its interpretation into a learnable thought material
where is the material-construction process and denotes an observed, inferred, or subsequently obtained outcome when such an outcome is available. A material may contain the recognized situation, attended factors, reasoning result, selected action, and observed consequence.
The accumulated material memory is therefore
where is the number of thought materials available at time t. Environmental events that are not noticed or represented by the thinking process do not necessarily enter and therefore may never become training examples.
This separation establishes the key information pathway of the proposed model:
There is deliberately no direct parameter-update path.
3.2. Thought-Mediated Formation of Learning Materials
The central characteristic of the proposed framework is that the learning process does not directly learn from everything that occurs in the environment. Instead, it learns from materials formed through deliberative thinking. Consequently, the learning history of the agent is not identical to its physical experience history. Two agents may encounter the same events but eventually learn different automatic behaviors if their thinking processes notice, interpret, associate, or reconsider those events differently.
A thought material can originate from an external observation, a previously stored experience, information obtained from reading or communication, or an internally imagined situation. However, merely receiving such information is not sufficient for learning in the proposed model. The information must first be represented within a thinking episode and converted into a learnable material. This mechanism allows thinking to mediate learning without directly controlling the learning algorithm.
3.2.1. From Experience to Learnable Material
Consider an environmental event containing a complete set of potentially relevant factors
The agent may physically observe many of these factors, but the thinking process does not necessarily represent all of them. Let
where is a thought-dependent attention mask and ⊙ denotes element-wise multiplication. A value indicates that factor is represented in the current thought, whereas indicates that the factor is not incorporated into the thought material.
The resulting material therefore reflects the interpreted experience rather than the complete physical experience. For example, suppose the success of grasping an object actually depends on its visual appearance, weight, surface condition, and grasping force. If the thinking process only notices appearance and weight, the corresponding material may encode
while surface condition and grasping force are absent. A subsequent learning model cannot recover the omitted relationship merely by using a more powerful training algorithm because the missing factors were never supplied as training information.
This distinction also applies to apparently passive information acquisition. For example, reading a document does not automatically make every sentence a training sample. Only information that is interpreted, connected to the current problem, recalled, compared, questioned, or otherwise processed by thinking contributes to the thought-material memory. Thus, in the proposed model,
whereas
This property separates environmental exposure from actual learning opportunity.
3.2.2. Repeated Thinking as Implicit Learning Emphasis
The thinking process is intentionally prevented from directly assigning a training importance value to a material. In particular, the framework does not allow the LLM to issue an instruction such as “learn this example with high weight” or to directly change the parameters of the automatic model.
Instead, importance can influence learning through repeated thinking. If an event is considered once, it may produce one material,
If the same event or concept is repeatedly recalled, reconsidered, practiced, or connected to new situations, additional related materials can be produced,
where is the number of thought occurrences associated with the underlying issue.
The effective frequency of this issue in the material memory can be expressed as
where is the relative frequency of materials related to issue i. An issue that is repeatedly reconsidered therefore naturally occupies a larger proportion of the training data and can exert a greater influence on the learned automatic model.
This provides an indirect mechanism for representing importance:
Importantly, the learning process itself does not need to know why a certain material appears repeatedly. It simply learns from the statistical distribution of the available materials.
The same mechanism can explain practice and repeated memory. For example, if a robot repeatedly reasons that a fragile object should be grasped with a smaller force, similar thought materials may gradually accumulate. After sufficient training, the association between the fragile-object context and the low-force action can be handled directly by the automatic model without invoking the LLM.
Conversely, an event that is encountered only once and is never reconsidered may have little influence on the resulting automatic behavior even if the event was objectively important. Hence, objective importance and learned importance are not necessarily identical; learning reflects the materials actually produced by thought.
3.2.3. Real and Imagined Thought Materials
Thinking is not restricted to events that are currently occurring. A deliberative process can recall previous situations, consider counterfactual conditions, predict possible consequences, or imagine situations that have never been physically experienced. We therefore distinguish two broad material sources,
A real material is grounded in an actually observed event or outcome. For example,
may be generated after an actual manipulation failure.
In contrast, an imagined material may be produced by internally considering a possible situation,
From the viewpoint of the independent background learner, both can become training materials. This allows deliberative reasoning to expand learning beyond situations that have already been directly experienced. For rare or dangerous conditions, such internally generated materials may be especially useful because collecting many real failures can be undesirable or impractical.
However, imagined materials have an important limitation. The thinking process can only include factors that it knows or considers during imagination. Suppose an actual outcome depends on
while the imagined scenario contains only
The missing factor may cause the imagined relationship to differ from the real environmental relationship. Repeated learning from such incomplete imagined materials can therefore produce an automatic behavior that appears reasonable under the imagined model but fails in the real environment.
Thus, imagination has two opposing effects. It can increase the diversity and coverage of learning materials, but it can also introduce inaccurate relationships when relevant physical or contextual factors are omitted. The framework does not assume that imagined materials are inherently correct. Their validity can later be challenged by real outcomes, which may trigger new thought materials and subsequent behavior revision as described in Section 3.3.
3.2.4. Thought Association and Incomplete Representation
Another important consequence of thought-mediated learning is that events represented together may become learnable together, even when their co-occurrence is accidental rather than causal.
Suppose two events and are objectively unrelated. If they occur close in time and are repeatedly considered within the same thinking episode, the generated material may contain a joint representation such as
The background learning process does not have direct access to the complete causal structure of the world. It only receives the generated materials. Repeated co-occurrence of and in these materials can therefore lead the automatic model to learn an association between them.
For example, suppose a person repeatedly washes their hands immediately before performing another routine activity. If the two events are repeatedly represented as a single thought sequence, a learner operating only on these materials may eventually treat the first activity as a predictor or component of the second, although no necessary causal relationship exists.
More generally,
but
We refer to this effect as thought-induced association. It provides a mechanism through which automatic behavior can inherit not only useful regularities from deliberative reasoning but also its incomplete or erroneous associations.
The opposite problem can also occur. A single real event may contain several important features, but only a subset may be represented in thought. In this case, the learning process receives an incomplete relation. Therefore, thought-mediated learning can introduce two complementary forms of representation bias: missing association, in which relevant factors are omitted, and spurious association, in which unrelated factors are repeatedly represented together.
These effects are particularly important because they distinguish the proposed framework from conventional training-data generation. Thought materials are not treated simply as synthetic samples produced by an LLM. Rather, their formation process is itself part of the model. Attention determines which factors enter a material, repeated thinking changes material frequency, imagination creates experiences beyond direct observation, and joint thinking determines which events may become associated. The independent learning process subsequently extracts regularities from this thought-shaped material distribution.
Consequently, the proposed framework can be summarized by
This relationship is the principal mechanism through which deliberative reasoning mediates automatic learning in the proposed model.
3.3. Background Learning and Behavior Revision
The learning process operates independently in the background and periodically updates the automatic model using the accumulated thought materials. A generic update can be written as
where is the set of materials used in the k-th background training round, is the task-dependent training loss, and denotes the updated automatic-model parameters. The formulation is intentionally model-agnostic: depending on the task, can be implemented using a neural network, classifier, sequence model, or another trainable ML/DL model.
A behavior that initially requires deliberative reasoning may therefore gradually become automatic as sufficiently consistent thought materials accumulate. For a familiar state, the resulting model can directly produce an appropriate response through Eq. (1), avoiding a new LLM reasoning process.
Previously learned automatic behavior is not assumed to be permanent. Suppose an action that was previously considered appropriate later produces an undesirable outcome. The outcome may trigger additional reasoning and produce corrective materials,
These materials are added to and participate in subsequent training rounds. The automatic model is therefore modified indirectly through
This mechanism permits an automatic response to be strengthened, weakened, or replaced while preserving the independence between deliberative thinking and background learning.
Overall, the proposed model treats automatic behavior as a compressed consequence of accumulated thought-mediated experience. Thinking determines which experiences are represented and how often they are represented, whereas the learning process independently extracts regularities from those representations. This separation allows familiar situations to be handled efficiently by the learned model while unfamiliar, uncertain, or problematic situations continue to invoke deliberative reasoning.
4. Verification
We evaluate whether the proposed Thought-Mediated Learning (TML) framework exhibits the mechanisms predicted by the model rather than only improving a single task accuracy. The experiments therefore test four questions: (i) whether an autonomous agent can convert repeatedly deliberated situations into automatic behavior without a manually specified learning target, (ii) whether the content of thought materials determines what is subsequently learned, (iii) whether imagined thought materials can support learning before direct experience and whether later outcomes can revise an inaccurate automatic behavior, and (iv) whether a fixed background-thinking budget can be preferentially allocated to memories judged more important by the thinking process. Unless otherwise stated, values are reported as mean ± standard deviation over independent random seeds. For paired comparisons we use a two-sided Wilcoxon signed-rank test.
4.1. Experimental Environment and Protocol
- Robot-oriented scenarios.
The experiments use a controlled robot-oriented simulation rather than a single static classification benchmark. This choice is important because the proposed model is intended for long-running autonomous agents in which most observations are routine while a smaller number of novel, risky, conflicting, or outcome-relevant situations deserve deliberation. A simple environment in which every sample is equally informative would largely remove the need for a thought-mediated material formation process.
Experiment 1 simulates a mobile manipulation robot that continuously performs three types of activities: object handling, cabinet operation, and delivery. The robot first receives 120 ordinary familiar examples and then executes a 240-episode stream. Three previously unspecified patterns are introduced only after the stream has started: a wet glass requiring a safer grasp after episode 55, a previously unseen sticky cabinet requiring slower opening after episode 100, and a low-battery long-distance delivery requiring charging before execution after episode 135. These patterns are not supplied to TML as learning targets. The setting therefore tests whether the robot’s own reasoning can form materials about newly encountered issues and whether the background learner can later handle them automatically.
Experiment 1b isolates the effect of repeated thinking using a tightly controlled safety scenario. The robot has 60 ordinary container-handling examples and encounters exactly one external critical event: a high-temperature container with substantial seal damage. The number of separate thought occurrences generated from this same event is varied as . Each occurrence produces another material, but the learning model receives no importance weight or loss multiplier. Evaluation uses 220 critical variants and 220 hard ordinary counterexamples. This experiment is intentionally simple because its purpose is causal isolation of repetition rather than task complexity.
Experiment 1c returns to the multi-task environment and tests allocation of a fixed background-thinking budget. All primary semantic memories are first formed from experience. The LLM then performs three independent ranking passes over the candidate memories. Both Focused Rehearsal and Random Rehearsal receive exactly 30 background reconsideration calls, the same number of resulting training materials, and complete coverage of the primary memory set. Focused Rehearsal first revisits every memory once and then allocates the remaining budget to the top-ranked 25% of memories; Random Rehearsal distributes the same budget across all memories without using the ranking. The hidden “true important” flag is used only for evaluation and is never provided to the LLM or learner.
Experiment 2 constructs a manipulation setting in which the true action rule depends on object fragility, weight, and wetness. Two nuisance context variables, room-light state and background-music state, are made strongly correlated with the action in the training and in-distribution (IID) data ( and agreement, respectively), but these correlations are reversed in the out-of-distribution (OOD) test while the underlying action rule is unchanged. Each seed contains 420 raw training examples, 300 IID test examples, and 300 OOD examples. TML forms 220 thought materials. Feature relevance is determined using five LLM reflection rounds, each observing a balanced subset of up to 48 examples; a factor is retained when its vote rate is at least 0.60. The LLM determines only which observed variables are represented in the material. The action label attached to a material is the observed required action, not an LLM-generated label. This design isolates the influence of thought-mediated representation from label-generation errors.
Experiment 3 evaluates imagination and outcome-driven revision in a rare-risk manipulation task. The automatic learner first receives 360 ordinary real samples containing no heavy-load experience and no rare high-temperature combination. TML then generates approximately 60 imagined hot, fragile glass situations. During imagination, the exact load is deliberately unavailable; therefore imagined materials can help with rare light cases but cannot encode a hidden heavy-load dependency. Evaluation uses 240 rare cases, approximately 30% of which contain the hidden heavy-load condition. Up to 20 subsequent real hidden events can expose a failure; when a predicted action fails and the recovery action succeeds, the LLM forms a corrective material. The automatic model is retrained after every two corrective materials. This scenario tests both the benefit and the limitation of imagination, followed by behavioral revision from real outcomes.
- Thinking, learning, and model configuration.
Deliberative thinking is implemented with deepseek-chat through an OpenAI-compatible chat interface with temperature 0.2. The LLM is allowed to interpret events, select represented factors, imagine situations, rank memories for reconsideration, and form corrective materials, but it never receives gradients and never calls the optimizer or directly modifies the automatic model. Automatic behavior is implemented consistently with a PyTorch multilayer perceptron having two hidden layers of 64 and 32 units, ReLU activations, and dropout 0.10. Training uses Adam with learning rate , weight decay , batch size 32, cross-entropy loss, and 70 epochs for a full fit. Experiment 1 additionally uses a target-agnostic k-nearest-neighbor familiarity detector with Euclidean radius 0.45 and a minimum of 10 nearby memories; an automatic decision is accepted only when the neural-network confidence is at least 0.78. The familiarity detector decides only whether a case is familiar and does not know any hidden task rule.
The core Exp. 1 comparisons (TML, TML-No-Repetition, and Predefined Target) and the core Exp. 3 comparisons were run over 10 seeds. Raw Experience and Always-LLM in Exp. 1, and Always-LLM in Exp. 3, were retained as five-seed reference baselines because they are respectively an optimistic annotation-dependent baseline and a non-automatic inference baseline. Experiment 1b uses five seeds, while Exp. 1c and Exp. 2 use 10 seeds.
- Comparison methods and terminology.
TML is the complete framework. TML-No-Repetition disables independent background reconsideration after a primary thought material has been formed. Predefined Target uses a human-fixed feature scope specified before the later hidden patterns appear. Raw Experience is an optimistic baseline in which every real event is directly supplied to the learner with its correct post-event action label; thus it should not be interpreted as an equally autonomous method. Always-LLM invokes the LLM for every decision and never forms an automatic model.
For Experiment 2, Raw MLP learns from all observed variables, MI Feature Selection retains the three variables with the largest mutual information on the training set, and Oracle Causal is an upper reference supplied with the three true relevant factors. The Partial-Attention ablation removes wetness from the thought material even though the robot observes it, whereas Joint-Association forces the two nuisance context variables to be represented together with the true factors. For Experiment 3, No-Imagination removes imagined materials and No-Correction prevents outcome-driven corrective materials. In Experiment 1c, Focused Rehearsal and Random Rehearsal use the same LLM and training budgets but differ in where additional thought occurrences are allocated.
- Metrics.
For Experiment 1, hidden-pattern acquisition is the fraction of the three new patterns whose final automatic-model accuracy reaches at least . Post-learning hidden MLP accuracy evaluates the learned neural network alone, without an LLM decision. Thought focus enrichment is the ratio between the fraction of truly important events among thoughts marked important and their prevalence in the underlying event stream. The LLM attention ratio similarly compares the probability of invoking deliberation on important versus ordinary events. Experiment 1b reports critical success and ordinary false-positive rate as K changes. Experiment 1c reports the fraction of rehearsal devoted to truly important memories, its enrichment over the primary-memory prevalence, and final hidden-pattern accuracy.
For Experiment 2, OOD means that the nuisance correlations are reversed while the causal action rule is held fixed. In addition to IID and OOD accuracy, we measure the spurious association rate (SAR), the fraction of counterfactual pairs for which the predicted action changes when only the two nuisance variables are flipped. Relevant-factor coverage and irrelevant-factor inclusion directly measure the content of the selected representation. For Experiment 3, zero-real rare success is measured before any real hidden correction event, while final hidden success is measured after the correction phase. Old-behavior probability tracks the probability of the previously imagined gentle_slow response on heavy hidden cases.
4.2. Experiment 1: Autonomous Material Formation and Automaticity
Table 1 shows the continuous-task results. TML achieved task success and acquired of the three previously unspecified hidden patterns. More importantly, after training, the MLP alone classified the hidden-pattern validation banks with accuracy. Removing background repetition reduced hidden acquisition from to and post-learning MLP accuracy from to ( for acquisition), even though TML-No-Repetition could still use the LLM online and therefore retained a relatively high overall task success of . This separation between online task success and learned automatic competence is important: it shows that successful deliberation alone is not equivalent to automatic learning.
The results also support autonomous focus on potentially important experience. Only of the event stream consists of the three hidden important patterns, whereas of thoughts judged important by TML are truly important, yielding a thought-focus enrichment of . TML invoked deliberation on important events with a rate of , compared with for ordinary events, an attention ratio of . Thus, the robot is not given a list of what must become a skill; its own deliberative process preferentially exposes some experiences to the background learner.
The Raw Experience baseline reaches hidden-pattern acquisition because it is deliberately optimistic: all 240 real events, including their correct post-event labels, are made available to the learner. In contrast, TML must first create thought materials and does not receive this oracle stream. Predefined Target reaches exactly two of the three hidden patterns (), illustrating the limitation of a fixed human-defined scope when a new relation such as the unseen cabinet identity or trip-length interaction appears after deployment. Always-LLM avoids automatic learning entirely and requires deliberation for every decision.
4.2.1. Controlled Repetition Dose
Figure 1 isolates the effect of repeated thought while holding the number of external critical experiences fixed at one. With , 1, or 3 thought occurrences, critical success remains . At it increases to , then rises sharply to at and at . This increase is not obtained by predicting the special action everywhere: the ordinary false-positive rate is only at and at . Because every additional material is generated by a separate LLM thought occurrence and no explicit learner weight is changed, the dose-response provides direct evidence for the proposed frequency-mediated pathway from repeated thinking to automaticity.
4.2.2. Fixed-Budget Rehearsal Allocation
Experiment 1c asks a stricter question: under the same thinking and training budget, can deliberative reasoning allocate additional rehearsal toward more important memories? To ensure a controlled comparison, Focused and Random Rehearsal both cover of the primary memories, use exactly 30 background reconsiderations, and contain the same average number of training materials (). Thus, the two methods differ only in how the additional rehearsal opportunities are allocated.
Table 2 reports the resulting rehearsal concentration and automatic-learning performance. Focused Rehearsal allocates of its reconsiderations to experiment-defined important memories, compared with for Random Rehearsal. A two-sided paired Wilcoxon test confirms that this difference is statistically significant (). Relative to the importance prevalence in the original primary-memory pool, the corresponding rehearsal enrichment is for Focused Rehearsal and for Random Rehearsal. These results show that the deliberative process can autonomously concentrate a fixed rehearsal budget on memories that are more important to subsequent robot behavior, rather than distributing the same budget approximately uniformly.
Focused Rehearsal also yields a higher mean hidden-pattern accuracy, increasing it from under Random Rehearsal to . Hidden-pattern acquisition similarly increases from to . However, the paired difference in mean hidden accuracy does not reach statistical significance over the ten seeds (). We therefore interpret this result as a positive downstream trend rather than as a statistically established performance advantage.
In contrast, both rehearsal strategies substantially outperform No Rehearsal, whose mean hidden-pattern accuracy is only and whose hidden acquisition is . This large gap is consistent with the controlled repetition-dose study in Experiment 1b, where increasing the number of repeated thought materials progressively strengthened automatic behavior. Taken together, Experiments 1b and 1c indicate that repeated thought is important for forming automatic responses, while deliberative ranking can further determine where a finite rehearsal budget is concentrated. The latter effect is statistically clear for rehearsal allocation, whereas its advantage in final hidden-pattern accuracy should be regarded as suggestive rather than conclusive.
4.3. Experiment 2: Thought Representation, Missing Factors, and Spurious Association
Experiment 2 directly tests whether the representation formed by thought changes what the independent MLP learns. As shown in Table 3, Raw MLP obtains the best IID accuracy () because the nuisance variables are highly predictive in the training distribution. Once those correlations are reversed, however, its OOD accuracy collapses to and its SAR reaches . MI Feature Selection behaves similarly: it achieves IID accuracy but only OOD accuracy because it selects statistically predictive nuisance variables.
TML retains all three task-relevant factors and none of the nuisance factors in all ten seeds, giving OOD accuracy and zero measured counterfactual spurious association. Its OOD accuracy is only 0.77 percentage points below the Oracle Causal reference (), while exceeding Raw MLP by 88.33 points and MI Feature Selection by 92.40 points; both paired comparisons give . Figure 2 visualizes this contrast. The fact that Raw MLP and MI Feature Selection are very strong on IID data but fail after correlation reversal shows why a complex non-stationary scenario is necessary: a simple IID benchmark would reward shortcut learning rather than the intended thought-mediated representation.
The two ablations further isolate the proposed mechanism. When wetness is physically observed but deliberately omitted from the thought material, Partial Attention drops from to OOD accuracy (). Thus, a sufficiently powerful learner cannot recover a factor that never enters its thought-mediated training representation. Conversely, Joint Association retains the correct three factors but repeatedly binds the room light and background music into the same material. It maintains IID accuracy, yet OOD accuracy falls to and SAR rises to . This result supports the model prediction that repeatedly co-represented but non-causal events can be learned as a spurious automatic association.
4.4. Experiment 3: Imagination and Outcome-Driven Revision
Table 4 evaluates whether thought can extend learning beyond direct experience and subsequently revise an inaccurate automatic behavior. Before any real hidden correction event, TML reaches success over rare cases, compared with only without imagination (). On rare light cases, where the hidden load factor is not required, the benefit is larger: versus . These results show that imagined thought materials can provide useful automatic competence before corresponding rare situations are directly experienced.
The experiment also deliberately exposes the limitation of imagination. As shown in Table 4, TML achieves zero-real success on rare light cases, but on the hidden heavy-load subset. The imagined materials do not contain the exact load, so the automatic learner cannot infer the load-dependent action before observing real heavy-load outcomes. This result illustrates that imagined experience can extend learning beyond direct experience, but cannot recover physical factors absent from the thought materials.
Once real heavy-load failures reveal the missing relation, corrective thought materials enable the automatic model to revise its behavior. Final hidden success increases from before correction to , whereas No Correction remains at (). At the same time, the probability of the old imagined behavior on heavy cases decreases from without correction to with TML. Thus, the observed outcome does not directly alter the learned model; rather, it triggers new thought materials that subsequently participate in background retraining.
Figure 3 shows this revision process in more detail. Hidden-case success rises from before correction to approximately after four corrective materials, after six, and after ten. Over the same trajectory, the probability of the old behavior falls from to approximately . Across seeds, TML requires on average corrective materials to first reach at least hidden-case success. These results support the proposed indirect revision path:
, where the automatic model is revised through newly generated thought materials and subsequent retraining rather than through a direct LLM-to-parameter update.
4.5. Overall Findings
The experiments support the main mechanisms of the proposed model rather than a single monolithic performance claim. First, the continuous-task experiment shows that previously unspecified situations can move from LLM-mediated handling to a learned automatic model, and the controlled dose experiment shows that this transition strengthens as independent thought occurrences accumulate. Second, Experiment 2 demonstrates that automatic learning is constrained by what thought represents: omitting a relevant factor produces incomplete learning, whereas jointly representing nuisance events produces a strong spurious association that is hidden by IID evaluation but exposed under OOD correlation reversal. Third, imagined materials improve performance before rare direct experience, while the deliberately missing load factor shows why imagination can also be incomplete; real outcomes then generate corrective materials that suppress the old behavior and restore hidden-case success. Finally, under an equal rehearsal budget, thought-based ranking significantly concentrates repetitions on important memories, although its additional accuracy benefit over random allocation is a trend rather than a statistically significant result in the present ten-seed study. Together, these findings are consistent with the central TML hypothesis that deliberative thinking shapes what enters the learning stream, while an independent learner converts the resulting material distribution into automatic behavior.
5. Conclusion
This paper presented Thought-Mediated Learning (TML), a framework that separates LLM-based deliberative reasoning from an independent ML/DL-based automatic learning process. Rather than allowing the LLM to directly modify the learner, TML uses thought-generated materials as the only interaction channel between the two processes. This design enables attention, repeated thinking, imagination, and observed outcomes to determine what becomes learnable and how previously learned behavior can later be revised.
Experimental results show that repeated thought can gradually transform deliberative reasoning into automatic behavior, while thought-mediated factor selection greatly improves robustness under correlation shifts. The experiments also demonstrate that imagined materials can support learning before corresponding real events occur and that corrective thought materials can effectively revise inaccurate automatic responses. These results suggest that autonomous learning can emerge from what an agent repeatedly considers important, rather than requiring all learning targets to be predefined manually.
Future work will investigate richer physical environments, longer-term autonomous operation, and more adaptive mechanisms for balancing broad experience coverage with focused repeated thinking.
References
- Chowa, S.S.; Alvi, R.; Rahman, S.S.; Rahman, M.A.; Raiaan, M.A.K.; Islam, M.R.; Hussain, M.; Azam, S. From language to action: a review of large language models as autonomous agents and tool users. Artif. Intell. Rev. 2026, 59, 71. [Google Scholar] [CrossRef]
- Huang, W.; Xia, F.; Xiao, T.; Chan, H.; Liang, J.; Florence, P.; Zeng, A.; Tompson, J.; Mordatch, I.; Chebotar, Y.; et al. Inner monologue: Embodied reasoning through planning with language models. arXiv 2022, arXiv:2207.05608. [Google Scholar]
- Hsieh, C.Y.; Li, C.L.; Yeh, C.K.; Nakhost, H.; Fujii, Y.; Ratner, A.; Krishna, R.; Lee, C.Y.; Pfister, T. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Proceedings of the Findings of the association for computational linguistics: ACL 2023, 2023; pp. 8003–8017. [Google Scholar]
- Fang, L.; Yu, X.; Cai, J.; Chen, Y.; Wu, S.; Liu, Z.; Yang, Z.; Lu, H.; Gong, X.; Liu, Y.; et al. Knowledge distillation and dataset distillation of large language models: Emerging trends, challenges, and future directions. Artif. Intell. Rev. 2025, 59, 17. [Google Scholar] [CrossRef] [PubMed]
- Brohan, A.; Chebotar, Y.; Finn, C.; Hausman, K.; Herzog, A.; Ho, D.; Ibarz, J.; Irpan, A.; Jang, E.; Julian, R.; et al. Do as i can, not as i say: Grounding language in robotic affordances. In Proceedings of the Conference on robot learning. Pmlr, 2023; pp. 287–318. [Google Scholar]
- Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; Yao, S. Reflexion: Language agents with verbal reinforcement learning. Adv. Neural Inf. Process. Syst. 2023, 36, 8634–8652. [Google Scholar] [CrossRef]
- Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; Anandkumar, A. Voyager: An open-ended embodied agent with large language models. arXiv 2023, arXiv:2305.16291. [Google Scholar]
- Park, J.S.; O’Brien, J.; Cai, C.J.; Morris, M.R.; Liang, P.; Bernstein, M.S. Generative agents: Interactive simulacra of human behavior. In Proceedings of the Proceedings of the 36th annual acm symposium on user interface software and technology, 2023; pp. 1–22. [Google Scholar]
- Rusu, A.A.; Colmenarejo, S.G.; Gulcehre, C.; Desjardins, G.; Kirkpatrick, J.; Pascanu, R.; Mnih, V.; Kavukcuoglu, K.; Hadsell, R. Policy distillation. arXiv 2015, arXiv:1511.06295. [Google Scholar]
- Li, L.H.; Hessel, J.; Yu, Y.; Ren, X.; Chang, K.W.; Choi, Y. Symbolic chain-of-thought distillation: Small models can also “think” step-by-step. In Proceedings of the Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023; pp. 2665–2679. [Google Scholar]
- Rolnick, D.; Ahuja, A.; Schwarz, J.; Lillicrap, T.; Wayne, G. Experience replay for continual learning. Adv. Neural Inf. Process. Syst. 2019, 32. [Google Scholar]
- Aljundi, R.; Belilovsky, E.; Tuytelaars, T.; Charlin, L.; Caccia, M.; Lin, M.; Page-Caccia, L. Online continual learning with maximal interfered retrieval. Adv. Neural Inf. Process. Syst. 2019, 32. [Google Scholar]
- Hafner, D.; Pasukonis, J.; Ba, J.; Lillicrap, T. Mastering diverse control tasks through world models. Nature 2025, 640, 647–653. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Experiment 1b: automatic critical behavior as a function of the number of thought occurrences generated from one external critical event. Error bars show standard deviation over five seeds.
Figure 1.
Experiment 1b: automatic critical behavior as a function of the number of thought occurrences generated from one external critical event. Error bars show standard deviation over five seeds.

Figure 2.
Experiment 2: OOD accuracy when the two strong nuisance correlations are reversed while the underlying manipulation rule remains unchanged. Error bars show standard deviation over ten seeds.
Figure 2.
Experiment 2: OOD accuracy when the two strong nuisance correlations are reversed while the underlying manipulation rule remains unchanged. Error bars show standard deviation over ten seeds.

Figure 3.
Experiment 3: outcome-driven revision of hidden heavy-load behavior. The success curve increases while the probability of the old imagined behavior decreases as corrective thought materials accumulate. For seeds requiring fewer corrections, the final learned state is carried forward for visualization. Error bars show standard deviation over ten seeds.
Figure 3.
Experiment 3: outcome-driven revision of hidden heavy-load behavior. The success curve increases while the probability of the old imagined behavior decreases as corrective thought materials accumulate. For seeds requiring fewer corrections, the final learned state is carried forward for visualization. Error bars show standard deviation over ten seeds.

Table 1.
Experiment 1: autonomous discovery and conversion to automatic behavior. Values are mean ± standard deviation. Raw Experience and Always-LLM use five seeds; the other methods use ten.
Table 1.
Experiment 1: autonomous discovery and conversion to automatic behavior. Values are mean ± standard deviation. Raw Experience and Always-LLM use five seeds; the other methods use ten.
| Method | Seeds | Task success (%) | Hidden acquisition (%) | Hidden MLP accuracy (%) | Decision LLM rate (%) |
|---|---|---|---|---|---|
| TML | 10 | ||||
| TML-No-Repetition | 10 | ||||
| Predefined Target | 10 | ||||
| Raw Experience† | 5 | ||||
| Always-LLM | 5 | – | – |
†Optimistic annotation-dependent baseline: every real event is supplied with its correct post-event action label.
Table 2.
Experiment 1c: equal-budget autonomous rehearsal allocation over ten seeds.
| Method | Important rehearsal (%) | Enrichment | Mean hidden accuracy (%) | Hidden acquisition (%) |
|---|---|---|---|---|
| Focused | ||||
| Random | ||||
| No Rehearsal | – | – |
Table 3.
Experiment 2: representation under correlation reversal, averaged over ten seeds. SAR denotes spurious association rate.
Table 3.
Experiment 2: representation under correlation reversal, averaged over ten seeds. SAR denotes spurious association rate.
| Method | IID acc. (%) | OOD acc. (%) | SAR (%) | Relevant coverage (%) | Irrelevant inclusion (%) | Training materials |
|---|---|---|---|---|---|---|
| TML | 220 | |||||
| Oracle Causal | 420 | |||||
| Raw MLP | 420 | |||||
| MI Feature Selection | 420 | |||||
| Partial Attention | 220 | |||||
| Joint Association | 220 |
Table 4.
Experiment 3: imagination and outcome-driven revision. Core TML variants use ten seeds; Always-LLM uses five.
Table 4.
Experiment 3: imagination and outcome-driven revision. Core TML variants use ten seeds; Always-LLM uses five.
| Method | Seeds | Zero-real rare (%) | Zero-real light (%) | Zero-real hidden (%) | Final hidden (%) | Final rare (%) | Old-behavior prob. (%) |
|---|---|---|---|---|---|---|---|
| TML | 10 | ||||||
| No Imagination | 10 | ||||||
| No Correction | 10 | ||||||
| Always-LLM | 5 | – | – |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.