Thought-Guided Training Material Acquisition for Autonomous Robot Learning

1. Introduction

Autonomous robots are expected to operate for long periods in open and changing environments rather than repeatedly execute only tasks that were completely specified during offline training. During continued operation, a robot may encounter unfamiliar objects, altered environmental conditions, previously unseen failures, or situations in which its existing behavior policy is unreliable. To handle such changes, the robot must not only select actions but also recognize what it does not know, determine what capability should be improved, and obtain appropriate experience for learning.

Recent advances in large language models, planning, embodied agents, and robot learning have improved the ability of robots to reason about tasks and adapt their behavior. A flexible reasoning model can interpret unfamiliar situations, decompose goals, analyze failures, and propose possible solutions. However, reasoning alone is generally too slow and computationally expensive for repeatedly executing familiar low-level behaviors. In contrast, a learned behavior policy can rapidly map perceived states to actions, but its performance is limited by the coverage and quality of its training experience. A practical autonomous robot therefore requires two complementary processes: a relatively slow and flexible thinking process for handling unfamiliar situations and a fast learned policy for routine behavior.

A central but insufficiently studied problem lies between these two processes: how thinking should guide the construction of materials used to improve the behavior policy. Existing robot-learning methods commonly assume that training samples are already available, are collected using predefined sensor configurations, or can be obtained through repeated interaction. Such assumptions are often violated in real operating environments. The supervision signal of an action may appear only after a delay; the consequences of an action may be distributed across vision, touch, force, audio, and internal robot states; relevant factors may be hidden among numerous unrelated environmental changes; and important situations may be too rare, expensive, or dangerous to reproduce directly.

For example, a robot may initially regard a grasp as successful, although the object slips several seconds later during transportation. A visual observation alone may not distinguish a light movable container from an identical-looking object fixed to a table, while force and motor-current signals provide the necessary evidence. In a complex environment, background motion or unrelated sounds may change more strongly than the factors that actually caused a task failure. Moreover, repeatedly dropping fragile objects or manipulating high-temperature materials merely to collect negative examples would be unsafe. These cases indicate that autonomous learning cannot be reduced to passively recording immediate state–action pairs.

The difficulty is not simply a lack of data. Rather, the robot faces a training-material gap between the materials required for a specific learning goal and those currently available. The gap may be temporal, because the final outcome has not yet appeared; observational, because the available sensor evidence is incomplete; semantic, because the relevant factors have not been identified; or experiential, because the required condition is rare or unavailable. Addressing these different deficiencies requires the robot to reason about when additional evidence may appear, which sensing modalities and time intervals should be examined, which factors should be retained, and whether missing materials should be physically acquired, retrieved, inferred, simulated, or generated.

This paper proposes TG-MAC, a Thought-Guided Material Acquisition and Construction model for autonomous robot learning. TG-MAC does not require the thinking module to replace the learned behavior policy. Instead, the behavior policy continues to execute familiar tasks efficiently, while the thinking module is activated when the robot identifies a capability limitation, unexpected outcome, unfamiliar situation, or external correction. For each learning issue, thinking determines the learning goal, analyzes the current environment and available information sources, constructs a structured description of the required training materials, and identifies the remaining material gap.

In TG-MAC, a training material is broader than a conventional input–label pair. It may contain multimodal observations, executed actions, environmental context, immediate and delayed outcomes, data provenance, and reliability information. This representation allows the robot to preserve the relationship between an action and its consequences across different sensors, time intervals, and material sources. According to the identified gap, TG-MAC dynamically selects or combines four material-construction mechanisms.

First, delayed-feedback retrospective association creates pending learning records and revisits them when a temporal, event-based, task-stage, or external-feedback condition is satisfied. This prevents an immediate observation from being incorrectly treated as the final outcome. Second, change-aware multimodal spatiotemporal evidence expansion examines complementary observations across sensors and time intervals. It considers not only observed changes but also expected changes that fail to occur. Third, goal-relevant factor identification and selection removes coincidental or noisy variables and retains a compact set of factors that is sufficient for the current learning goal. Fourth, thought-guided active material acquisition and supplementation decides whether missing materials should be obtained through physical interaction or supplemented through retrieval, analogy, inference, simulation, or generation. Supplemented materials preserve their provenance and confidence and are validated before being used for policy learning.

The distinction between material acquisition and material construction is important. Acquisition obtains additional evidence from the robot, its environment, humans, other agents, or external sources. Construction further includes retrospective association, cross-modal alignment, factor selection, inference, supplementation, and validation. TG-MAC therefore enables thinking to guide not only the collection of more data, but also the formation of learning-ready experience from incomplete, distributed, and heterogeneous evidence.

The main contributions of this paper are summarized as follows:

• We formulate autonomous robot learning as an environment-dependent training-material construction problem. The proposed model explicitly represents the gap between the materials required by a learning goal and those currently available.
• We propose four complementary mechanisms for resolving temporal incompleteness, observational incompleteness, relevance uncertainty, and experience unavailability. These mechanisms support delayed association, multimodal evidence expansion, goal-relevant factor selection, and active acquisition and supplementation.
• We introduce a thought-guided mechanism-selection process that dynamically chooses and combines material-construction operations according to their expected material gain, reliability, cost, uncertainty, and physical risk.

The remainder of this paper is organized as follows. Section 2 reviews autonomous robot learning, multimodal evidence processing, delayed feedback, and active data acquisition. Section 3 presents TG-MAC, including environment-aware material requirement modeling, the four material-construction mechanisms, and policy updating. Section 4 describes the experimental settings and analyzes the verification results. Finally, Section 5 concludes the paper and discusses future work.

2. Related Work

2.1. Autonomous Robot Learning and Self-Improvement

Robot learning commonly relies on imitation learning or reinforcement learning to transform interaction experience into executable behavior policies. Imitation learning can efficiently acquire complex skills from demonstrations, but the collection of high-quality demonstrations requires considerable human effort and normally limits the diversity of encountered conditions. Reinforcement learning allows a robot to improve through environmental interaction, but real-world deployment often requires task-specific reward design, repeated environmental resetting, and extensive safety supervision. To reduce such intervention, Sharma et al. proposed MEDAL++, in which a robot learns both to perform and undo a task and infers a reward function from initial demonstrations [1]. This design supports relatively long autonomous practice, but its data collection remains centered on a predefined task whose executable and reset behaviors have already been specified.

Generalist robot policies further enable experience reuse across tasks and embodiments. RoboCat learns from heterogeneous action-labelled trajectories and can use its current policy to generate additional data for subsequent training iterations [2]. Similarly, Zhou et al. use foundation models to guide semantically meaningful autonomous data collection and evaluate collected trajectories without requiring manual annotation [3]. These methods demonstrate that autonomous interaction and pretrained knowledge can jointly improve robot policies. However, they primarily address how to generate more trajectories or determine whether a collected trajectory is useful. They generally assume that the target task, observation structure, and principal outcome signal are already known, rather than explicitly reasoning about which components of a complete training material are missing.

Autonomous data collection also introduces practical limitations that cannot be addressed by increasing trajectory quantity alone. Mirchandani et al. show that autonomous collection pipelines may still require substantial environment engineering, reliable success detection, recovery procedures, and manual resetting, while additional human data can sometimes provide greater improvement than autonomously collected experience [4]. This observation suggests that autonomous learning should focus not only on collecting more data, but also on identifying what evidence is actually required before collection begins. In contrast to existing self-improvement systems, the proposed method represents a goal-dependent training-material gap containing missing inputs, contexts, immediate outcomes, delayed outcomes, sources, and reliability information. The thinking module then selects and combines dedicated mechanisms to resolve different forms of material incompleteness instead of treating every deficiency as a need for additional undifferentiated trajectories.

2.2. Multimodal Perception and Goal-Relevant Evidence

Multimodal perception provides complementary information that cannot always be recovered from vision alone. Li et al. combine vision, audio, and touch for robotic manipulation and show that the three modalities play different roles: vision describes global scene states, audio captures contact events that may be visually hidden, and touch provides detailed local interaction information [5]. Such multisensory systems are particularly beneficial in partially observable environments involving occlusion, contact, or subtle material properties. Nevertheless, conventional multimodal policies usually treat sensor measurements as predefined model inputs and optimize their fusion for a known downstream task. They do not normally determine whether an observed sensor change should become an input factor, a contextual condition, an outcome label, or evidence requiring later verification.

Recent studies have considered heterogeneous sensors and more selective fusion structures. FuSe grounds vision, touch, audio, and other heterogeneous sensory observations through language, allowing generalist policies to be adapted when large aligned datasets are unavailable for every modality [6]. Li et al. further argue that different modalities should not always be assigned symmetric roles and propose hierarchical audio–visual–proprioceptive fusion for contact-sensitive manipulation [7]. These approaches reduce the limitations of flat feature concatenation and demonstrate that task-relevant modalities should be processed according to their particular characteristics. However, the selected modality set and fusion architecture are still mainly established during model design or training. They do not continuously compare expected and observed sensor changes to decide which types of evidence should be collected for a newly identified learning goal.

The proposed framework therefore separates evidence expansion from factor selection. Change-aware multimodal spatiotemporal evidence expansion maintains access to all available sensing modalities and searches across sensors, time windows, locations, external devices, and evaluators when the current observation is incomplete. It considers not only observed changes but also expected changes that fail to occur, such as an object that does not move after a pushing action or a human who does not respond after a spoken message. Goal-relevant factor selection subsequently removes unrelated or unreliable information and determines the role of each retained factor in the training material. This separation differs from direct multimodal fusion: the objective is not simply to improve action prediction using more modalities, but to autonomously construct a compact and sufficiently supported training sample for a specific capability gap.

2.3. Active Perception, Delayed Feedback, and Material Acquisition

Active perception allows a robot to modify its viewpoint or behavior to obtain information that passive sensors cannot provide. Zaky et al. integrate viewpoint control with manipulation and representation learning, demonstrating that perception actions can improve both task performance and sample efficiency [8]. Schneider et al. use model uncertainty and expected information gain to plan exploratory manipulation actions in sparse-reward tasks [9]. He et al. further study the discrepancy between expected and realized information gain and improve the robustness of active information gathering [10]. These methods provide principled approaches for choosing informative actions, but they usually aim to reduce uncertainty about the current environmental state, map, dynamics model, or immediate manipulation decision.

The temporal availability of feedback creates another important learning problem. In sparse- and delayed-reward settings, the robot may observe only an episodic or later outcome and must determine which earlier actions contributed to it. Chen and Lin assign intermediate learning signals according to final episodic rewards to alleviate this credit-assignment problem [11]. Tang et al. consider composite delayed rewards whose values may depend on non-Markovian combinations of multiple steps and use sequence modeling to estimate their contributions [12]. These approaches improve policy optimization when reward signals arrive late. However, they generally assume that a delayed reward will eventually be supplied and focus on distributing that reward across a known trajectory. They do not address how a robot autonomously decides what future condition should be checked, which sensors should be consulted, or how the resulting feedback should be combined with the original context to construct a reusable training material.

The proposed method connects active acquisition and delayed feedback through explicit training-material requirements. When a required factor or experience does not occur naturally, the thinking module can propose actions that change the observation viewpoint, manipulate an object, vary an operating parameter, repeat a behavior, extend the observation period, enter another environment, or request information from humans or other robots. The selected action is evaluated according to its expected reduction of the current material gap, as well as its cost and safety risk. If the required outcome will only appear later, the robot creates a pending learning record with an event- or time-based checking condition and retrospectively associates the future result with the earlier state, action, and context. Thus, active perception and delayed feedback are not isolated solutions for immediate uncertainty or reward assignment; they are coordinated with multimodal evidence expansion and goal-relevant factor selection to construct complete materials for future behavior learning.

Overall, previous studies have made substantial progress in autonomous policy improvement, multimodal fusion, active perception, and learning from delayed outcomes. Their advantages provide important foundations for reducing human supervision and improving robot adaptability. Nevertheless, these research directions are commonly studied as separate learning or perception problems, and most assume that the task-specific training representation has already been defined. The proposed framework instead treats autonomous learning as the identification and resolution of training-material gaps. It unifies delayed-feedback retrospective association, change-aware multimodal spatiotemporal evidence expansion, goal-relevant factor selection, and thought-guided active acquisition under a common thinking-guided decision process. This enables the robot to reason not only about how to execute or explore, but also about what must be observed, retained, associated, or actively created before a behavior policy can be reliably updated.

3. TG-MAC: Thought-Guided Material Acquisition and Construction

3.1. Environment-Aware Material Requirement Modeling

We propose TG-MAC, a Thought-Guided Material Acquisition and Construction model for autonomous robot learning. TG-MAC enables a robot to reason not only about what capability should be learned, but also about what training materials are required under the current environmental conditions and how these materials should be acquired, associated, selected, supplemented, and validated.

An autonomous robot usually operates through two complementary decision processes. The first is a relatively slow and flexible thinking process, which may be implemented using an LLM, a planner, or another reasoning model. The second is a fast autonomous behavior policy learned from previous experience. The thinking process handles unfamiliar situations, analyzes capability limitations, and determines what should be learned, whereas the behavior policy repeatedly performs familiar tasks without requiring extensive reasoning.

Let $s_t$ denote the perceived robot and environmental state at time t, and let ${\pi }_{\theta }$ denote the current autonomous behavior policy. During normal operation, the robot generates a behavior command as

$$a_t={\pi }_{\theta }\left(s_t\right),$$

(1)

where $a_t$ is subsequently converted into physical actions by the execution module. The action changes the environment and the internal state of the robot, and the resulting changes are observed through available sensors.

TG-MAC does not require the thinking module to directly replace the learned behavior policy. Instead, thinking guides learning when the current policy cannot reliably handle a situation. A learning issue may be created when the robot detects a large difference between an expected outcome and an observed outcome, repeated task failure, high uncertainty, a previously unseen situation, an environmental change, or an externally provided correction.

Training-material construction is grounded in the actual operating environment. The thinking module analyzes the current state, recent interaction history, available sensing modalities, task constraints, and expected behavior outcomes. It then determines which observations, actions, contextual factors, and supervision signals are required for a learning goal g.

This environment-aware process is necessary because useful training evidence may not be immediately or directly available. An action outcome may appear only after a delay, different parts of the outcome may be observed by different sensors, and relevant factors may be hidden among many unrelated environmental changes. Moreover, rare, dangerous, or inaccessible conditions may require active interaction, retrieval, inference, simulation, or other forms of material supplementation.

Let ${\mathcal{H}}_t$ denote the recent interaction history and let ${\mathcal{S}}_t$ denote the currently accessible sensing and information sources. The required training-material description is generated as

$${\mathcal{M}}_g^{\mathrm{req}}\left(t\right)=\Psi \left(g,s_t,{\mathcal{H}}_t,{\mathcal{S}}_t\right),$$

(2)

where $\Psi (·)$ represents the thinking process that converts the learning goal and actual environmental conditions into a structured description of the required materials. Therefore, the required material structure is not fixed in advance and may change when the task, environment, available sensors, or expected outcomes change.

A training material is not limited to a conventional input–label pair. It may contain multimodal observations, executed actions, environmental context, immediate outcomes, delayed outcomes, provenance, and confidence information:

$$m_i=\left(X_i,a_i,c_i,Y_i,p_i,q_i\right),$$

(3)

where $X_i$ contains multimodal observations, $a_i$ is the executed behavior, $c_i$ represents contextual conditions, $Y_i$ contains one or more observed or supplemented outcomes, $p_i$ records the material source, and $q_i$ represents material quality or reliability.

The thinking module compares the required materials with the currently available materials ${\mathcal{M}}_g^{\mathrm{ava}}$ and identifies a training-material gap:

$$\Delta {\mathcal{M}}_g={\mathcal{M}}_g^{\mathrm{req}}\setminus {\mathcal{M}}_g^{\mathrm{ava}}.$$

(4)

The gap may indicate that an outcome has not yet appeared, the current observation is incomplete, relevant factors have not been identified, a required experience is unavailable, or an existing material has insufficient reliability. TG-MAC resolves these deficiencies through four complementary material-construction mechanisms.

3.2. Training Material Construction in Dynamic and Complex Environments

TG-MAC classifies training-material deficiencies into four main categories: temporal incompleteness, observational incompleteness, relevance uncertainty, and experience unavailability. The corresponding mechanisms can be selected independently or combined according to the current learning goal and environmental conditions.

3.2.1. Delayed-Feedback Retrospective Association

The outcome of a robot action may not be observable immediately. For example, an object may initially appear to be grasped successfully but slide from the gripper several seconds later. Similarly, the effects of parameter adjustment, navigation decisions, or environmental intervention may become visible only at a later task stage.

When such a delay is expected, the thinking module creates a pending learning record containing the learning goal, historical state, executed action, context, expected outcome, and future checking condition:

$$r_i=\left(g_i,s_{t_i},a_{t_i},c_{t_i},{\widehat{Y}}_i,{\varphi }_i^{\mathrm{check}}\right),$$

(5)

where ${\widehat{Y}}_i$ denotes the expected outcome and ${\varphi }_i^{\mathrm{check}}$ specifies when or under what condition the record should be revisited. The checking condition may depend on elapsed time, a sensor event, completion of a later task stage, or the arrival of external feedback.

The retrospective check is activated by

$${\chi }_i\left(t\right)=\mathbb{I}\left[t\ge t_i^{\mathrm{check}}\vee E_i\left(t\right)=1\vee F_i\left(t\right)=1\right],$$

(6)

where $E_i\left(t\right)$ indicates the occurrence of a predefined environmental or sensor event, and $F_i\left(t\right)$ indicates the arrival of external feedback. When ${\chi }_i\left(t\right)=1$, the robot recalls the corresponding record and associates the new outcome with the original state, action, and context:

$$\left(s_{t_i},a_{t_i},c_{t_i}\right)⟶Y_{t_i+\Delta t}.$$

(7)

This mechanism prevents the robot from incorrectly treating an immediate observation as the final result. It also enables the construction of training samples whose supervision signals are distributed over time. Unlike simple logging, the retrospective process is explicitly scheduled, triggered, and interpreted by the thinking module according to the learning goal.

3.2.2. Change-Aware Multimodal Spatiotemporal Evidence Expansion

A single sensor or observation time may reveal only part of an action outcome. The robot should therefore maintain access to multiple sensing modalities, such as vision, audio, touch, force, proprioception, temperature, internal system states, external devices, and human feedback.

However, TG-MAC does not indiscriminately use all raw sensor data as model inputs. Lightweight perception components continuously detect changes and summarize them into events. The thinking module then determines which observed changes, expected-but-absent changes, and unchanged reference states may contribute to the current learning goal.

For sensing modality k, a goal-dependent evidence score is defined as

$$S_{k,t}^g=\alpha {\Delta }_{k,t}+\beta R_{k,t}^g+\gamma C_{k,t}+\eta P_{k,t}-\lambda N_{k,t},$$

(8)

where ${\Delta }_{k,t}$ represents an observed change or a deviation from the expected change, $R_{k,t}^g$ is relevance to learning goal g, $C_{k,t}$ is sensor confidence, $P_{k,t}$ measures complementarity with other modalities, and $N_{k,t}$ represents noise or possible sensor failure.

The robot selects a complementary set of evidence rather than a single best sensor. For example, a grasping failure may be jointly indicated by visual displacement, reduced tactile pressure, decreased force, and a collision sound. Agreement among independent modalities increases confidence, whereas conflicting evidence can trigger further observation or thinking.

Different modalities may have different sampling frequencies and response delays. TG-MAC therefore aligns evidence around an action- or event-centered time window, including observations before, during, and after the action. This allows one behavior to be associated with direct sensory outcomes, internal robot outcomes, task-level outcomes, delayed environmental outcomes, and external evaluations.

Importantly, a sensor that does not change may also provide useful evidence. For example, if an object is expected to move after a pushing action but its visual position remains unchanged, the absence of movement may indicate that the object is fixed or too heavy. TG-MAC therefore considers both observed changes and expected changes that fail to occur.

3.2.3. Goal-Relevant Factor Identification and Selection

Multimodal evidence expansion may produce a large number of candidate factors, many of which are unrelated to the learning goal. A detected sensor change is not necessarily a useful learning factor. For example, background movement may coincide with a grasping failure without contributing to it.

The thinking module first proposes factors that may affect the target behavior, including perceptual inputs, action parameters, contextual conditions, immediate outcomes, delayed outcomes, and reliability indicators. These candidates are then evaluated using current observations, historical experience, statistical relationships, model predictions, or controlled interactions.

Let ${\mathcal{F}}_g$ denote the set of candidate factors for learning goal g. A relevance score can be represented as

$${\rho }_j^g={\lambda }_s{\rho }_j^{\mathrm{sem}}+{\lambda }_d{\rho }_j^{\mathrm{data}}+{\lambda }_c{\rho }_j^{\mathrm{causal}},$$

(9)

where ${\rho }_j^{\mathrm{sem}}$ represents semantic relevance inferred by the thinking module, ${\rho }_j^{\mathrm{data}}$ represents support from observed data, and ${\rho }_j^{\mathrm{causal}}$ represents evidence that the factor may influence the target outcome.

The selected factor set is

$${\mathcal{F}}_g^{*}=\left\{f_j\in {\mathcal{F}}_g\mid {\rho }_j^g\ge {\tau }_f\right\}.$$

(10)

The objective is not simply to select the factors with the largest numerical changes. Instead, TG-MAC constructs a compact but sufficient set that explains or predicts the target outcome. A retained factor may serve as a model input, a supervision signal, a contextual condition, a confidence value, or a future checking condition.

Thus, multimodal evidence expansion determines where potentially useful evidence may exist, whereas goal-relevant factor selection determines which parts of that evidence should finally be included in the training materials.

3.2.4. Thought-Guided Active Material Acquisition and Supplementation

Some objects, environmental conditions, outcomes, or failure cases required for learning may not naturally occur during normal robot operation. Other materials may be difficult, costly, unsafe, or time-consuming to obtain through physical interaction. Dangerous failures, for example, should not be repeatedly reproduced merely to collect data.

To address this problem, the robot uses thinking not only to determine how missing materials can be physically acquired or created, but also to supplement them when direct acquisition is impractical. Given the material gap $\Delta {\mathcal{M}}_g$, the thinking module may choose additional observation, active environmental interaction, retrieval of analogous experiences, inference from existing evidence, simulation, or generation of synthetic and counterfactual examples.

Physical acquisition may involve changing the observation viewpoint, manipulating an object, varying force or speed, repeating an operation, extending the observation period, entering another environment, or requesting information from a human or another robot. Supplementation may retrieve related experiences from memory, transfer materials from similar tasks, infer missing factors or outcomes, or construct candidate examples using a world model, simulator, or reasoning model.

Let ${\mathcal{O}}_g$ denote the candidate acquisition and supplementation operations. TG-MAC selects an operation according to expected material gain, verifiability, cost, uncertainty, and safety risk:

$$\begin{array}{c}\hfill o_g^{*}=arg\underset{o\in {\mathcal{O}}_g}{max}\left[G\left(o,\Delta {\mathcal{M}}_g\right)+{\lambda }_{v}V\left(o\right)-{\lambda }_{c}C\left(o\right)\right.\\ \hfill \left.-{\lambda }_{u}U\left(o\right)-{\lambda }_{r}R\left(o\right)\right],\end{array}$$

(11)

where $G(o,\Delta {\mathcal{M}}_g)$ estimates how much operation o can reduce the material gap, $V\left(o\right)$ denotes its verifiability, $C\left(o\right)$ is its acquisition or computational cost, $U\left(o\right)$ represents uncertainty, and $R\left(o\right)$ denotes physical or operational risk.

Direct acquisition is preferred when the required evidence can be obtained safely and efficiently. Supplementation is preferred when real-world acquisition is unavailable, expensive, dangerous, or excessively slow. The two strategies may also be combined. For example, the thinking module may first generate provisional counterfactual materials and then direct the robot to collect only the real evidence required to validate their most uncertain factors.

A supplemented material is represented as

$${\tilde{m}}_i=\left({\tilde{X}}_i,{\tilde{a}}_i,{\tilde{c}}_i,{\tilde{Y}}_i,p_i^{\sup },q_i^{\sup }\right),$$

(12)

where the tilde indicates that one or more components are retrieved, inferred, simulated, or generated rather than directly observed. The provenance term $p_i^{\sup }$ records the source and supplementation method, while $q_i^{\sup }$ represents confidence.

Because supplemented materials may contain incorrect assumptions, unrealistic combinations, or reasoning errors, they must be validated before policy learning. A candidate is accepted only when

$$V\left({\tilde{m}}_i\right)\ge {\tau }_v,{\omega }_i=q_i^{\sup }V\left({\tilde{m}}_i\right),0\le {\omega }_i\le 1,$$

(13)

where ${\tau }_v$ is the validation threshold and ${\omega }_i$ is the training weight assigned to the supplemented material. Insufficiently validated materials remain provisional, trigger targeted physical acquisition, or are discarded.

This mechanism extends conventional active data collection by allowing the robot to reason explicitly about the material gap. Thinking determines not only how missing evidence can be physically collected, but also how difficult-to-obtain materials can be retrieved, inferred, simulated, or generated while preserving their provenance and uncertainty.

3.3. Mechanism Selection, Material Sufficiency, and Policy Update

The four mechanisms are not required to operate in a fixed sequence. The thinking module selects one or more mechanisms according to the type of material gap and the current environment.

Delayed-feedback association is selected when an outcome is not yet available. Multimodal evidence expansion is selected when the observation is incomplete or unreliable. Factor selection is used when the available evidence contains many uncertain or irrelevant variables. Active acquisition and supplementation are selected when the required experience is unavailable or impractical to obtain.

Let

$$\mathcal{Z}=\left\{z_{\mathrm{delay}},z_{\mathrm{multi}},z_{\mathrm{factor}},z_{\mathrm{active}}\right\}$$

denote the four mechanisms. The selected mechanism set is

$$Z_g^{*}=arg\underset{Z\subseteq \mathcal{Z}}{max}\left[G_{\mathrm{mat}}\left(Z,\Delta {\mathcal{M}}_g\right)-{\lambda }_{c}C\left(Z\right)-{\lambda }_{r}R\left(Z\right)\right],$$

(14)

where $G_{\mathrm{mat}}$ estimates the expected reduction of the training-material gap, $C\left(Z\right)$ is the execution and computation cost, and $R\left(Z\right)$ is the associated risk.

This formulation allows several mechanisms to be combined. For example, the robot may actively create a difficult condition, observe the resulting changes through several modalities, select the relevant factors, and later associate a delayed outcome with the original action.

The acquired and constructed materials are continuously evaluated before policy learning is triggered. Material sufficiency should consider not only sample quantity, but also coverage, diversity, reliability, balance, and goal relevance:

$$\begin{array}{cc}\hfill Q\left({\mathcal{D}}_g\right)& =w_1{Q}_{\mathrm{quantity}}+w_2{Q}_{\mathrm{coverage}}+w_3{Q}_{\mathrm{diversity}}\hfill \\ \hfill & +w_4{Q}_{\mathrm{reliability}}+w_5{Q}_{\mathrm{relevance}}.\hfill \end{array}$$

(15)

If $Q\left({\mathcal{D}}_g\right)$ is below the required threshold, the thinking module identifies the remaining deficiencies and continues material construction. Once the materials are sufficient, the learning module trains a candidate behavior policy:

$${\theta }^{\prime }=Learn\left(\theta ,{\mathcal{D}}_g\right).$$

(16)

The candidate policy is then compared with the current policy in terms of target-task performance, safety, generalization, and preservation of previously learned capabilities. It is deployed only when the improvement is sufficient and no unacceptable risk or degradation is introduced. Otherwise, the robot retains the original policy, revises the material requirement, and continues collecting or constructing evidence.

The complete TG-MAC learning loop can be summarized as

$$\begin{array}{cc}\hfill \mathrm{capability}\mathrm{gap}& \to \mathrm{learning}\mathrm{goal}\hfill \\ \hfill & \to \mathrm{environment}-\mathrm{aware}\mathrm{material}\mathrm{requirement}\hfill \\ \hfill & \to \mathrm{material}-\mathrm{gap}\mathrm{identification}\hfill \\ \hfill & \to \mathrm{mechanism}\mathrm{selection}\hfill \\ \hfill & \to \mathrm{material}\mathrm{acquisition}\mathrm{and}\mathrm{construction}\hfill \\ \hfill & \to \mathrm{policy}\mathrm{learning}\hfill \\ \hfill & \to \mathrm{validation}\hfill \\ \hfill & \to \mathrm{continued}\mathrm{operation}.\hfill \end{array}$$

(17)

Through this loop, thinking is not limited to generating immediate robot actions. It determines what the robot needs to learn, what evidence is missing, where and when that evidence may appear, which sensors and factors should be used, and whether unavailable materials should be actively acquired or carefully supplemented. The learned behavior policy then converts the constructed experience into fast and repeatable autonomous behavior, enabling continued adaptation in complex, partially observable, and time-varying environments.

4. Verification

This section evaluates whether TG-MAC can construct more useful training materials under delayed outcomes, complex multimodal observations, and rare or unsafe operating conditions. The verification focuses on five questions: (1) whether delayed outcomes can be correctly associated with earlier robot actions; (2) whether complementary evidence can be obtained from multiple modalities; (3) whether irrelevant environmental factors can be removed without losing essential evidence; (4) whether active acquisition and supplementation can reduce the cost and risk of obtaining rare materials; and (5) whether material validation improves the resulting behavior policy.

4.1. Experimental Environment and Protocol

4.1.0.1. Verification platform.

The experiments were conducted in a controlled mobile-manipulation simulation. The simulated robot contains a mobile base, a manipulator, and a gripper. Its accessible information sources include RGB-D vision, touch, force/torque, audio, joint states, motor current, internal control errors, external observations, and human feedback. The simulation does not aim to reproduce a specific physical robot. Instead, it provides controllable temporal, multimodal, and safety-related conditions that allow the individual training-material construction mechanisms to be evaluated separately.

The current verification uses deterministic high-level reasoning rules to instantiate the thinking interface of TG-MAC. Therefore, the evaluated mechanisms do not depend on a particular LLM. All compared methods use the same generated scenarios and the same downstream behavior-performance evaluation procedure. Only the process used to construct training materials is changed. Consequently, the experiments evaluate the contribution of material construction rather than differences in policy-network architecture or language-model capability.

Each experiment contains 100 paired rounds, resulting in 300 test scenarios in total. In each round, all compared methods receive the same initial state, environmental conditions, available sensor information, and ground-truth outcome. The base random seed is fixed to 20260804. A round is stored only after all methods have completed successfully, and a completion flag allows an interrupted experiment to resume from the first incomplete round. Reported results are means over the 100 rounds. For the principal comparisons, paired bootstrap confidence intervals were calculated using 20,000 resamples.

4.1.0.2. Test scenarios.

Table 1 summarizes the three experiments. Experiment 1 simulates grasping and transportation with delayed failures. Experiment 2 simulates manipulation in complex multimodal environments with sensor corruption and irrelevant environmental changes. Experiment 3 simulates learning under rare, unavailable, or unsafe conditions in which materials may need to be actively acquired or supplemented.

4.1.0.3. Metrics.

For Experiment 1, delayed-outcome association accuracy measures whether the final outcome is assigned to the correct historical state–action record. Delayed-failure recall is calculated only over rounds containing an actual final failure. The complete-sample rate measures the proportion of materials containing the initial state, action, context, and final outcome, while the false-success rate measures how often an operation that eventually fails is incorrectly recorded as successful.

For Experiment 2, factor-selection precision, recall, and F1 are calculated by comparing the selected factors with the known goal-relevant factors used to generate each scenario. Outcome completeness measures the coverage of the required sensing modalities. Irrelevant-factor retention is the proportion of known distractors retained in the constructed material. Processing cost reflects the amount of selected evidence and the additional overhead caused by corrupted modalities.

For Experiment 3, material-gap reduction is defined as

$$G_{\mathrm{gap}}={\displaystyle \frac{∥\Delta {\mathcal{M}}_g^{\mathrm{before}}∥-∥\Delta {\mathcal{M}}_g^{\mathrm{after}}∥}{∥\Delta {\mathcal{M}}_g^{\mathrm{before}}∥}}.$$

(18)

The experiment also records rare-condition coverage, real interactions, dangerous attempts, physical failures, erroneous-material acceptance, and total acquisition cost. The policy-success and safety-success scores are common downstream performance estimates computed using the same fixed evaluation procedure for all compared methods; they are not physical-robot success rates obtained after deploying a separately trained control policy.

4.2. Experiment 1: Delayed-Outcome Association

4.2.0.4. Scenario and configuration.

The simulated robot grasps an object from a table and transports it to a target location. The objects include metal cans, sealed bottles, glass cups, plastic cups, and small boxes. Across the 100 rounds, 39 objects have wet surfaces, 28 have smooth surfaces, and 33 have dry surfaces. Object weight, transportation speed, feedback type, and outcome delay are varied independently.

Seventeen rounds have a delay of at most 1 s, 34 have delays between 1 and 5 s, 27 have delays between 5 and 9 s, and 22 have delays between 9 and 12 s. The outcome may be triggered by a force decrease, collision sound, visual slippage, tactile-pressure decrease, or task completion. Additional background motions, unrelated sounds, and object changes are inserted between the initial grasp and the final outcome. Thirty-eight rounds contain a final grasp or transportation failure, whereas 62 rounds are ultimately successful.

The compared methods are Immediate only, which uses only the immediate post-action observation; Fixed delay, which checks the result after a fixed delay; Fixed window, which retains a fixed-length observation window; Event only, which relies on predefined sensor events; TG-MAC w/o thinking trigger, which maintains delayed records but does not reason about the checking condition; and the complete TG-MAC method.

4.2.0.5. Results.

Table 2 reports the results. TG-MAC achieves an association accuracy of 98.0%, compared with 87.0% for TG-MAC without the thinking trigger, 86.0% for event-only checking, and 83.0% for the fixed-window method. Its complete-sample rate reaches 96.5%, which is 12.1 percentage points higher than the version without a thinking trigger.

The false-success rate is particularly important because an immediate grasp may appear successful even though the object later slips. Immediate-only recording incorrectly labels 36.0% of all rounds as successful, whereas TG-MAC reduces this rate to 2.0%. Among the 38 actual failure rounds, TG-MAC detects 94.7% of delayed failures, compared with 65.8% without a thinking-generated trigger, 63.2% with event-only checking, and 55.3% with a fixed observation window.

The advantage is more pronounced for long delays. Among the seven failure rounds with delays of at least 9 s, TG-MAC detects 85.7% of the failures. Both fixed-delay and fixed-window recording detect none of these long-delay failures, event-only checking detects 57.1%, and the method without a thinking trigger detects 42.9%. This result shows that a single fixed temporal window cannot cover outcomes whose appearance time and trigger conditions vary across tasks.

Figure 1 visualizes the two most representative metrics. TG-MAC provides the highest association accuracy while producing the lowest false-success rate. Relative to TG-MAC without the thinking trigger, the association accuracy improves by 11.0 percentage points, with a paired bootstrap 95% confidence interval of $[5.0,18.0]$ percentage points. The estimated post-learning success improves by 4.36 percentage points, with a 95% confidence interval of $[2.58,6.16]$ percentage points.

The improvement has a moderate resource cost. TG-MAC retains four units of data and performs 2.5 checks per round on average, whereas immediate-only recording uses one data unit and one check. Nevertheless, TG-MAC requires less retained data than the fixed-window method, whose cost is 8.0. The results therefore demonstrate that environment-dependent retrospective checking provides a substantially better accuracy–cost balance than continuously retaining a long fixed window.

4.3. Experiment 2: Multimodal Evidence Expansion and Factor Selection

4.3.0.6. Scenario and configuration.

This experiment evaluates whether TG-MAC can construct useful materials from complex and partially reliable observations. The robot performs three kinds of tasks: grasp-stability evaluation in 31 rounds, object-mobility evaluation in 27 rounds, and container-state evaluation in 42 rounds.

The available modalities include vision, touch, force/torque, audio, and proprioception. Each scenario contains between two and seven irrelevant factors, including background motion, unrelated sound, lighting changes, room temperature, and non-target-object motion. Thirty-two rounds contain no corrupted modality, 34 contain one corrupted modality, and 34 contain two corrupted modalities. In 17 rounds, an expected visual change does not occur; for example, an object is pushed but does not move. Such absence is itself a goal-relevant result.

The compared methods are vision-only input, a predefined best single sensor, concatenation of all sensor factors, fixed multimodal fusion, selection based only on observed changes, TG-MAC without factor selection, TG-MAC without evidence expansion, and complete TG-MAC.

4.3.0.7. Results.

As shown in Table 3, no single baseline dominates all metrics. Concatenating all sensors obtains 100.0% recall and outcome completeness, but its precision is only 41.1%, it retains all irrelevant factors, and its processing cost reaches 23.66. The best-single-sensor method has high precision and low cost, but its recall is only 41.7% and its outcome completeness is 27.3%. This confirms that neither indiscriminate multimodal input nor an isolated sensor is sufficient.

TG-MAC achieves the highest factor F1 of 71.2% and the highest policy-success score of 84.7%. Compared with change-only selection, its precision increases from 61.8% to 74.9%, while its irrelevant-factor retention decreases from 67.5% to 27.4%. Its processing cost is 7.64, which is 35.7% lower than the 11.88 cost of change-only selection.

The F1 improvement over change-only selection is 4.94 percentage points, with a paired bootstrap 95% confidence interval of $[2.47,7.45]$ percentage points. The policy-success improvement is 9.69 percentage points, with a 95% confidence interval of $[8.32,11.12]$ percentage points. The processing cost decreases by 4.25 units, with a 95% confidence interval of $[-5.02,-3.48]$.

Figure 2 shows that the advantage does not arise from retaining the largest number of factors. All-sensor concatenation and TG-MAC without factor selection obtain high coverage but retain many distractors and produce lower policy performance. Conversely, TG-MAC without evidence expansion has low processing cost but misses important modalities, producing an F1 of only 37.5%. The complete method therefore benefits from first expanding possible evidence and then selecting factors according to the learning goal.

The difficult subsets further clarify the source of the improvement. In the 34 rounds containing two corrupted modalities, TG-MAC obtains a policy-success score of 81.7%, compared with 67.6% for change-only selection. The factor F1 scores are 66.4% and 65.8%, respectively. Thus, although their factor F1 scores are similar in this subset, TG-MAC obtains substantially better downstream behavior performance by considering modality reliability and cross-modal consistency.

In the 25 rounds containing six or seven distractors, TG-MAC obtains a factor F1 of 65.8% and policy success of 84.2%, whereas change-only selection obtains 56.8% and 73.2%, respectively. This result demonstrates that merely selecting factors with large observed changes is insufficient when the environment contains many unrelated changes.

In the 17 expected-but-absent-change scenarios, TG-MAC identifies the absent result in all cases, whereas change-only selection detects only 70.6%. The corresponding policy-success scores are 84.7% and 74.1%. Vision-only input does not detect any of these absent outcomes and obtains a policy-success score of 61.0%. This demonstrates that the absence of an expected change can be as informative as an observed change and should be explicitly represented in the training material.

TG-MAC does not have the highest raw recall or outcome completeness. Its recall of 70.3% is lower than the 74.1% obtained by change-only selection and the 100.0% obtained by all-sensor concatenation. The main advantage is therefore not indiscriminate evidence completeness, but a better balance among relevance, coverage, computational cost, and downstream behavior performance.

4.4. Experiment 3: Active Acquisition and Material Supplementation

4.4.0.8. Scenario and configuration.

This experiment considers materials that are rare, unavailable, expensive, or unsafe to obtain directly. The robot learns how to transport and place objects under six types of difficult conditions: unavailable sensing in 27 rounds, rare liquid movement in 17 rounds, hot objects in 15 rounds, wet objects in 15 rounds, fragile containers in 15 rounds, and novel objects in 11 rounds.

Each scenario is assigned a rarity level, physical risk, simulation fidelity, similarity to stored experience, and external verifiability. Thirty-three rounds have a physical-risk value of at least 0.7, and 31 have a rarity value of at least 0.8. Direct interaction is possible in some conditions, while others are better handled through retrieval, inference, simulation, or synthetic and counterfactual supplementation.

The compared methods are passive accumulation, random exploration, uncertainty-based active acquisition, active acquisition without supplementation, supplementation without physical acquisition, TG-MAC without material validation, and complete TG-MAC.

4.4.0.9. Results.

Table 4 shows that TG-MAC provides the highest policy-success score of 80.7%. It reduces 71.4% of the material gap using only 2.12 real interactions per round. In comparison, active acquisition alone reduces 62.0% of the gap using 4.34 interactions, while passive accumulation reduces only 13.4% despite using 10 interactions.

Compared with active acquisition alone, TG-MAC improves material-gap reduction by 9.34 percentage points and policy success by 7.41 percentage points. At the same time, it reduces the number of real interactions from 4.34 to 2.12, corresponding to a reduction of 51.2%. Its dangerous-attempt count decreases from 0.410 to 0.069, and its total cost decreases from 5.70 to 3.86.

The paired bootstrap 95% confidence intervals are $[8.56,10.11]$ percentage points for gap reduction, $[6.89,7.91]$ percentage points for policy success, and $[-2.44,-1.99]$ interactions for real-world acquisition.

Figure 3 illustrates the central trade-off. Supplementation-only learning requires no real interaction and achieves 100.0% simulated safety, but its policy-success score is only 65.6%. Active-only learning obtains a higher policy score of 73.3%, but requires more than twice as many real interactions as TG-MAC. The complete method combines selective real acquisition with supplementation and validation, resulting in the highest final policy performance.

The advantage remains under difficult subsets. In the 33 high-risk rounds, active-only learning obtains 70.7% policy success, 89.7% safety success, and 0.517 dangerous attempts per round. TG-MAC increases policy success to 80.1% and safety success to 97.6%, while reducing dangerous attempts to 0.119. Supplementation-only learning avoids all dangerous attempts and achieves 100.0% safety success, but its policy success remains at 65.2%.

In the 31 high-rarity rounds, TG-MAC reduces 70.7% of the material gap using 2.16 real interactions and obtains 80.7% policy success. Active-only learning reduces 59.4% of the gap using 4.29 interactions and obtains 72.3% policy success, while supplementation-only learning reduces 59.2% of the gap and obtains 66.0% policy success. These results show that active acquisition and supplementation are complementary: real interaction improves grounding, whereas supplementation improves coverage and reduces physical cost.

The validation ablation provides an additional result. TG-MAC without validation reports the largest gap reduction, 81.1%, but accepts 20.2% of incorrect supplemented materials and achieves only 72.2% policy success. The complete method accepts only 5.3% incorrect materials and raises policy success to 80.7%. Relative to the no-validation version, the error-acceptance rate decreases by 14.81 percentage points and policy success increases by 8.50 percentage points. Their paired bootstrap 95% confidence intervals are $[-14.98,-14.64]$ percentage points for erroneous-material acceptance and $[8.08,8.93]$ percentage points for policy success.

This result demonstrates that reducing the apparent material gap does not necessarily improve learning unless inferred or generated materials are assigned provenance, confidence, and validation status.

4.5. Overall Findings and Discussion

The three experiments support the main design choices of TG-MAC. First, environment-dependent retrospective checking is more reliable than immediate feedback, a fixed temporal window, or a single event trigger when outcomes have variable delays. Second, evidence expansion and factor selection should be used together. Expansion alone preserves excessive noise, whereas selection without expansion misses necessary modalities. Third, active acquisition and supplementation address different limitations: acquisition provides grounded evidence, while supplementation reduces cost and risk when real experience is rare or unsafe. Finally, the validation results show that material quantity or gap coverage alone is insufficient; material reliability has a direct effect on the final behavior policy.

The results also reveal several trade-offs. TG-MAC requires more checking than immediate-only feedback, and it does not maximize raw multimodal recall. Similarly, supplementation-only learning remains cheaper and safer when physical performance is not considered. TG-MAC instead aims to obtain a better joint balance among material completeness, relevance, policy performance, physical risk, and acquisition cost.

The current results constitute a controlled mechanism-level verification. They show that the four material-construction mechanisms operate as intended under systematically varied temporal, multimodal, and safety conditions. Because the current experiments use a simulated manipulation environment and a common policy-performance proxy, they do not by themselves establish performance on a specific physical robot. A subsequent physical-robot or high-fidelity simulation study can retain the same paired protocol while replacing the scenario generator and downstream performance proxy with real sensor streams, policy training, and task execution.

5. Conclusion

This paper proposed TG-MAC, a thought-guided material acquisition and construction model for autonomous robot learning. Instead of assuming that appropriate training data are already available, TG-MAC allows the robot to identify environment-dependent material requirements and resolve temporal, observational, relevance, and availability deficiencies. The model combines delayed-feedback retrospective association, multimodal spatiotemporal evidence expansion, goal-relevant factor selection, and active acquisition and supplementation with provenance and validation.

Controlled experiments showed that TG-MAC improved delayed-outcome association, reduced irrelevant multimodal evidence, and achieved a better balance among policy performance, real interaction cost, and operational safety. The results also demonstrated that collecting or generating more materials is not sufficient unless their relevance and reliability are properly evaluated. Future work will integrate TG-MAC with physical robots, real sensor streams, and online policy training to evaluate its long-term adaptation in open and changing environments.

Appendix A.1. Composition of the Test Scenarios

Table A1 reports the scenario distributions used in the three experiments. These distributions are included to clarify the number of cases supporting the aggregate and subset results reported in the main text.

Table A1. Composition of the test scenarios used in the three experiments.

Experiment	Scenario category	Number of rounds
Exp. 1	Wet, smooth, and dry object surfaces	39, 28, and 33
Exp. 1	Outcome delay $\le 1$ s, $(1,5]$ s, $(5,9]$ s, and $(9,12]$ s	17, 34, 27, and 22
Exp. 1	Final failure and final success	38 and 62
Exp. 1	Final failures with delay $\ge 9$ s	7
Exp. 2	Grasp-stability, object-mobility, and container-state tasks	31, 27, and 42
Exp. 2	Zero, one, and two corrupted modalities	32, 34, and 34
Exp. 2	Expected change does not occur	17
Exp. 2	Six or seven irrelevant distractors	25
Exp. 3	Unavailable sensor, rare liquid shift, hot object, wet object, fragile container, and novel object	27, 17, 15, 15, 15, and 11
Exp. 3	High-risk conditions with risk $\ge 0.7$	33
Exp. 3	High-rarity conditions with rarity $\ge 0.8$	31

Table A1. Composition of the test scenarios used in the three experiments.

Experiment	Scenario category	Number of rounds
Exp. 1	Wet, smooth, and dry object surfaces	39, 28, and 33
Exp. 1	Outcome delay $\le 1$ s, $(1,5]$ s, $(5,9]$ s, and $(9,12]$ s	17, 34, 27, and 22
Exp. 1	Final failure and final success	38 and 62
Exp. 1	Final failures with delay $\ge 9$ s	7
Exp. 2	Grasp-stability, object-mobility, and container-state tasks	31, 27, and 42
Exp. 2	Zero, one, and two corrupted modalities	32, 34, and 34
Exp. 2	Expected change does not occur	17
Exp. 2	Six or seven irrelevant distractors	25
Exp. 3	Unavailable sensor, rare liquid shift, hot object, wet object, fragile container, and novel object	27, 17, 15, 15, 15, and 11
Exp. 3	High-risk conditions with risk $\ge 0.7$	33
Exp. 3	High-rarity conditions with rarity $\ge 0.8$	31

Appendix A.2. Complete Results for Delayed-Feedback Association

Table A2 presents all primary metrics used to evaluate delayed-feedback retrospective association. Failure recall is calculated over the 38 rounds in which the operation ultimately fails. Association accuracy, complete-sample rate, false-success rate, and estimated post-learning success are calculated over all 100 rounds.

Table A2. Complete primary results for delayed-feedback retrospective association. Higher is better except for false-success rate, checking cost, and data cost.

Method	Association accuracy (%)	Complete samples (%)	Failure recall (%)	False success (%) ↓	Post-learning success (%)	Checks ↓	Data cost ↓
Immediate only	13.0	34.8	5.3	36.0	64.6	1.0	1.0
Fixed delay	76.0	78.0	36.8	24.0	81.5	1.0	2.0
Fixed window	83.0	78.8	55.3	17.0	83.4	1.0	8.0
Event only	86.0	77.5	63.2	14.0	84.1	1.0–2.5	3.0
TG-MAC w/o thinking trigger	87.0	84.4	65.8	13.0	85.2	2.5	4.5
TG-MAC	98.0	96.5	94.7	2.0	89.5	2.5	4.0

Table A2. Complete primary results for delayed-feedback retrospective association. Higher is better except for false-success rate, checking cost, and data cost.

Method	Association accuracy (%)	Complete samples (%)	Failure recall (%)	False success (%) ↓	Post-learning success (%)	Checks ↓	Data cost ↓
Immediate only	13.0	34.8	5.3	36.0	64.6	1.0	1.0
Fixed delay	76.0	78.0	36.8	24.0	81.5	1.0	2.0
Fixed window	83.0	78.8	55.3	17.0	83.4	1.0	8.0
Event only	86.0	77.5	63.2	14.0	84.1	1.0–2.5	3.0
TG-MAC w/o thinking trigger	87.0	84.4	65.8	13.0	85.2	2.5	4.5
TG-MAC	98.0	96.5	94.7	2.0	89.5	2.5	4.0

The complete method increases association accuracy from 87.0% to 98.0% relative to TG-MAC without the thinking-generated trigger. It also increases failure recall from 65.8% to 94.7% and decreases the false-success rate from 13.0% to 2.0%. These results indicate that maintaining a delayed record alone is insufficient when the appearance time and trigger type of the final outcome vary across tasks.

Appendix A.3. Complete Results for Multimodal Evidence and Factor Selection

Table A3 reports the complete primary results for multimodal evidence expansion and goal-relevant factor selection. Precision, recall, and F1 are calculated by comparing the selected factors with the known goal-relevant factors used to generate each scenario.

Table A3. Complete primary results for multimodal evidence expansion and goal-relevant factor selection. Higher is better except for irrelevant-factor retention and processing cost.

Method	Precision (%)	Recall (%)	Factor F1 (%)	Outcome completeness (%)	Irrelevant retention (%) ↓	Policy success (%)	Cost ↓
Vision only	33.3	16.7	22.1	27.3	29.0	67.0	4.98
Best single sensor	86.0	41.7	55.8	27.3	4.6	75.0	4.86
All-sensor concatenation	41.1	100.0	57.9	100.0	100.0	64.3	23.66
Fixed multimodal fusion	51.3	75.6	60.8	72.8	53.2	69.4	14.52
Change-only selection	61.8	74.1	66.2	90.8	67.5	75.0	11.88
TG-MAC w/o factor selection	51.1	83.5	62.5	95.9	79.6	74.8	15.15
TG-MAC w/o evidence expansion	59.2	29.4	37.5	45.6	30.5	73.1	5.05
TG-MAC	74.9	70.3	71.2	77.3	27.4	84.7	7.64

Table A3. Complete primary results for multimodal evidence expansion and goal-relevant factor selection. Higher is better except for irrelevant-factor retention and processing cost.

Method	Precision (%)	Recall (%)	Factor F1 (%)	Outcome completeness (%)	Irrelevant retention (%) ↓	Policy success (%)	Cost ↓
Vision only	33.3	16.7	22.1	27.3	29.0	67.0	4.98
Best single sensor	86.0	41.7	55.8	27.3	4.6	75.0	4.86
All-sensor concatenation	41.1	100.0	57.9	100.0	100.0	64.3	23.66
Fixed multimodal fusion	51.3	75.6	60.8	72.8	53.2	69.4	14.52
Change-only selection	61.8	74.1	66.2	90.8	67.5	75.0	11.88
TG-MAC w/o factor selection	51.1	83.5	62.5	95.9	79.6	74.8	15.15
TG-MAC w/o evidence expansion	59.2	29.4	37.5	45.6	30.5	73.1	5.05
TG-MAC	74.9	70.3	71.2	77.3	27.4	84.7	7.64

The all-sensor method obtains complete recall but retains all irrelevant factors and produces a policy-success score of only 64.3%. Conversely, TG-MAC without evidence expansion has a low processing cost but obtains only 37.5% factor F1. The complete TG-MAC method obtains the highest factor F1 of 71.2% and policy-success score of 84.7%, while retaining only 27.4% of the known irrelevant factors.

These results show that evidence expansion and factor selection are complementary. Expansion is required to avoid missing useful modalities, whereas selection is required to remove coincidental environmental changes and unreliable sensing factors.

Appendix A.4. Complete Results for Active Acquisition and Supplementation

Table A4 presents the complete primary results for active material acquisition and supplementation. The compared methods differ in whether they rely on passive accumulation, random exploration, uncertainty-based acquisition, physical acquisition, material supplementation, or material validation.

Table A4. Complete primary results for active material acquisition and supplementation. Higher is better for gap reduction, policy success, and safety success; lower is better for the remaining metrics.

Method	Gap reduction (%)	Real interactions ↓	Dangerous attempts ↓	Error acceptance (%) ↓	Policy success (%)	Safety success (%)	Total cost ↓
Passive accumulation	13.4	10.00	0.289	8.0	51.9	94.2	10.58
Random exploration	29.6	8.00	1.886	14.0	50.4	62.3	11.77
Uncertainty-based active	46.3	6.00	0.880	10.0	63.2	82.4	8.21
Active acquisition only	62.0	4.34	0.410	6.0	73.3	91.8	5.70
Supplementation only	58.6	0.00	0.000	17.8	65.6	100.0	1.41
TG-MAC w/o validation	81.1	2.67	0.154	20.2	72.2	96.9	4.03
TG-MAC	71.4	2.12	0.069	5.3	80.7	98.6	3.86

Table A4. Complete primary results for active material acquisition and supplementation. Higher is better for gap reduction, policy success, and safety success; lower is better for the remaining metrics.

Method	Gap reduction (%)	Real interactions ↓	Dangerous attempts ↓	Error acceptance (%) ↓	Policy success (%)	Safety success (%)	Total cost ↓
Passive accumulation	13.4	10.00	0.289	8.0	51.9	94.2	10.58
Random exploration	29.6	8.00	1.886	14.0	50.4	62.3	11.77
Uncertainty-based active	46.3	6.00	0.880	10.0	63.2	82.4	8.21
Active acquisition only	62.0	4.34	0.410	6.0	73.3	91.8	5.70
Supplementation only	58.6	0.00	0.000	17.8	65.6	100.0	1.41
TG-MAC w/o validation	81.1	2.67	0.154	20.2	72.2	96.9	4.03
TG-MAC	71.4	2.12	0.069	5.3	80.7	98.6	3.86

TG-MAC achieves the highest policy-success score of 80.7% while requiring only 2.12 real interactions per round. Compared with active acquisition only, TG-MAC reduces real interactions by 51.2%, decreases dangerous attempts from 0.410 to 0.069, and increases policy success from 73.3% to 80.7%.

The no-validation version obtains a larger apparent material-gap reduction of 81.1%, but its erroneous-material acceptance rate reaches 20.2%, and its policy-success score remains at 72.2%. The complete method reduces erroneous acceptance to 5.3% and increases policy success by 8.5 percentage points. This result shows that material-gap reduction alone is not a sufficient measure of learning quality.

Appendix A.5. Results under Specially Designed Difficult Conditions

Table A5 reports the subset results used in the main-text discussion. These subsets correspond directly to the main problems addressed by TG-MAC: variable temporal delays, incomplete or corrupted sensing, irrelevant environmental changes, absent expected outcomes, rare experience, and unsafe physical interaction.

Table A5. Results under the specially designed difficult conditions.

Experiment	Difficult condition and metric	Rounds	Compared method	Compared result	TG-MAC
Exp. 1	Delay $\ge 9$ s: failure recall	7	TG-MAC w/o thinking trigger	42.9%	85.7%
Exp. 1	Delay $\ge 9$ s: failure recall	7	Fixed window	0.0%	85.7%
Exp. 2	Two corrupted modalities: policy success	34	Change-only selection	67.6%	81.7%
Exp. 2	Two corrupted modalities: factor F1	34	Change-only selection	65.8%	66.4%
Exp. 2	Six or seven distractors: factor F1	25	Change-only selection	56.8%	65.8%
Exp. 2	Six or seven distractors: policy success	25	Change-only selection	73.2%	84.2%
Exp. 2	Expected change does not occur: detection rate	17	Change-only selection	70.6%	100.0%
Exp. 2	Expected change does not occur: policy success	17	Change-only selection	74.1%	84.7%
Exp. 3	High-risk conditions: policy success	33	Active acquisition only	70.7%	80.1%
Exp. 3	High-risk conditions: safety success	33	Active acquisition only	89.7%	97.6%
Exp. 3	High-risk conditions: dangerous attempts	33	Active acquisition only	0.517	0.119
Exp. 3	High-rarity conditions: gap reduction	31	Active acquisition only	59.4%	70.7%
Exp. 3	High-rarity conditions: real interactions	31	Active acquisition only	4.29	2.16
Exp. 3	High-rarity conditions: policy success	31	Active acquisition only	72.3%	80.7%

Table A5. Results under the specially designed difficult conditions.

Experiment	Difficult condition and metric	Rounds	Compared method	Compared result	TG-MAC
Exp. 1	Delay $\ge 9$ s: failure recall	7	TG-MAC w/o thinking trigger	42.9%	85.7%
Exp. 1	Delay $\ge 9$ s: failure recall	7	Fixed window	0.0%	85.7%
Exp. 2	Two corrupted modalities: policy success	34	Change-only selection	67.6%	81.7%
Exp. 2	Two corrupted modalities: factor F1	34	Change-only selection	65.8%	66.4%
Exp. 2	Six or seven distractors: factor F1	25	Change-only selection	56.8%	65.8%
Exp. 2	Six or seven distractors: policy success	25	Change-only selection	73.2%	84.2%
Exp. 2	Expected change does not occur: detection rate	17	Change-only selection	70.6%	100.0%
Exp. 2	Expected change does not occur: policy success	17	Change-only selection	74.1%	84.7%
Exp. 3	High-risk conditions: policy success	33	Active acquisition only	70.7%	80.1%
Exp. 3	High-risk conditions: safety success	33	Active acquisition only	89.7%	97.6%
Exp. 3	High-risk conditions: dangerous attempts	33	Active acquisition only	0.517	0.119
Exp. 3	High-rarity conditions: gap reduction	31	Active acquisition only	59.4%	70.7%
Exp. 3	High-rarity conditions: real interactions	31	Active acquisition only	4.29	2.16
Exp. 3	High-rarity conditions: policy success	31	Active acquisition only	72.3%	80.7%

The long-delay subset demonstrates that fixed observation windows cannot reliably capture outcomes whose appearance time varies. The corrupted-modality and high-distractor subsets demonstrate that selecting only the largest sensor changes is insufficient in complex environments. The expected-but-absent subset shows that non-occurrence can itself be an informative outcome. Finally, the high-risk and high-rarity subsets demonstrate that physical acquisition and material supplementation provide complementary benefits.

Appendix A.6. Paired Bootstrap Confidence Intervals

Table A6 reports the paired differences and 95% bootstrap confidence intervals cited in the main text. A positive difference indicates that TG-MAC has a larger metric value than the compared method. For metrics where lower is better, such as cost, real interactions, and erroneous-material acceptance, a negative difference favors TG-MAC.

Table A6. Paired bootstrap confidence intervals for the principal comparisons. The intervals are calculated over paired experimental rounds using 20,000 bootstrap resamples.

Experiment	Compared method	Metric	TG-MAC difference	95% confidence interval
Exp. 1	TG-MAC w/o thinking trigger	Association accuracy	$+11.00$ percentage points	$[+5.00,+18.00]$
Exp. 1	TG-MAC w/o thinking trigger	Post-learning success	$+4.36$ percentage points	$[+2.58,+6.16]$
Exp. 2	Change-only selection	Factor F1	$+4.94$ percentage points	$[+2.47,+7.45]$
Exp. 2	Change-only selection	Policy success	$+9.69$ percentage points	$[+8.32,+11.12]$
Exp. 2	Change-only selection	Processing cost	$-4.25$	$[-5.02,-3.48]$
Exp. 3	Active acquisition only	Material-gap reduction	$+9.34$ percentage points	$[+8.56,+10.11]$
Exp. 3	Active acquisition only	Policy success	$+7.41$ percentage points	$[+6.89,+7.91]$
Exp. 3	Active acquisition only	Real interactions	$-2.22$	$[-2.44,-1.99]$
Exp. 3	TG-MAC w/o validation	Error acceptance	$-14.81$ percentage points	$[-14.98,-14.64]$
Exp. 3	TG-MAC w/o validation	Policy success	$+8.50$ percentage points	$[+8.08,+8.93]$

Table A6. Paired bootstrap confidence intervals for the principal comparisons. The intervals are calculated over paired experimental rounds using 20,000 bootstrap resamples.

Experiment	Compared method	Metric	TG-MAC difference	95% confidence interval
Exp. 1	TG-MAC w/o thinking trigger	Association accuracy	$+11.00$ percentage points	$[+5.00,+18.00]$
Exp. 1	TG-MAC w/o thinking trigger	Post-learning success	$+4.36$ percentage points	$[+2.58,+6.16]$
Exp. 2	Change-only selection	Factor F1	$+4.94$ percentage points	$[+2.47,+7.45]$
Exp. 2	Change-only selection	Policy success	$+9.69$ percentage points	$[+8.32,+11.12]$
Exp. 2	Change-only selection	Processing cost	$-4.25$	$[-5.02,-3.48]$
Exp. 3	Active acquisition only	Material-gap reduction	$+9.34$ percentage points	$[+8.56,+10.11]$
Exp. 3	Active acquisition only	Policy success	$+7.41$ percentage points	$[+6.89,+7.91]$
Exp. 3	Active acquisition only	Real interactions	$-2.22$	$[-2.44,-1.99]$
Exp. 3	TG-MAC w/o validation	Error acceptance	$-14.81$ percentage points	$[-14.98,-14.64]$
Exp. 3	TG-MAC w/o validation	Policy success	$+8.50$ percentage points	$[+8.08,+8.93]$

The confidence intervals for the principal comparisons do not cross zero. Within the current controlled simulation, this indicates that the observed advantages are consistent across the paired test rounds rather than being caused by a small number of isolated scenarios.

Appendix A.7. Interpretation and Reporting Scope

The complete results reveal that TG-MAC does not maximize every individual metric. The best-single-sensor method obtains the highest factor precision and the lowest processing cost in Experiment 2. All-sensor concatenation obtains the highest raw recall and outcome completeness. Supplementation-only learning requires no physical interaction and achieves the highest safety score and lowest total cost in Experiment 3. TG-MAC without validation obtains the largest apparent material-gap reduction.

TG-MAC instead provides a stronger joint balance among temporal correctness, multimodal relevance, downstream policy performance, physical risk, and material-construction cost. The results therefore support the proposed mechanisms without implying that TG-MAC dominates every baseline on every individual metric.

The current experiments constitute a controlled mechanism-level verification. The policy-success values are common downstream performance estimates computed under the same evaluation procedure for all methods. They should not be interpreted as physical-robot success rates obtained after training and deploying a complete control policy. Complete per-round records can be retained with the supplementary material to support further statistical analysis and independent reproduction.