Progressive Importance-Guided Observation with Memory for Resource-Constrained Autonomous Robots

1. Introduction

Autonomous robots are expected to operate for long periods in open and changing environments [1,2], yet their sensing and reasoning resources are inherently limited. Continuous high-rate observation of every visible object or region is often unnecessary, while insufficient observation can cause the robot to miss newly important events, delayed changes, or previously inconspicuous information. This makes observation itself a decision problem: a robot should determine not only what it currently sees, but also what deserves detailed processing first, what can temporarily receive less attention, when broader coverage is necessary, and when observation frequency should increase again because the environment has changed. An effective observation strategy should therefore balance immediate importance, long-term coverage, environmental change, and the cost of repeatedly processing information that is already understood.

However, existing solutions usually address only part of this problem. Active perception and next-best-view methods [3,4] are effective for selecting informative viewpoints and improving geometric coverage, but they are often driven mainly by information gain or unexplored space and do not explicitly control whether already processed information should continue to consume detailed processing capacity. Exploration and semantic-navigation methods use spatial memory and semantic relevance to guide where the robot should search, but they are commonly tied to a specific navigation goal or assume relatively frequent observation. Persistent mapping and semantic memory reduce the need to rediscover the environment, yet memory alone does not determine whether a known item should be suppressed, whether a familiar object should regain priority after its relation changes, or whether a visual difference is caused by the environment or merely by a different viewpoint. As a result, a long-term robot may either waste resources by repeatedly observing stable low-impact information or fail to revisit initially low-priority information and newly changed situations at the right time.

This paper proposes Progressive Importance-Guided Observation (PIGO), a resource-aware observation framework for autonomous robots. PIGO treats detailed observation as a limited budget and allocates this budget according to multiple complementary factors, including importance, uncertainty, novelty, and coverage demand. The framework first protects information with immediate potential impact, then progressively shifts observation capacity toward candidates that have received insufficient attention. In this way, important information can be processed early without permanently starving less salient information. PIGO also adapts observation intervals according to environmental familiarity and change: stable situations can be observed less frequently, whereas meaningful changes restore faster and broader observation.

To support long-term operation, PIGO further introduces viewpoint-conditioned observation memory and processed-information suppression. Previously observed information is compared only with compatible viewpoints, reducing false environmental-change judgments caused by robot or camera motion. Memory mismatch can raise novelty and reactivate previously deprioritized information, while stable, already processed, low-impact information can be suppressed so that it does not repeatedly consume detailed-processing capacity. Suppression is not permanent: it is released when appearance, relation, novelty, or estimated importance changes. The framework therefore uses memory not only as a repository of past observations, but also as a control signal for future observation allocation. The resulting mechanism integrates importance-first observation, progressive coverage, adaptive observation frequency, memory-guided change detection, and reversible suppression within one observation-scheduling process.

The main contributions of this paper are as follows:

• We formulate autonomous robot observation as a limited-resource scheduling problem and propose a progressive allocation mechanism that prioritizes immediately important information while gradually extending coverage toward initially neglected candidates.
• We introduce an adaptive observation strategy in which stable familiar environments can be sampled less frequently, while meaningful environmental changes trigger renewed observation and processing.
• We develop viewpoint-conditioned observation memory together with reversible suppression of stable processed information, allowing the robot to distinguish viewpoint variation from environmental change, reduce repeated benign processing, and restore priority when a known object or relation changes.

The remainder of this paper is organized as follows. Section 2 reviews active perception, exploration and resource-aware observation, and spatial or semantic memory for autonomous robots. The following section presents the proposed PIGO model, including progressive scheduling, adaptive observation frequency, and viewpoint-conditioned memory. Section 4 describes the controlled simulation and real-robot verification and analyzes the experimental results. Finally, the last section concludes the paper and discusses the main limitations and future directions.

2. Related Work

2.1. Active Perception and Viewpoint Selection

Active perception treats sensing as a controlled process rather than passive data collection. Classical work argued that a robot should actively choose how to acquire information according to the current state of interpretation, so that sensing actions, camera configuration, and robot motion become part of the perception loop [5,6]. Later work revisiting active perception further emphasized that perception, attention, sensing, and control should be considered together rather than as independent modules [7]. The main advantage of this line of research is its closed-loop formulation: perception influences sensing actions, and newly acquired observations in turn modify the internal state.

A large body of robotic exploration work instantiates this idea through next-best-view (NBV) planning. Bircher et al., for example, select a branch of a receding-horizon search tree according to the amount of unexplored space expected to be revealed and repeatedly execute only the first edge before replanning [8]. Such methods are effective for geometric coverage, reconstruction, and exploration of unmapped space. Their limitation for long-term autonomous robots is that the value of a view is commonly dominated by geometric novelty or coverage. A mapped region may still require urgent attention because something important has changed, while a newly visible region may be low-impact. In addition, NBV methods usually decide where to look next, but do not explicitly decide whether already observed information should continue to consume detailed processing capacity.

The proposed method follows the active-perception principle but addresses a different allocation problem. PIGO separates the selection of information for detailed processing from the acquisition of a geometrically novel view. It first protects high-impact information, then progressively broadens observation toward neglected candidates, and can expand the viewpoint when the current view is insufficient. Previously processed stable information can also be suppressed and later reactivated when novelty, relation, or importance changes. Thus, compared with conventional NBV planning, the emphasis is not only on maximizing newly visible space but on deciding which information continues to deserve scarce observation and reasoning resources over time.

2.2. Exploration, Selective Attention, and Resource-Aware Observation

Learning-based exploration has improved the ability of agents to navigate efficiently in large unknown environments. Neural SLAM introduced an external spatial memory that encourages SLAM-like behavior inside a learned policy and showed that persistent memory can improve exploration when long-term spatial information is required [9]. Active Neural SLAM later combined a learned mapper and global policy with analytical planning and a local navigation policy [10]. Such modular designs are advantageous over purely reactive navigation because the agent can accumulate spatial knowledge and use that knowledge to select future exploration goals instead of repeatedly acting from the current image alone.

Semantic exploration further extends this idea by making exploration goal-dependent. Goal-Oriented Semantic Exploration builds an episodic semantic map and uses object-level semantic priors to choose where to search for a requested target [11]. These methods demonstrate that observation need not be uniform: semantic relevance can make one region more useful than another. However, their allocation objective is usually tied to a specific navigation goal, such as finding an object category or maximizing unexplored map coverage. They generally do not explicitly distinguish an initially important item from an item that becomes important later, nor do they model a visually salient object that has already been processed and can safely be deprioritized. Continuous or frequent sensing is also commonly assumed, so the question of when detailed observation can be reduced in a familiar stable environment is less central.

PIGO adopts the useful principle that observation should depend on accumulated knowledge, but makes the observation budget itself a first-class constraint. Importance, uncertainty, novelty, and coverage demand compete for a limited number of detailed-processing slots. This creates an explicit progression from urgent information toward information that has accumulated coverage debt, instead of assigning a fixed fraction of capacity to exploration or applying a single navigation objective. Furthermore, the observation interval can be lengthened in stable familiar conditions and shortened again after significant change. This distinction is important for long-horizon autonomous operation: a robot should neither observe every part of the environment continuously nor allow initially low-priority information to remain unobserved indefinitely.

2.3. Spatial Memory, Semantic Maps, and Change-Aware Observation

Persistent spatial and semantic memories provide another foundation for long-horizon robot perception. Neural SLAM represents the environment in external memory [9], while Semantic MapNet accumulates egocentric RGB-D features into an allocentric spatial memory tensor that supports later semantic mapping and downstream embodied tasks [12]. Goal-Oriented Semantic Exploration similarly maintains an episodic semantic map for object-goal navigation [11]. The main benefit of these approaches is that the agent does not need to rediscover the entire environment from each new frame; spatially organized memory supports long-term planning, target search, and reasoning over places that are temporarily outside the current field of view.

Nevertheless, a map or memory is not by itself an observation policy. Many mapping systems are designed to improve completeness or semantic accuracy of the stored representation, and repeated observations are fused whenever they are available. This is useful for reconstruction, but it does not directly answer three questions central to resource-constrained autonomy: whether unchanged information should continue to be processed in detail, whether a relation change involving an already known object should restore its priority, and whether a difference caused only by camera motion should be treated as an environmental change. In particular, memory that is not conditioned on viewpoint can make image-level comparison unreliable because two valid observations of the same environment may differ substantially only because of camera pose.

The proposed method therefore uses memory not only as storage but also as a control signal for future observation. Memory mismatch increases novelty and can release previously suppressed information, whereas stable, already processed, low-impact information can reduce its future priority. The memory is conditioned on compatible viewpoints so that changes caused by robot or camera motion are separated from changes in the environment. This creates a complementary role to semantic mapping: mapping answers what is known and where it was observed, while PIGO additionally decides whether that knowledge warrants renewed observation now. The resulting mechanism is designed for long-term autonomous robots in which observation, processing, and semantic reasoning are limited resources rather than continuously available operations.

3. Memory-Guided Progressive Observation Model

The proposed model treats observation as a persistent resource-allocation process rather than as continuous uniform sensing. Its objective is to determine what should be processed in detail, what can temporarily receive less attention, and when observation should become broader or more frequent. The robot therefore maintains both the current perceptual state and an observation memory of previously processed information. At each observation cycle, currently exposed candidates compete for a limited detailed-processing budget according to their estimated importance, uncertainty, memory-guided novelty, and accumulated observation demand. Information with immediate potential influence is handled first, while candidates that have received insufficient attention gradually gain priority through progressive coverage.

The result of detailed processing is fed back into subsequent observation. Stable, low-impact information that has already been sufficiently analyzed can be temporarily suppressed so that it does not repeatedly consume scarce processing capacity. In contrast, a change in appearance, task relation, uncertainty, or estimated importance releases this suppression and restores attention. Observation memory is conditioned on compatible viewpoints so that differences caused only by robot or camera motion are not automatically interpreted as environmental changes. Environmental familiarity also affects the observation interval: stable scenes may be sampled less frequently, whereas meaningful change triggers denser observation. Figure 1 summarizes this closed-loop process.

3.1. Problem Setting and Observation State

At observation cycle t, the robot receives an observation $o_t$ from its current viewpoint $v_t$. From this observation it constructs a set of currently exposed candidates

$${\mathcal{C}}_t={\left\{c_i^t\right\}}_{i=1}^{N_t}.$$

(1)

A candidate can represent an observed object, a local relation, or an unresolved currently observable region. Importantly, ${\mathcal{C}}_t$ contains only information that is available from the robot’s current sensing state. Objects outside the current field of view are not treated as known candidates merely because they exist in the environment.

Detailed processing is limited. The robot therefore selects

$${\mathcal{S}}_t\subseteq {\mathcal{C}}_t,\left|{\mathcal{S}}_t\right|\le K,$$

(2)

where K is the maximum number of candidates that can receive detailed processing in one observation cycle. This budget represents expensive perception, semantic interpretation, or task-related reasoning rather than the cost of simply receiving an image.

The robot also maintains a compact observation memory

$${\mathcal{M}}_t=\left\{m_j\right\},m_j=(v_j,{\varphi }_j,{\rho }_j,{\tau }_j,q_j,h_j),$$

(3)

where $v_j$ identifies the compatible viewpoint, ${\varphi }_j$ is a compact visual/semantic representation, ${\rho }_j$ stores task-relevant relations, ${\tau }_j$ is the most recent reliable observation time, $q_j$ is confidence, and $h_j$ records whether the information has already been sufficiently processed. The observation state used for scheduling is therefore not only the current image, but the combination of currently exposed evidence and relevant historical information.

3.2. Importance-Guided Progressive Scheduling

Each candidate is assigned an observation priority

$$P_i^t=\alpha I_i^t+\beta U_i^t+\gamma N_i^t+\delta C_i^t-\eta H_i^t,$$

(4)

where $I_i^t$ is estimated impact or task relevance, $U_i^t$ is uncertainty, $N_i^t$ is memory-guided novelty or change evidence, $C_i^t$ is accumulated observation demand, and $H_i^t$ is processed-information suppression. The nonnegative weights $\alpha ,\beta ,\gamma ,\delta ,\eta$ control their relative influence. The selected set ${\mathcal{S}}_t$ contains at most the K highest-priority candidates, subject to safety or feasibility constraints.

The role of $C_i^t$ is to prevent permanent starvation of initially low-priority information. When a candidate or observable region repeatedly receives no detailed attention, its accumulated observation demand increases. A simple update is

$$C_i^{t+1}=\left\{\begin{array}{cc}(1-{\lambda }_c)C_i^t,\hfill & c_i^t\in {\mathcal{S}}_t,\hfill \\ min(C_{max},C_i^t+{\mathsf{\Delta }}_c),\hfill & c_i^t\notin {\mathcal{S}}_t,\hfill \end{array}\right.$$

(5)

where ${\mathsf{\Delta }}_c>0$ is the debt increment and ${\lambda }_c\in (0,1]$ is the reduction applied after observation. Consequently, observation is progressive rather than purely greedy: high-impact information dominates early cycles, but information that has remained unprocessed gradually becomes competitive.

This mechanism differs from reserving a fixed number of slots for exploration. A fixed split can waste capacity when urgent information is abundant or can delay important observations when too much capacity is permanently assigned to coverage. In PIGO, the shift toward broader observation emerges from the evolving candidate priorities and accumulated observation demand. When the current field of view has been sufficiently processed, the same principle can request observation of another available viewpoint or region; the model does not assume knowledge of objects that have not yet been exposed.

Observation frequency is controlled separately from candidate priority. Let ${\mathsf{\Delta }}_t$ denote the interval before the next observation cycle. We use

$${\mathsf{\Delta }}_{t+1}=\mathrm{clip}\left({\mathsf{\Delta }}_t+{\kappa }_s{S}_t-{\kappa }_m{M}_t,{\mathsf{\Delta }}_{min},{\mathsf{\Delta }}_{max}\right),$$

(6)

where $S_t$ summarizes recent stability/familiarity and $M_t$ summarizes meaningful change or unresolved uncertainty. Stable familiar observations increase the interval, whereas novelty, major change, or unresolved information decreases it. Thus the robot can reduce redundant sensing without becoming permanently slow to respond.

3.3. Viewpoint-Conditioned Memory and Change Detection

Observation memory is indexed by compatible viewpoint rather than treated as one global visual history. For a current observation $x_t$ from viewpoint $v_t$, the robot first retrieves

$${\mathcal{M}}_t\left(v_t\right)=\{m_j\in {\mathcal{M}}_t:\mathrm{compat}(v_j,v_t)=1\},$$

(7)

where $\mathrm{compat}(·)$ denotes viewpoint compatibility. A viewpoint may encode the camera identity together with a coarse robot/camera pose. For example, observations from a wide front camera and a narrow arm camera are not directly compared as though they were the same visual field.

Memory-guided novelty for candidate i is computed from the mismatch between current evidence and compatible historical evidence,

$$N_i^t=D\left(\mathsf{\Phi }\left(c_i^t\right),\mathsf{\Phi }\left({\mathcal{M}}_t(v_t,i)\right)\right),$$

(8)

where $\mathsf{\Phi }(·)$ is a compact visual/semantic representation and $D(·)$ is a difference measure. The comparison may include appearance as well as task-relevant relations. Therefore, $N_i^t$ can increase when an object newly appears, when a known object changes its relation to the robot’s route, or when current evidence disagrees with the remembered state.

Removal is handled as a special memory mismatch. If a previously reliable item is expected in a compatible view but supporting current evidence disappears persistently, the memory entry itself becomes a change candidate. This allows the absence of an object to compete for processing even though no new object is detected in the current image.

Viewpoint conditioning is essential because image difference alone cannot distinguish environmental change from camera motion. A mismatch produced by switching to an incompatible viewpoint is therefore not treated as evidence of object appearance or removal. When the robot returns to a compatible viewpoint, the corresponding memory can again be used for change comparison. Cross-view association may be added when reliable geometric or semantic correspondence is available, but it is not required by the core model.

3.4. Processed-Information Suppression and Reactivation

Memory is also used to represent that some information has already been handled. After candidate i receives detailed processing, the robot determines whether it is sufficiently understood, stable, and currently low-impact. If so, its suppression state $H_i^t\in [0,1]$ is increased. A generic update is

$$H_i^{t+1}=\mathrm{clip}\left(H_i^t+{\lambda }_h{B}_i^t-{\lambda }_r{R}_i^t,0,1\right),$$

(9)

where $B_i^t$ indicates consistent benign evidence and $R_i^t$ represents reactivation evidence such as novelty, uncertainty growth, changed relation, or increased task importance. Suppression therefore reduces priority through the $-\eta H_i^t$ term in Eq. (4), but it never deletes the corresponding memory.

The distinction between suppression and forgetting is important. A stable chair outside the robot’s route, for example, may initially deserve detailed processing because its consequence is unknown. Once repeatedly observed and judged benign, it need not occupy the same detailed-processing slot in every subsequent cycle. Nevertheless, its memory is retained. If the chair moves, if the planned route changes, or if current evidence becomes inconsistent with the remembered state, $R_i^t$ increases, suppression decreases, and the candidate regains priority. Accordingly,

$$\mathrm{stable}+\mathrm{processed}+\mathrm{low}\mathrm{impact}\to \mathrm{temporary}\mathrm{suppression},$$

whereas

$$\mathrm{novelty}\mathrm{or}\mathrm{changed}\mathrm{relevance}\to \mathrm{reactivation}\mathrm{and}\mathrm{reprocessing}.$$

This feedback is what allows the observation scheduler to use past reasoning to shape future sensing rather than repeatedly interpreting the scene from scratch.

3.5. Persistent Closed-Loop Procedure

The complete process runs continuously in parallel with task execution. Each cycle acquires a fresh observation, retrieves compatible memory, builds current candidates, computes their priorities, performs bounded detailed processing, and then updates memory, suppression, coverage demand, and the future observation interval. Algorithm 1 summarizes the procedure.

Algorithm 1 Memory-Guided Progressive Observation

1: Initialize observation memory $\mathcal{M}$, observation demand, and interval $\mathsf{\Delta }$   2: while the autonomous system is active do   3:     Acquire observation $o_t$ from current viewpoint $v_t$   4:     Retrieve compatible memory ${\mathcal{M}}_t\left(v_t\right)$   5:     Construct currently exposed candidates ${\mathcal{C}}_t$   6:     for each candidate $i\in {\mathcal{C}}_t$ do   7:         Estimate $I_i^t$, $U_i^t$, $N_i^t$, $C_i^t$, and $H_i^t$   8:         Compute priority $P_i^t$ by Eq. (4)   9:     end for 10:     Select ${\mathcal{S}}_t\subseteq {\mathcal{C}}_t$ with $|{\mathcal{S}}_t|\le K$ 11:     Perform detailed perception/semantic processing for ${\mathcal{S}}_t$ 12:     Generate change candidates for persistent memory mismatches or removals 13:     Update task-relevant relations and memory confidence 14:     for each processed candidate i do 15:         if stable, sufficiently understood, and low-impact then 16:            Increase $H_i^t$ 17:         else 18:            Reduce suppression according to reactivation evidence 19:         end if 20:     end for 21:     Update observation demand using Eq. (5) 22:     Update viewpoint-conditioned memory ${\mathcal{M}}_{t+1}$ 23:     Adapt the next observation interval using Eq. (6) 24:     if current exposed information is sufficiently processed and unresolved coverage remains then 25:         Request observation of another feasible viewpoint or region 26:     end if 27:     Wait until the next scheduled observation cycle 28: end while

The resulting closed loop has four intended effects. First, high-impact or uncertain information receives early attention. Second, accumulated observation demand prevents initially low-priority information from being permanently ignored. Third, viewpoint-conditioned memory raises the priority of genuine novelty, relation change, and removal while avoiding false change caused only by viewpoint variation. Fourth, processed-information suppression prevents stable benign information from repeatedly consuming scarce detailed-processing capacity, while reactivation preserves responsiveness when that information changes. These properties are evaluated separately in Section 4.

4. Verification

We evaluate the proposed observation framework with two complementary types of experiments. Controlled simulation is used for conditions that are difficult to reproduce repeatedly on a physical robot, namely large candidate sets under strict observation budgets and long-horizon changes in observation frequency. Real-robot experiments then test whether the memory, suppression, removal, and viewpoint mechanisms remain effective with actual camera images, detector errors, perspective variation, and controlled physical changes in the environment. The simulation and real-robot experiments therefore verify different parts of the same observation process rather than duplicating one another.

4.1. Experimental Configuration and Metrics

4.1.1. Controlled simulation

Two controlled simulation scenarios are used. Each reported condition is repeated with 20 random seeds. In both scenarios, the simulated robot repeatedly receives candidate observations but can process only a limited subset in detail. The scheduler therefore has to decide which information should be examined first and which information can be deferred.

Simulation S1: progressive observation in a crowded scene. The robot operates in a synthetic workspace containing 96 simultaneously represented objects. Only $K=4$ candidates can receive detailed observation in each round. The scene contains two types of critical information. Some critical objects have high initial importance and should be processed early; other critical objects have deliberately low initial priority and compete with many ordinary objects for later attention. We call the latter hidden-critical objects because they are hidden from the scheduler’s initial priority ordering, not because they are physically occluded. The experiment varies the total observation budget over 15%, 25%, 35%, and 50% of the full candidate set. It is designed to test whether the robot can first protect urgent information and then progressively broaden observation so that initially neglected critical information is eventually reached.

Simulation S2: long-horizon observation in a changing environment. The robot continuously monitors a simulated environment over an extended sequence. The sequence begins with a stable period in which repeated high-frequency observation is largely redundant. A major environmental change is then introduced, followed by another stable period and, later, a smaller local critical change. The robot must decide both what to inspect and how frequently to observe. This construction tests whether observation can become sparse after the environment becomes familiar, while a meaningful change can immediately restore denser and broader observation. All compared methods receive the same underlying scene evolution; only their scheduling and observation-interval policies differ.

For Simulation S1, visible-critical recall is the fraction of initially high-priority critical objects detected within the available budget, whereas hidden-critical recall is the corresponding fraction for initially neglected critical objects. Detection latency is measured in observation rounds. For Simulation S2, major-critical latency is the number of rounds between introduction of the major change and its detection. Normalized observation cost is the accumulated normalized sensing and processing effort in the simulator, so lower values indicate less continuous observation. Paired comparisons use the same 20 seeds and two-sided Wilcoxon signed-rank tests.

4.1.2. Real-Robot Setup and Physical Scenarios

The physical experiments use Hiwonder’s MasterPi Robot as the mobile robotic platform, as shown in Figure 2. We additionally mounted a USB front camera at the front of the robot, which serves as the primary visual sensor in the experiments. The camera is positioned relatively low with respect to the environment, providing a near-ground viewpoint that introduces realistic perspective variation and partial occlusion. Images are acquired at a resolution of $640\times 480$ pixels. The front camera is kept fixed as the main sensing source so that observed scene changes are not confounded with switches between cameras having different fields of view. YOLO11l is used for ordinary object detection with a confidence threshold of $0.18$, while a compact color-region detector is used to identify the small colored blocks employed in several experimental scenarios. Same-view visual registration and image differencing provide complementary evidence for persistent environmental changes and object removal. Detailed semantic interpretation is applied only to candidates selected by the observation scheduler. Operator annotations and ground-truth event descriptions are used only for experimental scoring and are not provided as inputs to the observation policy or semantic reasoning process.

The real experiments are conducted in an indoor floor-level workspace containing a marked travel route and ordinary objects such as a chair, backpack, and book. Small colored blocks are used as newly introduced objects because they can be placed at precise locations without changing the remainder of the scene. Each trial is constructed as a short observation sequence. The robot first observes a stable reference state for one or more frames; one controlled change is then introduced while unrelated objects are kept fixed; subsequent frames determine whether the scheduler reallocates detailed attention to the changed information. Except for the dedicated viewpoint experiment, the camera pose is kept unchanged within a trial so that measured differences arise from the intended scene event rather than robot motion.

Five physical scenarios are used, each with a different purpose. M01 introduces a new small object into an otherwise unchanged scene. M02 changes the task relation of an already known object by moving a backpack from outside the planned route into the route. M03 repeatedly presents a large but route-irrelevant chair so that the robot can first process it as benign; a small route-relevant object is then introduced to test whether the resolved chair stops consuming detailed-processing capacity. M04 removes a previously observed book and therefore tests whether absence relative to memory can be detected. M05 deliberately changes the robot viewpoint from $P0$ to $P1$ and back to $P0$ before a real object is added, allowing viewpoint-induced image differences to be separated from genuine environmental changes.

Each scenario is independently collected in six physical layouts (R01–R06), giving 30 scene trials. The layouts vary object side, distance, background arrangement, and, where appropriate, the color or location of the introduced object while preserving the same mechanism being tested. All 30 trials pass the final capture-quality check (30 PASS, 0 WARN, 0 FAIL). The detailed-processing budget is $B\in \{1,2,3\}$, meaning that at most B candidates may receive detailed processing at an observation step. This explicitly creates competition between visually salient but already understood information and newly relevant information.

Table 1. Verification scenarios. “Initial state” describes what the robot first observes; “controlled change” identifies the event introduced while other unrelated conditions are kept unchanged.

Experiment	Initial state	Controlled change / robot activity	Mechanism tested
Simulation S1	A synthetic workspace contains 96 candidate objects, including immediately important objects and initially low-priority critical objects.	The robot repeatedly selects only $K=4$ candidates for detailed observation while the total observation budget is increased from 15% to 50%.	Importance-first processing followed by progressive coverage of neglected information.
Simulation S2	The simulated environment remains stable for an extended period.	A major change is introduced, the environment becomes stable again, and a later local critical change occurs while the robot adapts its observation interval.	Stable-period sparsification and change-triggered restoration of observation.
M01: new object	Chair, backpack, and book remain in a stable arrangement around the marked route.	After the reference observations, a small colored block is added while the original objects and camera pose remain unchanged.	Memory-guided novelty and allocation of scarce attention to a newly appearing object.
M02: relation change	A backpack is visible but located outside the planned route; the remaining scene is stable.	The same backpack is moved into the route without changing its identity or the positions of unrelated objects.	Reprioritization caused by a changed object–route relation rather than a new identity.
M03: processed suppression	A large chair remains conspicuous but outside the route and is observed repeatedly until it can be treated as stable and benign.	A small colored object is then placed in the route while the chair remains unchanged.	Suppression of resolved benign information and release of capacity for a later relevant event.
M04: removal	A book is visible and stable together with the other scene objects.	The book is physically removed while the camera and other objects are kept fixed.	Memory-based detection of disappearance/removal.
M05: viewpoint memory	The robot records a reference scene from viewpoint $P0$.	The robot rotates to $P1$, returns to the marked $P0$, and only then a colored block is added at $P0$.	Separation of viewpoint-induced appearance change from true environmental change.

Table 1. Verification scenarios. “Initial state” describes what the robot first observes; “controlled change” identifies the event introduced while other unrelated conditions are kept unchanged.

Experiment	Initial state	Controlled change / robot activity	Mechanism tested
Simulation S1	A synthetic workspace contains 96 candidate objects, including immediately important objects and initially low-priority critical objects.	The robot repeatedly selects only $K=4$ candidates for detailed observation while the total observation budget is increased from 15% to 50%.	Importance-first processing followed by progressive coverage of neglected information.
Simulation S2	The simulated environment remains stable for an extended period.	A major change is introduced, the environment becomes stable again, and a later local critical change occurs while the robot adapts its observation interval.	Stable-period sparsification and change-triggered restoration of observation.
M01: new object	Chair, backpack, and book remain in a stable arrangement around the marked route.	After the reference observations, a small colored block is added while the original objects and camera pose remain unchanged.	Memory-guided novelty and allocation of scarce attention to a newly appearing object.
M02: relation change	A backpack is visible but located outside the planned route; the remaining scene is stable.	The same backpack is moved into the route without changing its identity or the positions of unrelated objects.	Reprioritization caused by a changed object–route relation rather than a new identity.
M03: processed suppression	A large chair remains conspicuous but outside the route and is observed repeatedly until it can be treated as stable and benign.	A small colored object is then placed in the route while the chair remains unchanged.	Suppression of resolved benign information and release of capacity for a later relevant event.
M04: removal	A book is visible and stable together with the other scene objects.	The book is physically removed while the camera and other objects are kept fixed.	Memory-based detection of disappearance/removal.
M05: viewpoint memory	The robot records a reference scene from viewpoint $P0$.	The robot rotates to $P1$, returns to the marked $P0$, and only then a colored block is added at $P0$.	Separation of viewpoint-induced appearance change from true environmental change.

4.1.3. Compared Methods

In the progressive simulation, Importance-only repeatedly favors the currently highest-impact candidates without an explicit broadening stage; Uniform spreads observation uniformly; Random allocates observation stochastically; and Proposed first protects urgent observations and then progressively transfers capacity toward neglected candidates. The No Progressive Shift ablation removes this transfer, while Fixed Coverage-3 permanently reserves three of the four detailed slots for coverage.

In the dynamic simulation, Fixed-low keeps a sparse fixed interval, Random Interval varies the interval without change-sensitive control, and Always-high observes at the maximum rate. No Change Reallocation removes change-triggered restoration of observation, while No Adaptive Interval keeps the interval fixed rather than adapting it to environmental stability.

For the real-robot tests, No Memory uses only current-view evidence; Memory w/o Suppression retains memory and change handling but disables hard suppression of already processed benign information; and No View Conditioning uses a single global memory across viewpoints. Proposed combines viewpoint-conditioned observation memory, memory-guided novelty/change, and suppression of repeatedly processed stable low-impact information.

4.2. Controlled Simulation Results

4.2.1. Progressive Coverage Under a Limited Observation Budget

Simulation S1 asks the robot to allocate only four detailed-observation slots at each round among 96 candidates. The challenge is deliberately asymmetric: important objects should be found immediately, whereas low-priority critical objects should not be permanently starved simply because many higher-ranked candidates appear earlier. The budget sweep therefore reveals whether the proposed scheduling actually changes from importance-dominated observation to broader coverage as more observation becomes available.

Figure 3 shows this staged behavior. With only 15% of the full observation budget, Proposed already reaches $0.950$ visible-critical recall, while hidden-critical recall is $0.233$. Increasing the budget to 25%, 35%, and 50% raises hidden-critical recall to $0.433$, $0.700$, and $1.000$, respectively, while visible-critical recall reaches $1.000$ from 25% onward. Thus, the additional observation budget is not spent merely repeating the initially dominant candidates; it is progressively redirected toward previously neglected critical information.

At the representative 35% budget, Table 2 shows hidden-critical recall of $0.700$ for Proposed, compared with $0.317$ for Uniform, $0.417$ for Random, and $0.000$ for Importance-only. The paired difference is significant against Uniform ($p=0.000373$) and Importance-only ($p=0.000065$); the difference against Random is not claimed as significant. Removing the progressive shift reduces hidden-critical recall from $0.700$ to $0.000$ ($p=0.000065$) and increases hidden-critical latency from $5.633$ to $26.567$ rounds. Conversely, permanently reserving three of four slots for coverage raises hidden recall to $0.950$, but reduces first-two-round visible-critical recall from $0.750$ to $0.400$ ($p=0.000511$). These two opposite ablations show why a staged policy is needed: importance-only observation misses low-priority critical information, whereas permanent aggressive coverage delays information that is important immediately.

4.2.2. Adaptive observation in a changing environment

Simulation S2 represents a robot that monitors an environment for a long period rather than performing a one-time scan. During the initial stable phase, repeatedly observing at the highest frequency provides little new information and should therefore be avoided. When the major change occurs, however, a policy that has become sparse must quickly restore observation; otherwise the cost saving is obtained at the expense of a long detection delay. The later local change checks that the policy remains responsive after returning to a familiar state.

Table 3 and Figure 4 show the resulting latency–cost tradeoff. Proposed detects a major critical change after only $0.300$ rounds on average at a normalized cost of $195.4$. Fixed-low reduces cost to $108.0$ but increases latency to $14.8$ rounds, while Random Interval requires $19.817$ rounds at cost $173.2$. Always-high reaches zero-round latency, but its cost is 3840, approximately $19.7$ times the cost of Proposed. Hence, the useful operating point is not obtained by simply observing as frequently as possible.

The mechanism ablations clarify why. Removing change-sensitive reallocation raises major-critical latency from $0.300$ to $14.083$ rounds while cost remains similar ($195.4$ versus $191.0$); the latency difference is significant ($p=0.000087$). Disabling adaptive intervals increases normalized cost from $195.4$ to $261.6$ ($p=0.000072$). Together, these results show two distinct functions: adaptive intervals reduce redundant sensing during familiar periods, and change-triggered reallocation restores rapid attention when the environment ceases to be familiar.

4.3. Real-Robot Results

4.3.1. Memory-Guided New-Object Discovery and Relation Change

M01 is constructed to isolate novelty rather than motion or viewpoint variation. The robot first observes a stable scene containing the chair, backpack, and book from a fixed camera pose. After this reference has been stored, a small colored block is placed in the scene while all original objects remain unchanged. The test asks whether a new but relatively small item can obtain detailed processing when it must compete with larger familiar objects. At the tightest budget $B=1$, Proposed achieves target-attention recall $1.000$, whereas No Memory achieves $0.000$ ($p=0.03125$). As shown in Table 4, Proposed remains at $1.000$ for all three budgets, while No Memory reaches only $0.167$ at $B=2$ and $0.500$ at $B=3$. Thus, memory changes the meaning of the current image: already familiar objects lose relative novelty, allowing the newly appearing block to win a scarce detailed-processing slot.

M02 keeps object identity fixed and changes only task relevance. The backpack is first placed outside the marked travel route, where it can be observed and treated as non-interfering. It is then moved into the route while the camera, chair, book, and other scene elements remain fixed. Proposed obtains $1.000$ target-attention recall at every budget, compared with $0.333$ for No Memory. At $B=1$, the paired test gives $p=0.125$; therefore, the six-layout difference is reported as a consistent numerical advantage rather than a statistically significant one. The experiment nevertheless shows the intended relation-based mechanism: a familiar object can regain priority because its relation to the robot’s planned motion has changed, even though no new object identity appears.

Figure 5. Real M01: target-attention recall under different detailed-processing budgets. Memory is especially useful when only one candidate can be processed.

Figure 5. Real M01: target-attention recall under different detailed-processing budgets. Memory is especially useful when only one candidate can be processed.

4.3.2. Suppressing Already Processed Benign Information

M03 tests whether a robot can stop spending detailed processing on information whose meaning has already been resolved. A large chair is deliberately kept conspicuous in the camera image but positioned outside the travel route. The robot observes this unchanged configuration repeatedly so that the chair can first be processed and judged stable and benign. Only after this repeated stable phase is a small colored object placed inside the route. The desired behavior is not to ignore the chair from the beginning, but to reduce its priority after it has been processed, so that the later route-relevant object can use the released capacity.

Both Proposed and Memory w/o Suppression retain $1.000$ target-attention and change-recognition recall at every budget, so the reduction in repeated processing is not obtained by sacrificing detection of the later event. At $B=3$, the repeated-benign fraction falls from $0.2964$ without suppression to $0.0965$ with Proposed ($p=0.03125$), a relative reduction of 67.4%. The corresponding detailed-processing cost decreases from $13.83$ to $11.50$ ($p=0.03125$), a reduction of 16.9%. Figure 6 shows that the benefit becomes more visible as the nominal budget grows: extra processing capacity need not be consumed by a stable object whose consequence has already been resolved.

Figure 6. Real M03: repeated-benign processing with and without hard suppression.

Figure 6. Real M03: repeated-benign processing with and without hard suppression.

Figure 7. Real M05: viewpoint false-change rate. Global memory mistakes viewpoint transitions for scene changes, whereas viewpoint-conditioned memory does not.

Figure 7. Real M05: viewpoint false-change rate. Global memory mistakes viewpoint transitions for scene changes, whereas viewpoint-conditioned memory does not.

Table 5. Real M03 processed-information suppression over six layouts.

	Repeated-benign fraction		Detailed-processing cost
B	Proposed	No Supp.	Proposed	No Supp.
1	0.0000	0.1000	5.00	5.00
2	0.0352	0.1852	8.83	9.83
3	0.0965	0.2964	11.50	13.83

Table 5. Real M03 processed-information suppression over six layouts.

	Repeated-benign fraction		Detailed-processing cost
B	Proposed	No Supp.	Proposed	No Supp.
1	0.0000	0.1000	5.00	5.00
2	0.0352	0.1852	8.83	9.83
3	0.0965	0.2964	11.50	13.83

Table 6. Real M04–M05 results over six layouts.

B	M04 removal recall	M05 false-change rate		M05 target recall
	Proposed	Proposed	No View Cond.	Proposed
1	0.667	0.000	1.000	0.667
2	1.000	0.000	1.000	1.000
3	1.000	0.000	1.000	1.000

Table 6. Real M04–M05 results over six layouts.

B	M04 removal recall	M05 false-change rate		M05 target recall
	Proposed	Proposed	No View Cond.	Proposed
1	0.667	0.000	1.000	0.667
2	1.000	0.000	1.000	1.000
3	1.000	0.000	1.000	1.000

4.3.3. Removal and Viewpoint-Conditioned Memory

M04 tests a change that is represented by absence rather than appearance. The robot first observes a book as a stable part of the scene from a fixed viewpoint. After the reference state has been established, the book is removed without moving the other objects or the camera. This construction requires the method to compare the current observation with memory and infer that previously present information is no longer there. The proposed method obtains target-attention recall of $0.667$, $1.000$, and $1.000$ for $B=1,2,3$, respectively. The single-slot budget misses two of the six layouts, whereas two or more detailed-processing slots recover all six removal events. This non-perfect $B=1$ result is retained because disappearance is based on absence relative to memory and is more sensitive to limited current evidence than the appearance of a new object.

M05 is designed specifically to prevent ordinary viewpoint change from being counted as environmental change. The robot first records the scene at a marked reference pose $P0$, rotates to a different viewpoint $P1$, and then returns to the same marked $P0$. No object is added or removed during this $P0\to P1\to P0$ sequence. Only after the robot has returned to $P0$ is a new colored block introduced. Consequently, a correct method should ignore the viewpoint-only transitions but react to the subsequent same-view scene change. No View Conditioning produces a viewpoint false-change rate of $1.000$ for all three budgets, whereas Proposed produces $0.000$ for all three; the paired p value is $0.03125$ at each tested budget. Proposed still obtains target-attention recall of $0.667$, $1.000$, and $1.000$ for the subsequent real same-view change. Thus, selecting historical observations by compatible viewpoint prevents robot motion from masquerading as scene change without suppressing a later genuine environmental event.

4.4. Summary and Limitations

The selected simulation experiments establish two properties that are difficult to demonstrate with only a few physical objects. Progressive observation preserves rapid attention to important information while gradually reaching initially neglected critical candidates in a 96-object scene, and adaptive observation avoids the extreme cost of continuous sensing while retaining near-immediate response to major change. The real-robot experiments then show that memory-guided novelty, relation-sensitive reprioritization, processed-information suppression, removal reasoning, and viewpoint-conditioned memory remain useful under actual visual sensing.

The results also identify two limits rather than hiding them. First, the M02 six-layout advantage ($1.000$ versus $0.333$) does not reach $p<0.05$. Second, under the most restrictive $B=1$ physical budget, M04 removal and the true same-view change in M05 reach $0.667$ recall rather than $1.000$. These cases indicate that observation memory improves allocation of limited processing but cannot replace sufficient current perceptual evidence. With $B\ge 2$, both physical mechanisms reach full recall in the six tested layouts.

5. Conclusion

This paper proposed Progressive Importance-Guided Observation (PIGO) for autonomous robots operating under limited observation and processing resources. The method combines importance-first scheduling, progressive coverage, adaptive observation frequency, viewpoint-conditioned memory, and reversible suppression of stable processed information. Controlled simulations showed that PIGO can preserve early attention to important information while gradually extending coverage and can respond quickly to environmental changes without the cost of continuous high-rate observation. Real-robot experiments further demonstrated effective memory-guided novelty detection, relation-change handling, suppression of repeated benign processing, removal detection, and separation of viewpoint changes from true environmental changes. These results indicate that observation memory can serve not only as stored history but also as an active control signal for deciding what should be observed again, what can temporarily receive less attention, and when observation should intensify. Future work will consider richer environments, longer physical deployments, more complex viewpoint changes, and tighter integration with learned semantic perception.