Pricing the Context Window: A Unified Write–Read Theory for Token-Efficient Agent Memory

1. Introduction

An LLM agent is, architecturally, a harness around a function with a bounded input. The harness decides what the model sees, and the model can only reason over what lands in its context window. Even with windows of one million tokens, filling them indiscriminately is neither cheap nor effective: models recall less reliably as input grows [14], and position effects make the marginal value of a token depend on where it sits [13]. Practitioners summarize the goal as finding the smallest possible set of high-signal tokens that makes the desired outcome most likely [1]. This is a principle, not a theory.

Consider an agent whose persistent memory is a knowledge graph with attached evidence. When a query Q arrives, three questions arise. What is the optimal strategy to retrieve from the graph? What is the optimal strategy to build the graph? And what structure should the graph have so that retrieval is cheap? These are usually treated as separate engineering problems.

Thesis. Agent memory is an investment problem. Writing is an upfront expense that changes the future cost of access: a fact that is stored may save a later derivation, a summary makes later reads cheaper but lossy, a shortcut makes a recurring pattern cheap, and a fact that is stored adds branching that makes every later traversal more expensive. Conversely, the distribution of future queries determines which of these investments pay off. Write policy and read policy therefore cannot be optimized separately; they are two halves of one optimization over future information value.

Contributions. (1) We formulate write and read as a single bilevel problem in one numeraire, with a shadow price $\lambda$ on context tokens (Section 3). (2) On the read side we derive local optimality conditions, a corrected statement of why the useful context size is bounded independently of the window, a depth law, and a value-of-information stopping rule with an explicit planner (Sections 5.1–5.3). (3) On the write side we derive an admission criterion measured against query-time re-derivation, quantify the externality a write imposes on retrieval depth, and prove a lower bound on the information content of a predictively sufficient memory (Sections 5.4–5.5). (4) For structure we derive rules for resolution levels, lazily built summaries, shortcuts, a cached core, refresh, and indexing versus lazy construction (Sections 5.6–5.8). (5) We state eight testable hypotheses (Section 6) and the limits of the theory (Section 7).

Status of claims. We separate three kinds of statements. Propositions are mathematical results that hold under the stated assumptions. Design rules are operational consequences of propositions, with the assumptions they inherit. Hypotheses are empirical predictions that have not been tested.

2. Related Work

Context engineering and token economics. Anthropic’s guidance on context engineering names the objective but offers no formalism [1]. A recent survey frames agent inference as cost minimization subject to a quality threshold, with shadow prices for tokens, and treats the context window as a rival resource [2]. It notes in a footnote that tokens differ in utility and information density, but its production function treats tokens as a homogeneous input. Our theory begins where that heterogeneity matters.

Agent memory systems. Temporal knowledge graphs [8], virtual-memory paging [9], extraction-and-consolidation pipelines [10], dynamically organized notes [27], hippocampus-inspired graph retrieval with personalized PageRank [28], and graph-indexed retrieval with multi-level search [29,7] are write-side or structure-side designs governed by heuristics. Hierarchical summary trees [30] build resolution levels. LongMemEval [31] evaluates extraction, multi-session and temporal reasoning, knowledge updates and abstention, but not cost. Notably, single-step graph retrieval has been reported to be 10–20 times cheaper than iterative LLM-driven retrieval [28], which already hints that the cost of retrieval lies in LLM calls, not in graph operations (Section 5.2).

Budgeted selection and learned retrieval. Greedy evidence selection by information gain per token [3], budgeted context construction with diversity constraints [4], prompt compression [32], and learned or adaptive retrieval policies [33,34] are read-side methods over flat chunks or fixed indexes.

Graph depth. Empirical studies report that retrieval performance peaks near depth three [5] and flow-based pruning applies per-hop decay with early stopping [6]. These are tuned heuristics without a dependence of the optimum on price, branching or scorer quality, and without a link to how the graph was built.

Classical theory. Table 1 lists the ingredients we draw on and what each leaves open.

Gap and claim. We claim no novelty for any single ingredient. The contribution is the coupling, and four things the synthesis yields that the ingredients do not: a price-dependent depth law (Eq. 3); a write criterion measured against query-time re-derivation, with an explicit externality of writes on retrieval depth (Eqs. 4, 6); a lower bound tying memory information content to predictive structure (Prop. 6); and one numeraire that prices every admission, placement, planning and refresh decision.

3. Problem Statement

3.1. The Scarcity

Context tokens are scarce in three senses. First, they have a price: money and latency per token. Second, their value is non-monotone: irrelevant tokens can reduce answer quality [13,14], so marginal value can be negative. Third, prices are tiered: tokens in a cached prefix cost a fraction $q<1$ of fresh tokens. A secondary scarcity is the write budget: tokens spent extracting, summarizing and maintaining the graph. Operations inside the graph store (joins, traversals, vector lookups) are a third, usually minor, cost, treated in Section 5.2.

3.2. Model

Definition 1 (Knowledge graph with evidence). A graph G stores a set of items $\mathcal{I}$ (nodes, edges, evidence passages). Each item i is available at resolutions $\ell \in \{1,\dots ,L\}$ (e.g., gist, summary, full text, full text with detailed evidence) with token costs ${\tau }_{i1}\le \dots \le {\tau }_{iL}$.

Definition 2 (Context and value). For a query $Q\sim P\left(Q\right)$ with correct decision variable A, a context C is a set of (item, resolution) pairs with total token cost $\tau \left(C\right)$. Its value is $V(Q,C)=R(Q,C)-\rho \left(C\right)$: a relevant gain R (for log-loss utility, $R=I(A;C\mid Q)$) minus a distraction penalty $\rho \left(C\right)\ge 0$ that increases with the number of irrelevant tokens.

Definition 3 (Prices and numeraire). We express all costs in one utility numeraire. $\lambda$ is the price of one fresh read token, $q\lambda$ of a cached prefix token, and ${\kappa }_w\lambda$ of one write token (writes may use cheaper or batched models, so ${\kappa }_w\ne 1$ in general). Treating these as proportional is a modeling assumption, not a fact; the results below hold for any fixed ratios. At an optimum, $\lambda$ equals the marginal value of a fresh read token.

Definition 4 (Policies). A write policy ${\pi }_w$ maps an observation stream $X_{1:T}$ to a graph G (including resolutions, shortcuts and a cached core). A read policy ${\pi }_r$ maps $(Q,G)$ to a context C.

Objective. Let N be the number of queries the graph will serve and $T_w\left(G\right)$ the tokens spent writing and maintaining it. The joint problem is

$$\underset{{\pi }_w,{\pi }_r}{\max }J({\pi }_w,{\pi }_r)={\mathbb{E}}_Q\left[V(Q,C)-\lambda \tau \left(C\right)\right]-{\displaystyle \frac{{\kappa }_w\lambda }{N}}{T}_w\left(G\right)$$

(1)

Because ${\pi }_w$ shapes the graph that ${\pi }_r$ sees, this is a bilevel problem: ${\pi }_w^{*}=\arg {\max }_{{\pi }_w}J({\pi }_w,{\pi }_r^{*}\left({\pi }_w\right))$.

Assumptions. (A1) Queries are drawn i.i.d. from $P\left(Q\right)$ over the graph’s lifetime. (A2) R is monotone submodular and additive across traversal shells; $\rho$ is separable and increasing. (A3) Uniform contraction: the decision-relevant value available in the shell at distance h from the query’s entry points is at most $g_0{\rho }_r^h$ with ${\rho }_r<1$ (Section 5.2). (A4) Predictive sufficiency: for results involving predictive information, the world is stationary and the stored graph is a sufficient statistic of the observation stream for the query-relevant future (Section 5.5).

4. The Pipeline

The theory addresses six stages of one pipeline (Figure 1).

5. Theory

5.1. Read: Selection

Proposition 1 (marginal-value conditions). Let $F\left(C\right)=V(Q,C)-\lambda \tau \left(C\right)$ and $\mathsf{\Delta }(i\mid C)=V(C\cup i)-V\left(C\right)$. (a) If V is additive over items with item values $v_i$, the optimal context is $C^{*}=\{i:v_i\ge \lambda {\tau }_i\}$. (b) In general, any local maximum C of F under single additions and removals satisfies

$${\displaystyle \frac{\mathsf{\Delta }(i\mid C)}{{\tau }_i}}\le \lambda \forall i\notin C,{\displaystyle \frac{V\left(C\right)-V(C\setminus i)}{{\tau }_i}}\ge \lambda \forall i\in C\left(2\right)$$

Proof. (a) maximizes ${\sum }_i(v_i-\lambda {\tau }_i)$ term by term. (b) is the condition that no single addition or removal improves F. □

Part (b) is a necessary condition for local optimality, not a characterization of the global optimum, and it does not say that included items earn exactly $\lambda$. It does justify a best-first read policy: repeatedly add the item maximizing $\mathsf{\Delta }(i\mid C)/{\tau }_i$ and stop when the ratio falls below $\lambda$. This is the marginal value theorem of optimal foraging [20]: leave a patch when the marginal rate of gain drops below the average rate.

Remark (approximation under a hard budget). If R is monotone submodular and the context is limited to a budget B, ratio-greedy combined with the best single item achieves a $\frac{1}{2}(1-1/e)$ approximation [16], and partial enumeration achieves $(1-1/e)$ [17,15]. The distraction penalty makes V non-monotone, so these guarantees apply to the relevant-gain part only.

Proposition 2 (useful context size is bounded independently of the window). Let $C^{\circ }\in \arg {\max }_{C}V(Q,C)$ and $B_d=\tau \left(C^{\circ }\right)$. Let $G\left(B\right)={\max }_{\tau \left(C\right)\le B}V(Q,C)$. (i) G is non-decreasing and constant for $B\ge B_d$. (ii) Every maximizer $C^{*}$ of F satisfies $\tau \left(C^{*}\right)\le \tau \left(C^{\circ }\right)$. (iii) Let $A_{\mathrm{fill}}\left(B\right)$ be the value of a policy that fills a budget B in decreasing order of relevant gain, ignoring distraction. If marginal relevant gain diminishes while marginal distraction cost is non-decreasing, then $A_{\mathrm{fill}}$ rises and then falls.

Proof. (i) The feasible sets are nested. (ii) From $F\left(C^{*}\right)\ge F\left(C^{\circ }\right)$ we get $\lambda (\tau \left(C^{\circ }\right)-\tau \left(C^{*}\right))\ge V\left(C^{\circ }\right)-V\left(C^{*}\right)\ge 0$. (iii) The derivative of $A_{\mathrm{fill}}$ is marginal gain minus marginal distraction cost, which changes sign once. □

The optimal policy’s value never decreases with the window size W (it merely saturates), but a policy that fills the window does degrade. The practical content is that the optimal context has a finite size $\tau \left(C^{*}\right)\le B_d$ however large W is. Enlarging W beyond $B_d$ cannot help an optimal policy and can only hurt a filling one.

5.2. Read: Depth

Traversal depth is where the three-hop folklore lives. We derive its dependence on the economics.

Lemma 1 (signal decay; sufficient condition for A3). Suppose evidence about Q reaches node $n_h$ along a Markov chain $Q\to n_1\to \dots \to n_h$ whose hop channels have strong-data-processing contraction coefficients at most ${\rho }_r<1$ [25]. Then $I(Q;n_h)\le {\rho }_r^{h}I(Q;n_0)$.

The data processing inequality alone gives only monotone decay [23]; geometric decay needs a uniform contraction coefficient, and graph traversal is not automatically such a chain (multiple paths between nodes can restore information). (A3) is therefore an assumption about the graph’s semantics, for instance chains of relations that lose specificity at each step, and ${\rho }_r$ is best treated as an effective parameter to be estimated.

Let the shell at distance h contain $b^h$ candidates of average token cost $\overline{\tau }$. Loading it without discrimination costs $b^h\overline{\tau }$ and yields value $g_h=g_0{\rho }_r^h$.

Proposition 3 (depth law). Under (A2) and (A3), shell h is worth loading iff $g_h/\left(b^h\overline{\tau }\right)\ge \lambda$, giving the optimal depth

$$h^{*}=\max \left(0,{\displaystyle \frac{\ln \left(g_0/\left(\lambda \overline{\tau }\right)\right)}{\ln \left(b/{\rho }_r\right)}}\right)\left(3\right)$$

Proof. Setting $g_0{\rho }_r^h/\left(b^h\overline{\tau }\right)=\lambda$ gives ${({\rho }_r/b)}^h=\lambda \overline{\tau }/g_0$. Since $b/{\rho }_r>1$, the ratio decreases in h, so all shells up to $h^{*}$ qualify. □

Example. With $g_0/\left(\lambda \overline{\tau }\right)=500$, $b=5$, ${\rho }_r=0.8$: $h^{*}=\ln 500/\ln 6.25\approx 3.4$. A depth-three optimum [5] is one point on a curve, not a constant.

Corollaries. (a) Cheaper tokens (smaller $\lambda$) deepen the optimum, logarithmically. (b) Higher branching b reduces it. (c) A scorer that discriminates within a shell replaces b by an effective branching $b_{\mathrm{eff}}\in [1,b]$, so better rerankers deepen the optimum. (d) Materializing a recurring k-hop pattern as one edge (Section 5.6) removes $k-1$ hops from the effective depth. (e) Branching externality. Differentiating (3),

$${\displaystyle \frac{\partial h^{*}}{\partial \ln b}}=-{\displaystyle \frac{h^{*}}{\ln (b/{\rho }_r)}}\left(4\right)$$

In the example, a 10% rise in average branching costs about $0.19$ hops. Every write that adds edges taxes all later traversals; Section 5.4 uses this.

Remark (non-token costs). Let $\kappa$ be the cost, in the units of $\lambda \overline{\tau }$, of examining one node: a database lookup, a join step, an embedding comparison. With best-first expansion each loaded node entails at most about b examinations, so (3) holds with $\lambda \overline{\tau }$ replaced by $\lambda \overline{\tau }+b\kappa$. Non-token costs are negligible iff $b\kappa \ll \lambda \overline{\tau }$.

Two orders of magnitude settle which case applies. A node of a hundred tokens at a few dollars per million input tokens costs a few hundredths of a cent, whereas a lookup or join step inside a database costs microseconds, many orders of magnitude less. Examination is not negligible when each examination is an LLM call (agentic scoring), because then $\kappa \approx \lambda ·$(tokens of that call). The same holds for latency: a hop executed inside the database is nearly free, while a hop that needs an LLM decision costs a sequential round trip of seconds. The consistency of this picture with the reported 10–20 times cost gap between single-step graph retrieval and iterative LLM-driven retrieval [28] is suggestive but not a test.

Design rule D1. Execute graph hops inside the store and reserve LLM calls for decisions. The choice of store (relational with recursive queries, or a native graph database) matters through $\kappa$, latency tails at large fan-out, and the ability to express shortcuts, multi-resolution reads and temporal filters; it does not change the token economics.

5.3. Read: The Epistemic Layer and Planning

Relevance is not the right criterion; decision-sensitivity is. A cheap planner drafts the decision and its alternatives and identifies which unknowns the decision depends on.

Proposition 4 (myopic value-of-information stopping). Define

$${\mathrm{VOI}}_i\left(C\right)={\mathbb{E}}_{X_i}\left[\underset{a}{\max }\mathbb{E}[U\mid C,X_i]\right]-\underset{a}{\max }\mathbb{E}[U\mid C]\left(5\right)$$

Fetching item i is single-step optimal iff ${\mathrm{VOI}}_i\left(C\right)/{\tau }_i>\lambda$, and myopic retrieval stops when no item satisfies this [18].

Value of information is classical; the point here is to use it as the retrieval criterion inside a persistent-memory system and to price it with the same $\lambda$ that governs storage. Two cautions apply. First, the rule is myopic: VOI is not submodular, and complementary facts (a key and its lock) can each have zero VOI and a positive joint VOI, so myopic stopping can stop too early. Second, a known absence (“we checked; nothing exists”) has high VOI per token, so the graph should store confidence and absences explicitly.

Dependency analysis. Complementarity is exactly what dependency analysis finds. The planner builds a dependency graph of the query: which facts the decision depends on, and which fact identities depend on earlier fetched values. This is a backward slice from the decision [35], related to data provenance [36] and to query planning in databases [37]. Its structure determines the retrieval schedule. Independent branches can be fetched in parallel and evaluated as bundles (curing myopia). A chain of dependent lookups forces sequential rounds, so the dependency depth of the query, not only the traversal depth of the graph, sets the number of round trips. Shortcuts (Section 5.6) shorten graph paths; cached plan templates shorten dependency chains.

The same structure appears in epistemic foraging, where agents choose actions by expected information gain [38,39]. Decision-sensitivity differs from generic information gain in that information that cannot change the action has value zero.

Design rule D2 (plan only when it pays). Running the planner costs $\lambda {\tau }_{\mathrm{plan}}$ and is worthwhile iff its expected saving in retrieval cost and decision loss exceeds that cost. For recurring query types the plan is a reusable asset: caching a plan template for pattern r pays off under the same threshold as a shortcut (Proposition 8), with $t_b$ the cost of producing it.

5.4. Write: Admission

Retrieval is query-conditioned; writing is not. The write policy must decide, before the query is known, what to keep.

Proposition 5 (admission criterion). A candidate fact f should be written iff its expected lifetime read value, relative to the best query-time alternative, exceeds its write and maintenance cost:

$$N·{\mathbb{E}}_Q\left[J_{\mathrm{read}}(G\cup f)-J_{\mathrm{read}}\left(G\right)\right]>{\kappa }_w\lambda \left(t_w\left(f\right)+t_m\left(f\right)\right)\left(6\right)$$

where $J_{\mathrm{read}}\left(G\right)$ is computed with the best alternative available when f is absent (re-deriving it, calling a tool, asking the user), and $t_w$, $t_m$ are the write and lifetime maintenance tokens.

By itself (6) is close to a tautology: expected benefit must exceed cost. Its content lies in how $\mathsf{\Delta }{J}_{\mathrm{read}}$ is computed, which gives three operational consequences.

Design rule D3 (write less). (a) Re-derivation. A fact that is cheap to re-derive at query time is not worth storing, however often it is asked. A fact that is expensive or impossible to re-derive, or whose loss is costly (high stakes, rarely queried), is worth storing even when rare. (b) Branching externality. A fact that adds $\delta$ to the average branching changes read value by roughly $\mathbb{E}[\partial J_{\mathrm{read}}/\partial h^{*}]·(-h^{*}/\ln (b/{\rho }_r))·\delta /b$ through Equation (4), so (6) can be violated by a fact that is individually useful but raises branching. (c) Redundancy. If $I(f;Y\mid G)=0$ for the query-relevant future Y, then $\mathsf{\Delta }V=0$ and (6) rejects f. Storing everything is optimal only if every fact passes (6). This distinguishes the rule from storing everything, storing recent items, summarizing everything, and storing by embedding similarity.

5.5. Write: How Large Must Memory Be?

Proposition 6 (information lower bound, under A4). Let $S\left(T\right)=S_{0}T+S_1\left(T\right)$ split the entropy of the observation stream into extensive and subextensive parts. The predictive information, the mutual information between past and future, equals $I_{\mathrm{pred}}\left(T\right)=S_1\left(T\right)$ [12]. Let $G_T$ be a memory state that is predictively sufficient, $I(G_T;Y)=I(X_{1:T};Y)$ for the future Y. Then

$$H\left(G_T\right)\ge I(G_T;Y)=I_{\mathrm{pred}}\left(T\right)\left(7\right)$$

If sufficiency is only required for query-relevant answer variables $Y_Q$, the bound becomes $H\left(G_T\right)\ge I(X_{1:T};Y_Q)$, which is at most $I_{\mathrm{pred}}\left(T\right)$.

Proof.$H\left(G_T\right)\ge I(G_T;Y)$ for discrete $G_T$, and sufficiency gives equality with $I(X_{1:T};Y)$. □

For a world describable by K independent parameters, $I_{\mathrm{pred}}\left(T\right)\approx \frac{K}{2}{\log }_{2}T$, and for nonparametric worlds it grows as $T^a$ with $a<1$ [12]. Two-part codes attain $\frac{K}{2}{\log }_{2}N+O\left(1\right)$ for regular parametric families [19], so in such worlds the bound is tight to leading order. Statistical complexity likewise bounds excess entropy from above in computational mechanics [40]; (7) is the same inequality in our setting.

Three qualifications keep this honest. (i) The result is about the description length of a sufficient memory, not the number of nodes: in a finite-parameter world the growth is in the precision of stored quantities, while the node count can stay constant. (ii) It is a lower bound; it does not say that any practical write policy attains it. (iii) It holds under (A4), which excludes non-stationarity (Section 5.7) and requires a sufficiency assumption that real graphs may not meet. Connecting this bound to admission, D3(c) shows that under (A4) criterion (6) never admits redundant facts, which pushes the admitted store toward a minimal sufficient statistic; we state this as a design consequence, not a theorem. Equation (7) defines value by predictive compression of the stream; Equation (6) defines it by future queries. They agree only when the query distribution is supported on the world’s predictive structure, which is the content of (A4).

5.6. Structure: Resolution, Summary Levels, Shortcuts, Cache

Proposition 7 (resolution choice, separable distortion). If the value lost at resolution ℓ is attributable to item i alone, with distortion $d_{\ell }(i\mid Q)$ relative to full evidence, the optimal resolution is

$${\ell }^{*}(i\mid Q)=\arg \underset{\ell }{\min }\left[d_{\ell }(i\mid Q)+\lambda {\tau }_{i\ell }\right]\left(8\right)$$

This is a rate–distortion tradeoff [24] with $\lambda$ as the slope. With interacting facts the distortion becomes $d_{\ell }(i\mid Q,C)$ and (8) is only a heuristic. Resolution rises with relevance and falls with $\lambda$; evidence-level resolution belongs to contested or high-stakes claims.

Design rule D4 (build summary levels lazily). Writing a summary level for item i costs about ${\kappa }_w\lambda ({\tau }_i^{\mathrm{full}}+{\tau }_i^{\mathrm{sum}})$ (read the item, write the summary). It saves $\lambda ({\tau }_i^{\mathrm{full}}-{\tau }_i^{\mathrm{sum}})$ on each retrieval at which the summary suffices. With $n_i$ lifetime retrievals, a sufficiency fraction ${\sigma }_i$, and $h_{i}T$ expected content changes (each forcing a re-summary), the summary pays iff

$${\sigma }_i{n}_i({\tau }_i^{\mathrm{full}}-{\tau }_i^{\mathrm{sum}})>{\kappa }_w({\tau }_i^{\mathrm{full}}+{\tau }_i^{\mathrm{sum}})(1+h_{i}T)\left(9\right)$$

When ${\tau }_i^{\mathrm{sum}}\ll {\tau }_i^{\mathrm{full}}$ and ${\kappa }_w\approx 1$, this is roughly ${\sigma }_i{n}_i>1+h_{i}T$: a leaf pays back its summary after one or two sufficient retrievals if its content is stable. Because $n_i$ is unknown at write time, the rule is implemented on read: generate the level when the observed retrieval count crosses the threshold in (9). Levels should be spaced roughly geometrically in token size (for example gist, summary, full, a factor of five to ten apart); this is a heuristic that follows from power-law distortion curves, not a theorem, and it implies few levels in practice. Internal summaries of subtrees or clusters serve a different purpose [30,7]: they give scent for deciding whether to descend, reducing the examined set from the branching fan-out to roughly $F{\log }_{F}n$ summaries for n leaves at fan-out F (a saving on the examination cost $\kappa$ of Section 5.2, not on the token side).

Proposition 8 (shortcut threshold). Suppose query patterns are Zipf-distributed over a fixed universe, $p_r=r^{-s}/H$ with $s>1$ and stable normalizer H [22], and materializing pattern r as a one-hop edge saves $\mathsf{\Delta }\tau$ tokens per query at a build and maintenance cost of $t_b$ tokens. The shortcut is worth building iff $N{p}_r\mathsf{\Delta }\tau >t_b$, i.e. for ranks

$$r<r^{*}={\left({\displaystyle \frac{N\mathsf{\Delta }\tau }{t_{b}H}}\right)}^{1/s}\left(10\right)$$

so the number of worthwhile shortcuts scales as $N^{1/s}$. If the pattern universe itself grows with N, H is not constant and the exponent changes.

Proposition 9 (cached core). Place fact f in the cached prefix iff its inclusion probability $p_f$ across queries satisfies

$$p_f>{\displaystyle \frac{q\lambda {\tau }_f+{\rho }_f}{\lambda {\tau }_f+{\rho }_f}}\left(11\right)$$

where ${\rho }_f$ is its distraction cost when not needed.

Proof. A core item costs $q\lambda {\tau }_f+(1-p_f){\rho }_f$ per query; fetching on demand costs $p_f(\lambda {\tau }_f+{\rho }_f)$. Compare. □

Example. With ${\rho }_f=0$ this reduces to $p_f>q$. If cached tokens cost a tenth of fresh tokens, a fact used in more than roughly ten percent of queries belongs in the core. A cache-write surcharge raises the effective q in proportion to how often the prefix is rebuilt.

5.7. Time: Maintenance and Staleness

Facts change, and (A4) holds only for the stable part of the world. Let fact f have hazard rate $h_f$, query rate ${\mu }_f$, staleness stakes $S_f$ (the cost of answering from an outdated fact), and verification cost c. Assume a stationary hazard and a fixed query rate. The expected staleness probability averaged over a refresh interval T is $1-(1-e^{-h_{f}T})/\left(h_{f}T\right)$, which is approximately $h_{f}T/2$ for $h_{f}T\ll 1$.

Proposition 10 (refresh interval, small hazard). For $h_{f}T\ll 1$, re-verification every T minimizes $c/T+{\mu }_f{S}_f{h}_{f}T/2$, giving

$$T^{*}=\sqrt{{\displaystyle \frac{2c}{{\mu }_f{S}_f{h}_f}}}\left(12\right)$$

Volatile, heavily queried, high-stakes facts are refreshed most often. For large $h_{f}T$ the exact expression above must be minimized numerically. Bi-temporal validity windows [8] keep the changed state recoverable without discarding history.

5.8. Coupling: Why Write and Read Are One Problem

Indexing versus lazy construction. Let $t_w$ be the up-front cost of an index and $\mathsf{\Delta }$ the per-query token saving it provides over building context lazily at query time. Index iff

$$N>N^{*}=t_w/\mathsf{\Delta }\left(13\right)$$

Below $N^{*}$, lazy read-time construction (agentic search) is cheaper; above it, write-time indexing wins. With partial indexing, (10) determines which head patterns to index first.

Remark (joint versus separate). For any write policy ${\pi }_w^0$, including one optimized without reference to retrieval (e.g., maximal stream compression), $J({\pi }_w^{*},{\pi }_r^{*}\left({\pi }_w^{*}\right))\ge J({\pi }_w^0,{\pi }_r^{*}\left({\pi }_w^0\right))$, since the joint optimum ranges over a superset. This is immediate and not the substantive claim. What is substantive is how large the gap is and where it comes from: the read-side parameters $(g_0,{\rho }_r,b_{\mathrm{eff}},\overline{\tau },\kappa )$ in (3) and (4) depend on the write policy. Summarization changes $\overline{\tau }$, shortcuts change the effective depth, fact admission changes b (Equation 4), and plan templates change the dependency depth. These are externalities of writing on reading, and they are the quantities Hypothesis 7 asks to measure.

A practical solution alternates: solve the read problem for the current graph, estimate $\mathsf{\Delta }J$ per candidate fact, summary and shortcut from the query log, update the graph, and repeat. We make no convergence claim.

6. Hypotheses

The theory yields the following testable hypotheses; none is yet tested.

1.

Depth is a function, not a constant. Optimal traversal depth follows (3): it grows with the log of the value-to-price ratio, falls with branching, and rises with scorer quality. A fixed depth of three is optimal only in a narrow parameter range.

2.

Local marginal-value conditions. At a locally optimal context, the inequalities (2) hold: marginal gain per token of included items is at least $\lambda$ and of excluded items at most $\lambda$.

3.

Window size versus accuracy. A policy that stops by marginal value reaches a plateau as the window grows (Proposition 2), while a policy that fills the window peaks and then degrades; the peak lies near $B_d$, independently of the window size.

4.

Information growth in precision. For a predictively sufficient store in a finite-parameter domain, the stored information grows like $\frac{K}{2}\log N$ in precision while node count saturates; in open-ended domains it grows as a power $N^a$, $a<1$. Fitting the growth curve diagnoses the regime. Raw logging grows linearly.

5.

Shortcut scaling. The number of worthwhile shortcuts scales as $N^{1/s}$ with the Zipf exponent s of the query distribution (Equation 10).

6.

Indexing threshold. Below $N^{*}=t_w/\mathsf{\Delta }$ queries, lazy construction dominates; above it, indexing does (Equation 13).

7.

Write externality. Jointly optimized write–read policies beat independently tuned ones at equal total cost, and the measured sensitivity of optimal depth to branching approaches $-h^{*}/\ln (b/{\rho }_r)$ (Equation 4); the gap widens as queries concentrate (larger s).

8.

Operational cost split. (a) With graph hops executed inside the store, the non-token cost share of a query is a small fraction of its token cost; it exceeds half only when each examination invokes an LLM. (b) Lazily built summaries break even after about $\left({\kappa }_w({\tau }^{\mathrm{full}}+{\tau }^{\mathrm{sum}})\right)/\left(\sigma ({\tau }^{\mathrm{full}}-{\tau }^{\mathrm{sum}})\right)$ retrievals, as in (9).

7. What This Theory Does Not Claim

–

It does not claim that knowledge graphs are always the best memory structure.

–

It does not claim that predictive information equals useful memory; Proposition 6 is a lower bound under a sufficiency assumption.

–

It does not claim that three-hop retrieval is universally wrong; it claims the optimum depends on measurable parameters.

–

It does not claim that the utility function of an LLM reader is known, or that $g_0$, ${\rho }_r$, $b_{\mathrm{eff}}$ and ${\sigma }_i$ are easy to estimate.

–

It does not claim that the optimum is directly computable in production; the alternating procedure has no convergence guarantee.

–

It treats $\lambda$ and the price ratios q and ${\kappa }_w$ as given. A single numeraire is a modeling choice.

–

Its closed forms rest on stylized assumptions (geometric signal decay, submodular value, Zipf queries, stationary hazards) and should be read as derived consequences of those assumptions, not as laws.