The Distribution of Lexical Information Within Dependency Spans: Cross-Linguistic Evidence from 22 Languages

1. Introduction

In dependency grammar, syntactic structure is organized through dependency relations between words. A syntactic dependency is typically represented as an asymmetric binary relation between a head and a dependent, and the difference between their positions in the linear sequence is referred to as dependency distance [1]. Dependency distance provides a concise measure of how far apart two directly related words occur and has been widely used in cross-linguistic comparisons of syntactic structure and in research on linguistic complexity [2,3,4]. Previous research has established a close relationship between dependency distance and syntactic processing demands: longer dependencies generally require comprehenders to maintain more unresolved structural information in working memory. The widespread tendency toward dependency length minimization in natural language has therefore often been interpreted as evidence that linguistic structure adapts to constraints on cognitive resources [5,6,7].

Dependency distance is, however, fundamentally a measure of length. Formally, it reduces the head, the dependent, and the complete linear interval between them to a single distance value, thereby providing a scalar representation of how a dependency unfolds in the linear sequence. Linear intervals containing different lexical items, part-of-speech sequences, and local dependency configurations may have the same dependency distance. Dependency distance thus constitutes a many-to-one mapping: it indicates how far apart two syntactically related items occur but does not preserve the specific sequential composition of the interval between them. Previous studies have recognized the limitations of measuring dependency length solely by the number of intervening words and have incorporated the syntactic properties of the intervening material. In particular, the Intervener Complexity Measure supplements conventional dependency length by quantifying the number of syntactic heads in the intervening region [8,9]. Because dependency distance is frequently used to discuss syntactic processing and cognitive load, it remains important to ask whether length alone adequately characterizes the linear realization of a dependency.

This question can be approached through the contextual predictability of individual words. The probability of a word depends on the preceding context. Psycholinguistic evidence shows that syntactic and semantic information jointly supports the prediction of upcoming words [10], while lexical surprisal, defined in terms of conditional probability, is a robust predictor of reading time [11]. Recent work has also directly linked dependency length to surprisal. Xu and Futrell [12] found that, in most of the languages they examined, the surprisal of the linearly first endpoint of a dependency was positively correlated with dependency length. This result suggests that a less predictable antecedent may receive a more robust memory representation and may consequently support a longer dependency. More recently, Jin and Liu [13] examined the association between dependency distance and surprisal across 34 languages. Their preprint reports that the association varies with dependency direction and relation type: positive associations are more consistent in head-final dependencies, weaker and more variable in head-initial dependencies, and stronger for content dependencies than for functional dependencies. Together, these findings indicate that the linear length of a dependency and the contextual predictability of its component words are not independent, but may reflect a systematic coupling between memory-related and prediction-related demands. They also leave a more fine-grained question open: if the complete interval between a head and its dependent is aligned by relative position, how does surprisal unfold across all the words in that interval, and do stable cross-linguistic distributional patterns emerge?

From an information-theoretic perspective, the surprisal of a word given its preceding context provides a measure of contextual predictability [14,15]. Surprisal has been consistently associated with reading times, eye movements, and neurocognitive measures and is therefore widely used as an index of incremental processing difficulty in language comprehension [16,17,18,19,20]. Previous studies have further shown that information is not distributed uniformly across the linear positions of a sentence: sentence-initial, medial, and final positions may display systematic differences [21,22]. Although the precise information contour of an individual sentence depends on its semantic content and lexical choices, averaging across many observations may therefore reveal relatively stable positional regularities.

Analyses based on whole sentences cannot, however, determine directly whether such regularities are associated with syntactic structure. Most existing studies use absolute sentence position as their reference frame, even though words at the same sentence position may have different syntactic roles and instances of the same dependency relation may occur at different positions. Sentence-based positional analyses consequently have difficulty separating general effects of linear position from syntactically conditioned effects. The present study instead uses the two endpoints of each dependency as structural anchors and aligns the continuous interval between them. This makes it possible to test whether surprisal varies systematically with relative position within a dependency span. The central question is whether the surprisal values across a span merely reflect an accidental aggregation of local lexical properties or form reproducible distributional patterns when many dependencies are considered together.

Answering this question has two main implications. First, it extends the analysis of dependency relations from endpoint distance to the way information unfolds between the endpoints. Second, a cross-linguistic comparison can reveal both the stability of this organization and the sources of its variation. Languages may share a basic framework for the linear distribution of information while producing different contours because of differences in head direction, part-of-speech composition, and structural roles.

We therefore adopt the dependency span as the unit of analysis. A dependency span is defined here as the continuous interval comprising a directly related head and dependent together with every word occurring between them in the linear sequence. Our concern is not whether every individual span follows a fixed trajectory, but whether stable information-distribution patterns emerge after large numbers of comparable spans have been aligned by relative position and statistically averaged.

Using a large pretrained language model, we estimate word-level surprisal and calculate mean surprisal at each position in dependency spans with dependency distances from 4 to 10 in 22 languages. We first test whether the resulting contours display reproducible distributional patterns. We then use clustering and segmented analysis to characterize their overall shapes and internal organization and compare similarities and differences across languages and dependency distances. Finally, on the basis of the principal patterns identified in these analyses, we examine lexical and syntactic variables that may be associated with them.

The study addresses three research questions:

• Does mean surprisal exhibit stable distributional patterns within dependency spans, and are these patterns consistent across dependency distances and languages?
• If stable distributional patterns exist, what structural properties characterize them, and what similarities and differences occur across languages?
• Which lexical and syntactic factors may be associated with the shared properties and cross-linguistic differences in the information distributions of dependency spans?

By addressing these questions, this study evaluates whether the interval defined by a dependency relation provides a structural reference frame for information distribution that differs from absolute sentence position. It further investigates whether the region traversed by a dependency exhibits statistically organized information patterns and how these patterns relate to syntactic structure, thereby establishing a closer connection between research on dependency distance and information-theoretic approaches to language.

2. Materials and Methods

2.1. Corpus Data

The syntactic data were obtained from version 2.14 of the Universal Dependencies (UD) treebanks [23]. The set of languages was determined jointly by the availability of UD treebanks and the pretraining coverage of XGLM [24], because all surprisal values were subsequently estimated with XGLM-2.9B. Candidate languages therefore had to meet two criteria: they had to be included in the pretraining data of XGLM and have at least one usable treebank in UD v2.14. When multiple UD treebanks were available for a language, all eligible treebanks were combined, and corpus size was calculated from the resulting number of sentences and word tokens.

All treebanks were downloaded from the UD website (https://universaldependencies.org). They provide annotated dependency structures together with word forms, parts of speech, and dependency relations. Table 1 reports the corpus sizes for the 22 languages.

2.2. Surprisal Estimation

We use surprisal to quantify the conditional predictability of a word in its preceding context. For the word $x_i$ at position $i$ in a sentence, surprisal is defined as the negative logarithm of its conditional probability given the preceding sequence:

$$S\left(x_i\right)=-{\mathrm{l}\mathrm{o}\mathrm{g}}_{2}P\left(x_i\mid x_{1:i-1}\right).$$

(1)

In information-theoretic terms, lower surprisal corresponds to a higher conditional probability and hence to greater model-estimated contextual predictability [14,15,25]. In this study, probabilities are estimated by the multilingual autoregressive language model XGLM from distributions learned during pretraining. The resulting surprisal values primarily represent model-level contextual predictability; they should not be equated directly with human linguistic knowledge, the inherent information value of a word, or real-time human processing load.

Model-derived surprisal is nevertheless relevant to human processing. Previous studies have shown that language-model surprisal explains part of the variation in reading times and eye-movement measures during naturalistic reading, and the relationship between surprisal and reading time has been tested across multiple languages [19,20,26]. These findings provide external-validity evidence for using model surprisal as an estimate of contextual predictability. At the same time, the correspondence is incomplete and model-dependent: lower perplexity or a larger model does not necessarily produce surprisal estimates that better predict human reading behavior, and knowledge acquired from massive training corpora may exceed the experience available to an average human reader [27]. We therefore treat XGLM surprisal as a computational approximation of contextual predictability and restrict our conclusions accordingly. Its relationship to human online processing must ultimately be evaluated with behavioral and neurocognitive data.

Word-level surprisal was estimated with XGLM-2.9B [24], a decoder-only causal Transformer that assigns a conditional probability to the current token on the basis of the preceding token sequence. Its left-to-right objective is therefore directly compatible with the definition in Equation (1). When a word was segmented into multiple subword tokens, the surprisal values of its subwords were summed to obtain word-level surprisal. XGLM was selected both because its autoregressive objective matches the required measure and because it was pretrained under a common architecture on data from 30 languages, allowing the languages in the present study to be analyzed without introducing methodological differences from separate monolingual models. Our sample was drawn from the intersection of the languages covered by XGLM pretraining and those represented by usable UD v2.14 treebanks, thereby limiting additional uncertainty from out-of-coverage inference.

2.3. Data Filtering

To reduce the influence of nonlinguistic material on dependency-distance and surprisal estimates, we excluded spans containing any token whose Universal Part-of-Speech tag was PUNCT, SYM, or X. A dependency span was retained only when its head, dependent, and all intervening tokens fell outside these three categories. We excluded the entire span rather than deleting individual tokens and reindexing the remainder, because the latter procedure would shift token positions and break the correspondence between relative positions, part-of-speech distributions, and mean surprisal.

We then retained only dependencies with distances from 4 to 10. Dependency distance was calculated as the absolute difference between the sentence indices of the head and dependent, $\left|positio{n}_{head}-positio{n}_{dependent}\right|$. The lower bound was set to 4 because shorter spans contain too few positional observations to support an analysis of internal information distributions and stage structure. The upper bound was set to 10 because the number of dependencies decreases rapidly as distance increases, leaving insufficient observations for stable estimates at greater distances. Cross-linguistic research has repeatedly shown that natural languages favor shorter dependencies [2,28,29,30,31,32,33]. In our data, dependencies of length 10 or less accounted for more than 95% of all dependencies in every language. The range from 4 to 10 thus provides enough internal positions for analysis while preserving adequate cross-linguistic sample sizes and stable estimates.

2.4. Principal Component Analysis

Principal component analysis (PCA) [34] was used to reduce the dimensionality of a set of linguistic variables that might be associated with information distribution inside dependency spans. The selection and standardization of these variables and the criterion used to retain principal components are described in Section 3.3.

3. Results

3.1. Basic Information-Distribution Patterns and Cluster Identification

To characterize the distribution of information within dependency spans, we calculated mean surprisal at each relative position for each of the 22 languages and each dependency distance from 4 to 10. Every language contributed one information-distribution curve at each of the seven distances, yielding 154 curves in total. Because curves associated with different dependency distances contained different numbers of positional observations, we used dynamic time warping (DTW) [35] to quantify pairwise differences in curve shape and clustered the curves on the basis of the resulting distance matrix. To select the number of clusters, we compared solutions with $k=2$ through $k=6$ using the silhouette coefficient [36]. The results are reported in Table 2.

The silhouette coefficient was highest for $k=2$ (0.4161) and was lower for every solution with more clusters. A two-cluster solution therefore provided the best balance of within-cluster cohesion and between-cluster separation in the present data and was adopted for the subsequent analyses. The value of 0.4161 nevertheless indicates only moderate separation. The two clusters should accordingly be interpreted as broad summaries of the principal differences in curve shape rather than as completely discrete categories.

The solution assigned 65 curves to Type 1 and 89 curves to Type 2, corresponding to 42.2% and 57.8% of the full set, respectively. Both types thus accounted for substantial proportions of the data, which indicates that neither resulted from a small number of anomalous curves. Figure 1 presents their representative DTW-aligned mean contours.

Type 1 curves show an approximately monotonic decline. Mean surprisal is comparatively high at the beginning of the span and then decreases progressively across the linear sequence. Curves within this type differ in their rate of decline and degree of local fluctuation, but their overall downward direction is relatively consistent.

Type 2 curves display a nonmonotonic pattern. Mean surprisal typically drops sharply near the beginning of the span, enters a comparatively stable interval, and then rises or fluctuates locally near the end. Their most distinctive feature relative to Type 1 is therefore the reversal in the direction of change at the end of the span.

Cluster membership was highly stable across dependency distances. Of the 22 languages, 20 were assigned to the same curve type at every distance from 4 to 10. Eight languages had all seven curves assigned to Type 1, and 12 had all seven assigned to Type 2. Only Russian and German displayed mixed membership. Three of the seven Russian curves were assigned to Type 1 and four to Type 2, whereas six of the seven German curves were assigned to Type 1 and one to Type 2. Table 3 summarizes these patterns; complete language-by-distance assignments are provided in Appendix A.

Both types therefore accounted for substantial numbers of curves, and most languages retained the same membership across dependency distances. The two patterns cannot be attributed solely to individual languages, particular distances, or a small set of outliers; rather, they represent two recurrent shapes in the information distributions of dependency spans. Both types also show changes in slope across relative position, suggesting that they may contain a further stage structure. The following section evaluates this possibility with segmented models and breakpoint estimation.

3.2. Segmented Structure and Statistical Validation

The clustering analysis revealed visually apparent changes across the course of both curve types. Curve shape alone, however, is insufficient to determine whether those changes constitute a stable stage structure. We therefore compared segmented linear models to test whether the two curve types could be characterized by a common number of stages.

Before fitting the models, we specified a reasonable upper bound on their complexity. Adding segments generally improves fit, but excessive segmentation reduces interpretability and makes it difficult to define a common comparison framework across curves of different lengths. We limited the maximum to three segments for two reasons. First, curves for a given language were generally assigned to the same type across dependency distances, suggesting that spans of different lengths follow the same or a closely related organization. A common segmentation scheme is therefore preferable to distance-specific numbers of segments. Second, the shortest curves, representing dependency distance 4, contain only five sampled relative positions. A model with four or more segments would leave only one or two observations per segment, preventing stable estimation of segment slopes and severely limiting interpretation. Three segments thus constitute the largest defensible model at the resolution of the present data.

We fitted one-, two-, and three-segment linear models and compared them with the Bayesian information criterion (BIC) [37]. As shown in Table 4, the three-segment model produced the lowest BIC for both curve types and clearly outperformed the one- and two-segment alternatives. Both types can therefore be represented more adequately as three continuous stages than as a uniform trend or a simple two-stage pattern.

After selecting the three-segment model, we estimated the two breakpoints and the slope of each segment. The results are shown in Figure 2 and Table 5.

The two curve types have broadly comparable stage boundaries but differ markedly in how surprisal changes within the stages. During Stage 1, surprisal decreases in both types, with a substantially steeper decline in Type 2. In Stage 2, the slopes of both types approach zero and the change in surprisal becomes much more gradual. The two types diverge in Stage 3: Type 1 continues to decline, whereas Type 2 changes from a negative to a positive slope. This final-stage reversal constitutes the principal morphological distinction between the two curve types.

To determine whether the type-level mean curves adequately represented their members, we mapped the estimated breakpoints onto every individual curve and calculated the consistency of slope direction within each segment. Table 6 reports the resulting counts.

Within-type consistency was high. All Type 1 curves had negative slopes in Stages 1 and 2, and more than 95% continued to decline during Stage 3. Type 2 curves likewise showed high agreement, with the great majority displaying an increase during Stage 3. The three-stage structure is therefore not merely an artifact of averaging but captures shared patterns across the member curves. In summary, both types can be divided reliably into three continuous stages with broadly similar boundaries, while their principal difference lies in the direction of change during the final stage.

3.3. Latent Variable Structure Across Stages

The preceding analyses showed a stable three-stage organization and localized the principal cross-linguistic difference to Stage 3. We next used the three stages as the units of analysis to identify latent linguistic dimensions associated with changes in surprisal and to compare the two final-stage patterns.

The PCA input consisted of linguistic variables measured at each relative position within the dependency spans. These included word-length measures (mean word length, standard deviation of word length, mean content-word length, and mean function-word length) and proportional measures (the proportion of content words and the proportion of each part of speech). The complete set is listed in Appendix B. Before analysis, we removed sparse variables that showed insufficient variation because low-frequency categories such as INTJ were zero at most observations. We retained proportional and mean-value variables but excluded raw counts, which are jointly affected by sample size and language-specific structure and therefore provide less stable indicators for cross-linguistic comparison.

To remove differences in overall surprisal levels and variable ranges across languages, every variable was standardized within language using a $z$ score:

$$z={\displaystyle \frac{x-\mu }{\sigma }}.$$

(2)

Here, $\mu$ and $\sigma$ denote the language-specific mean and standard deviation of the relevant variable. All subsequent PCA calculations used these standardized values.

Rather than pooling the entire dataset in a single PCA, we performed separate analyses at the level of curve type by stage. Curves were first assigned to Type 1 or Type 2 on the basis of the DTW clustering results and then divided into three stages using the breakpoint locations estimated for the corresponding type-level prototype. This procedure yielded six analysis units: Type 1 × Stages 1–3 and Type 2 × Stages 1–3. PCA was conducted separately within each unit. Components were retained until cumulative explained variance reached at least 80%, providing a common criterion across units while preserving most of the information in the original variables.

Table 7 summarizes the results. Between four and nine principal components were required to exceed 80% cumulative explained variance, indicating that the reduced representations preserved most of the original variation. The number of retained components nevertheless varied substantially across stages. The Stage 2 units for Types 1 and 2 required nine and eight components, respectively, whereas each of the other four units required only four or five. The variable structure of Stage 2 is consequently more dispersed and appears to reflect a larger number of latent dimensions than that of Stages 1 and 3.

The proportion of variance explained by PC1 reinforces this difference. For Type 1, PC1 accounted for 32.57%, 18.03%, and 48.79% of the variance in Stages 1, 2, and 3, respectively; the corresponding values for Type 2 were 45.25%, 30.62%, and 56.56%. For both types, PC1 explained the smallest share in Stage 2 and the largest share in Stage 3. Information change in the initial and final stages can therefore be described to a greater extent by a small set of dominant variables, whereas the middle stage requires a more distributed set of latent factors.

To identify the latent dimension most closely associated with mean surprisal, we calculated Spearman correlations between each retained component and mean surprisal in every analysis unit. PC1 showed the strongest absolute correlation in all six units and was therefore selected as the primary target for linguistic interpretation. The remaining components continue to represent secondary variation in the original variables; complete correlations are provided in Appendix C.

Table 8. Spearman correlations between PC1 and mean surprisal in the six analysis units.

Curve type	Stage 1	Stage 2	Stage 3
Type 1	-0.64	0.52	-0.56
Type 2	0.89	0.59	0.70

Table 8. Spearman correlations between PC1 and mean surprisal in the six analysis units.

Curve type	Stage 1	Stage 2	Stage 3
Type 1	-0.64	0.52	-0.56
Type 2	0.89	0.59	0.70

Across the six units, the absolute PC1–surprisal correlations ranged from 0.52 to 0.89 and exceeded those of all other retained components. Correlations involving secondary components were generally weak. With the exception of PC2 in Type 2 Stage 2 ($r_s=-0.42$), the absolute value of every secondary-component correlation was no greater than 0.29. Thus, although the number of retained components differs across stages, PC1 is consistently the latent dimension most closely associated with changes in surprisal.

The stages therefore differ not only in their surprisal contours but also in their latent variable structure. Stage 2 requires more components to reach the 80% threshold and assigns a smaller proportion of total variance to PC1, indicating a comparatively dispersed structure. Stages 1 and 3 require fewer components and have higher PC1 shares, indicating more concentrated dominant dimensions. Importantly, even in the more dispersed middle stage, PC1 remains the component most strongly associated with mean surprisal. We therefore examine its variable composition in the following section.

3.4. Composition of PC1 and Cross-Stage Comparison

For each of the six analysis units, we extracted the ten variables with the largest absolute loadings on PC1 and counted how often each variable occurred in these top-ten sets. Full PC1 loadings are provided in Appendices D and E. Table 9 lists the six variables that occurred most frequently.

Both content_ratio and VERB_ratio appeared among the ten largest loadings in all six analysis units. The variables avg_word_length, DET_ratio, AUX_ratio, and ADP_ratio each appeared in five units, whereas no other variable appeared more than four times. Although the precise ranking of variables therefore differs across units, the recurring core is relatively concentrated. It comprises the proportions of content words and verbs, mean word length, and the proportions of three function-word categories: determiners, auxiliaries, and adpositions.

To compare how these six variables relate to surprisal across stages, we calculated their Spearman correlations with mean surprisal. Complete values are reported in Appendix F and summarized in Figure 3.

Type 1 Stage 1, Type 2 Stage 1, and Type 2 Stage 3 exhibit a relatively clear shared pattern. The content-word ratio, verb ratio, and mean word length are generally positively correlated with surprisal, whereas the determiner, adposition, and auxiliary ratios are generally negatively correlated. In most stages, higher proportions of content words, longer words, and more verbs are therefore associated with higher surprisal, whereas larger proportions of structurally oriented function words are associated with lower surprisal.

The correlation pattern is weaker and more heterogeneous in Stage 2. This mixture of directions is consistent with the dispersed variable structure observed in the PCA and suggests that change during the middle stage cannot readily be reduced to a single stable combination of variables.

Type 1 Stage 3 is markedly different. In this unit, the correlations of the content-word ratio, verb ratio, and mean word length with surprisal reverse direction, while those of the determiner, adposition, and auxiliary ratios shift from negative to positive. The largest difference between the two curve types is therefore concentrated in the final stage, whereas their initial and middle stages remain broadly similar.

To assess the cross-linguistic consistency of this reversal, we calculated language-specific correlations between the six core variables and surprisal in Type 1 Stage 3. We then compared these values with the ratio between the frequencies of dependents and heads at the beginning of a span (Dep/Head). The correlations are reported in Table 10.

Note. Values are Spearman correlation coefficients. * $p<.05$; coefficients without an asterisk are not statistically significant.

The degree of cross-linguistic consistency differs across the six variables. The content-word and verb ratios are negatively correlated with surprisal in most languages, whereas the determiner and adposition ratios are usually positively correlated, and these relationships reach significance in a majority of cases. By contrast, mean word length and the auxiliary ratio vary more substantially in direction across languages. The final-stage reversal is therefore supported primarily by the first four variables and should not be interpreted as a uniform reversal of all six variables in every language.

We next tested whether the reversal in Type 1 Stage 3 was associated with dependency direction. For each language, we calculated the ratio between the frequency with which a dependent versus a head occupied the beginning of a dependency span. A ratio greater than 1 means that the left endpoint was more often a dependent and is therefore associated with a head-final tendency; a ratio below 1 indicates that the left endpoint was more often a head and is associated with a head-initial tendency. Figure 4 displays the log-transformed ratios.

All eight languages consistently assigned to Type 1 had Dep/Head ratios greater than 1. Of the 12 languages consistently assigned to Type 2, all except Japanese had ratios below 1. Russian and German, which contained both curve types, displayed mixed characteristics. Type 1 curves therefore tend to co-occur with spans that begin with a dependent and place the head toward the right endpoint. The results for Japanese, Chinese, and the mixed-membership languages nevertheless show that curve type and directionality do not stand in a strict one-to-one relationship. Complete language-specific ratios are reported in Appendix G.

Finally, to control more directly for part of speech, we selected VERB tokens in Type 1 Stage 3 and divided them into three structural roles within the current dependency span: Head, Dependent, and Internal. Figure 5 compares their surprisal distributions; language-specific means, medians, and standard deviations are provided in Appendix H.

Most languages display a relatively stable gradient: verbs functioning as heads have the lowest surprisal, span-internal verbs have the highest, and verbs functioning as dependents generally fall between the two. This pattern occurs not only in languages consistently assigned to Type 1 but also, to varying degrees, in the mixed-membership languages Russian and German.

In summary, PC1 is composed primarily of a limited set of recurring linguistic variables across stages, but these variables operate differently in the final stage. Cross-linguistic comparisons and the analysis controlling for verbal part of speech both localize the principal difference to Type 1 Stage 3, while Stages 1 and 2 remain comparatively similar across curve types. The following discussion considers the linguistic mechanisms and typological implications of these stage-specific patterns.

4. Discussion

4.1. The Three-Stage Pattern of Information Distribution

The clustering and segmented analyses identify a relatively stable three-stage structure in both curve types. Surprisal decreases during Stage 1, changes more gradually during Stage 2, and diverges during Stage 3: Type 1 continues to decline, whereas Type 2 turns upward. The two types thus share a comparable stage framework and differ primarily in the magnitude of change and, most notably, in the direction of the final stage.

The PCA results further show that the underlying variable structure differs across stages. Stages 1 and 3 require fewer principal components to reach 80% cumulative explained variance and assign a relatively large share of variance to PC1. Stage 2 requires more components, and PC1 explains a smaller share. Covariation among the measured variables is therefore more concentrated at the beginning and end of the span and more dispersed in the middle.

During Stage 1, the content-word ratio, verb ratio, and mean word length are generally positively correlated with surprisal, whereas the determiner, adposition, and auxiliary ratios are generally negatively correlated. This pattern is consistent with a span beginning that contains a relatively high concentration of information-rich content words, followed by an increase in function-word proportions. By contrast, the flatter contour of Stage 2 is accompanied by weaker and less consistent correlations. No single variable dominates this stage, and its relative stability may arise from the combined variation of multiple lexical and syntactic features.

4.2. Final-Stage Divergence and Dependency Directionality

Stage 3 contains the largest difference between the two curve types. In Type 2, the content-word ratio, verb ratio, and mean word length remain positively correlated with surprisal, while the determiner, adposition, and auxiliary ratios remain negatively correlated, and surprisal rises toward the endpoint. The final stage of Type 2 therefore preserves the usual association between lexical informativeness and lower predictability: a larger share of content-bearing material tends to coincide with greater prediction difficulty.

Type 1 Stage 3 shows a different pattern. The curve continues to decline, the content-word and verb ratios are generally negatively correlated with surprisal, and the determiner and adposition ratios are generally positively correlated; mean word length and the auxiliary ratio vary more substantially across languages. This pattern occurs most often in languages with a strong tendency toward head-final dependencies. One possible explanation is that, in head-final structures, preverbal or pre-head material supplies accumulating cues that strengthen expectations for the upcoming head, thereby increasing its conditional probability and reducing its surprisal. Research on Japanese indicates that comprehenders maintain expectations for a later syntactic head before it appears [38], while work on German verb-final structures shows that additional preverbal constituents can facilitate the subsequent verb under certain conditions [39,40]. Expectation-based facilitation is not limited to head-final structures, however, because antilocality effects can also occur in nonargument–verb dependencies [41]. Head-finality should therefore not be treated as a necessary condition for predictive facilitation, but as one structural configuration in which its effect may become concentrated near the end of a dependency span.

All eight languages consistently assigned to Type 1 had a Dep/Head ratio above 1 at span onset, indicating an overall tendency for the head to occur toward the right endpoint. This pattern is compatible with the hypothesis that preceding material progressively constrains a later head. The weak reversal in Chinese and the mixed pattern in German nevertheless show that dependency direction is one contributor to final-stage shape rather than a sufficient explanation.

4.3. Verb Role and Predictive Constraint

To determine whether the low surprisal in Type 1 Stage 3 could be attributed solely to part-of-speech composition, we held part of speech constant and compared VERB tokens functioning as heads, dependents, or span-internal elements. Verbs were selected because they carry substantial semantic content and frequently serve as dependency heads, making them suitable for testing structural-role differences while controlling lexical category.

The results generally show the lowest surprisal for heads, the highest for internal verbs, and intermediate values for dependents. Even within a single part of speech, information uncertainty therefore varies with a token’s structural role in the current dependency span. An internal verb does not close the focal dependency; when it occurs, the structure is still unfolding and the space of possible continuations remains comparatively open. A head verb, by contrast, often provides the governing center for preceding dependents and integrates structural constraints that have accumulated across the span, making it more predictable. Although a dependent occupies an endpoint, it generally does not govern the preceding structure and therefore tends to fall between the other two roles.

The continuing decline of Type 1 curves during Stage 3 cannot consequently be explained by lexical informativeness alone. It is also associated with the progressive accumulation of predictive constraints within dependency structure.

4.4. Implications and Limitations

The results show that a dependency span has not only a length but also a position-sensitive information profile. Dependency distance indicates how far apart two syntactically related words occur, whereas the three-stage contours reveal how information unfolds across that interval. Dependency-span analysis therefore provides a more direct bridge between dependency-distance research and information-theoretic approaches to language.

At the same time, the patterns reported here emerge from statistical averages across large numbers of spans and do not imply that every individual span follows the same trajectory. The maximum complexity of the segmented model was constrained by the fact that the shortest curves contained only five positions. Exceptions also remain in the relationship between dependency directionality and final-stage shape. Future work should control more finely for dependency-relation type, morphological marking, and word-order flexibility and should combine corpus-derived contours with reading-time, eye-tracking, or neurocognitive evidence to determine whether these statistical patterns have a corresponding processing basis.

5. Conclusions

This study used dependency spans as its unit of analysis and examined the distribution of information across their linear positions at dependency distances from 4 to 10. The analysis combined dependency treebanks from 22 languages with word surprisal estimated by XGLM. Instead of representing a dependency only through its endpoints and their distance, we included the complete interval between them and asked whether the traversed region displays a stable organization of information in the aggregate.

The information distributions within dependency spans exhibit a robust three-stage organization. Although the cross-linguistic contours cluster into two principal types, both types share a similar stage framework and broadly comparable boundaries. They are therefore better understood as distinct realizations of a common stage structure than as wholly independent organizational mechanisms. Descriptively, Stage 1 is characterized by rapid adjustment in surprisal, Stage 2 by relative maintenance, and Stage 3 by either continued decline or terminal reorganization. These labels should be treated as functional hypotheses derived from the observed contours rather than as directly demonstrated processing mechanisms.

The PCA results show that the three stages have different latent variable structures. PC1 explains a relatively large share of the variance in Stages 1 and 3, where changes are concentrated in a smaller set of core variables. Stage 2 requires more retained components and has the lowest PC1 share, indicating a more strongly distributed contribution from multiple factors. The content-word, verb, determiner, auxiliary, and adposition ratios, together with mean word length, recur as core variables across stages. Information organization within dependency spans is therefore associated with the relationship between lexical informativeness and the structural functions of syntactic categories.

Type 1 Stage 3 displays a distinctive reversal in correlation direction: the typically positive correlations of the content-word and verb ratios with surprisal become negative, while the determiner and adposition ratios become positive. Cross-linguistic comparison indicates that this reversal occurs mainly in languages with high Dep/Head ratios at span onset and a strong tendency toward head-final dependencies. The verb-role analysis further shows that most languages have the lowest surprisal for heads, the highest for span-internal verbs, and intermediate values for dependents. Low final-stage surprisal is therefore difficult to explain through part-of-speech composition alone. A plausible account is that, as the internal context of a span unfolds, constraints on the conditional probability of a later head accumulate and make the endpoint increasingly predictable. This interpretation is compatible with expectation-based incremental processing [15] but requires direct behavioral and neurocognitive testing.

Overall, the interval traversed by a dependency relation is not merely a linear gap connecting a head and its dependent; in the aggregate, it exhibits stable patterns of information organization. Dependency distance reveals how far apart syntactically related words occur, while dependency-span analysis additionally reveals how information unfolds within that distance. These findings extend the analytical scope of dependency-distance research and provide new cross-linguistic evidence concerning the relationship between dependency structure and information distribution. Future research can evaluate the cognitive mechanisms behind the three-stage pattern by integrating corpus analysis with reading experiments, eye movements, and neurocognitive measures.