Canonical and Compact Representations for Small Language Model Graph Classification

1. Introduction

Graphs are a natural representation for relational data, including molecules, proteins, social networks, and interaction systems. Graph classification is therefore a central problem in machine learning, traditionally addressed with graph kernels and graph neural networks (GNNs). Recently, language models have also been explored as graph classifiers by converting graphs into textual descriptions and fine-tuning the model on the resulting graph-as-text inputs. This direction is attractive because it reuses the broad modeling capacity of pretrained language models, but it also introduces a basic methodological question: how should a graph be written as text?

Unlike ordinary text, a graph has no intrinsic node order. Two adjacency lists may describe the same graph while looking very different to a language model. Consequently, graph-as-text classification is sensitive to arbitrary serialization choices. Prior work has attempted to reduce this sensitivity through permutation-based training or node-ordering heuristics, but these approaches can increase training cost and do not fully resolve the representation problem. At the same time, small language models have limited context windows and attention budgets, making the length and redundancy of graph serializations practically important.

In this paper, we study the role of graph representation in small language model graph classification. We focus on two complementary families of representations. The first consists of canonical graph texts, where the graph is serialized after applying a deterministic canonicalization procedure. We consider canonical forms based on Bliss, Nauty/Pynauty, and a BFS-profile ordering. The second consists of compact graph representations, including graph6 and NetLSD-style structural descriptors, which reduce input length and test whether shorter encodings preserve enough information for classification. We also evaluate invariant soft-prompt baselines and a late-fusion strategy that combines several canonical views using validation-derived weights.

Our experiments use five standard graph classification datasets from TUDataset [1]: PROTEINS, MUTAG, BZR, PTC_MR, and IMDB-BINARY. We evaluate three instruction-tuned small language models: Llama-3.2-1B-Instruct, Qwen2.5-1.5B-Instruct, and SmolLM2-1.7B-Instruct. The results show that representation choice has a substantial effect on performance. Canonical representations generally improve over raw adjacency lists, compact encodings can retain useful accuracy while greatly reducing token length, and validation-weighted late fusion over canonical views provides the strongest aggregate performance.

The main contributions of this work are as follows:

• We provide a systematic comparison of raw, canonical, compact, and soft-prompt graph representations for small language model graph classification.
• We show that canonical graph texts often improve classification accuracy over raw adjacency-list serialization.
• We quantify the accuracy–token trade-off of compact graph representations, showing that substantial compression can preserve useful predictive performance.
• We introduce a simple validation-weighted late-fusion strategy over canonical graph views, which achieves the best aggregate performance across the evaluated settings.

2. Related Work

2.1. Graph Classification

Graph classification has been studied extensively through graph kernels and graph neural networks. Graph kernels compare graphs through hand-designed structural features, such as walks, shortest paths, graphlets, and subtree patterns. Among them, the Weisfeiler–Lehman (WL) subtree kernel is especially influential, as it iteratively refines node labels and compares the resulting multiset features efficiently [2]. GNNs instead learn task-specific graph representations by message passing. Standard architectures include GCN [3], GraphSAGE [4], GAT [5], and GIN [6]. These models provide strong graph-specific baselines, but require architectures designed for relational input.

2.2. Language Models for Graphs

A growing line of work studies how language models can be applied to graph-structured data. One approach is to convert graphs into textual descriptions, such as edge lists, adjacency lists, paths, triples, or natural-language graph summaries. This graph-to-text direction makes it possible to reuse pretrained language models without modifying their internal architecture. Recent surveys describe this broader area as LLMs for graphs, distinguishing graph-to-text approaches from graph-to-token or embedding-based approaches [7]. However, graph-to-text methods inherit a central difficulty: graphs do not have a natural node order. As a result, two isomorphic graphs can produce different text strings, and the language model may treat them as different inputs. This makes serialization choice a methodological component of the model rather than a neutral preprocessing step.

2.3. Graph Canonicalization

Graph canonicalization addresses the node-order ambiguity by assigning an isomorphism-invariant labeling to a graph. Classical tools such as nauty and Traces compute canonical labels and automorphism groups using individualization-refinement search [8]. Bliss follows a related canonical-labeling tradition and is widely used for practical graph isomorphism and automorphism computation [9]. These methods are attractive for graph-as-text learning because they can transform arbitrary node orderings into deterministic serializations. In contrast, heuristic orderings such as degree sorting or BFS-based profiles may improve consistency and readability, but do not generally provide the same formal canonical-labeling guarantees. Our work compares exact canonical labeling tools with a deterministic BFS-profile representation to test both principled and lightweight ways of stabilizing graph text.

2.4. Compact Graph Representations

A separate issue is the length of graph serializations. Explicit adjacency lists can be verbose, especially for larger graphs, and this matters for small language models with limited context and attention budgets. Compact graph encodings provide an alternative. The graph6 format is a concise representation for storing simple graphs, commonly used together with graph generation and isomorphism tools such as nauty. Spectral descriptors such as NetLSD summarize graph structure through heat-trace signatures across multiple scales [10]. These representations are not equivalent to full natural-language graph descriptions, but they test whether shorter structural encodings preserve enough information for classification. Our experiments therefore evaluate both accuracy and token cost, making the compression trade-off explicit.

2.5. Position of This Work

Most prior work on language models for graphs focuses on whether graph information can be expressed as text or tokens. We focus instead on the representation layer itself: how canonicalization, compactness, and multi-view fusion affect small language model graph classification. Unlike purely architectural approaches, our methods are model-agnostic and can be applied to different instruction-tuned SLM backbones. The closest conceptual connection is to graph canonical labeling and graph descriptors, but we study them specifically as input representations for language-model-based graph classifiers.

3. Method

3.1. Problem Setting

We consider supervised graph classification. Let $\mathcal{D}={\left\{(G_i,y_i)\right\}}_{i=1}^N$ be a dataset of graphs and labels, where each graph is $G_i=(V_i,E_i,X_i)$ with node set $V_i$, edge set $E_i$, and optional node features $X_i$. The goal is to learn a classifier $f\left(G_i\right)\to y_i$. In this work, the classifier is a small language model fine-tuned on textual or prompt-based representations of graphs.

The central difficulty is that graphs do not have a canonical node order. A graph can be serialized in many equivalent ways, and different node permutations may yield different text inputs for the same underlying structure. We therefore study how the choice of graph representation affects classification accuracy and input length.

3.2. Graph-as-Text Classification

The raw baseline represents each graph as an adjacency-list text. Nodes are indexed from 0 to $\left|V\right|-1$, node features are serialized when available, and edges are written explicitly through neighbor lists. The resulting text is appended to a classification prompt and used as input to the language model. The model is fine-tuned with LoRA adapters [11] while the base model weights remain frozen.

This representation is simple and lossless, but it is not invariant to node renaming. If the same graph is relabeled, the adjacency list changes even though the graph structure and class label are unchanged.

3.3. Canonical Graph Texts

To reduce sensitivity to arbitrary node order, we evaluate canonical graph texts. A canonicalization procedure computes a deterministic node ordering before serialization. The graph is then written as an adjacency-style text under this order. We consider three canonical or deterministic structural views.

Bliss.

The Bliss view uses canonical labeling based on the Bliss graph isomorphism tool. It follows the individualization-refinement family of canonical labeling algorithms and is designed for efficient handling of large and sparse graphs.

Pynauty.

The Pynauty view uses a Python interface to nauty-style canonical labeling. Like Bliss, it aims to produce an isomorphism-invariant labeling, so that isomorphic graphs receive the same canonical form up to implementation details and feature handling.

BFS profile.

The BFS-profile view is a lightweight deterministic ordering. It constructs rooted breadth-first traversal profiles and orders nodes according to these structural signatures. This representation is not used as a formal isomorphism solver, but it provides a compact and stable structural view that is often shorter than exact canonical adjacency forms.

After ordering the nodes, all canonical views are serialized using the same graph-as-text template. This isolates the effect of the node ordering from the rest of the prompt format.

3.4. Compact Graph Texts

Canonical adjacency-style texts may improve consistency, but they can remain long. We therefore also evaluate compact graph representations.

Graph6.

Graph6 is a compact textual encoding of simple graphs. It stores the graph structure much more densely than an adjacency list. It is lossless for the graph topology, but less transparent to a language model than a natural adjacency-style serialization.

NetLSD.

NetLSD represents a graph through a spectral heat-trace signature. This produces a fixed-size structural descriptor that is invariant to node ordering and substantially shorter than explicit edge-based text. Unlike Graph6, it is a lossy descriptor: it summarizes the graph rather than encoding every edge.

These compact representations allow us to measure an accuracy–token trade-off: whether shorter inputs preserve enough structural information for small language model classification.

3.5. Soft-Prompt Baselines

We also include soft-prompt baselines [12] to compare textual graph representations with learned continuous graph conditioning. In these models, a fixed number of virtual prompt tokens is prepended to the language model input. We evaluate two variants. The first maps invariant graph statistics to soft prompt embeddings through a multilayer perceptron. The second uses a small graph neural network to encode the graph and maps the pooled graph embedding to soft prompt tokens. In both cases, the resulting prompt is combined with LoRA fine-tuning.

These baselines test whether learned continuous graph conditioning can replace or improve upon explicit graph serialization.

3.6. Validation-Weighted Late Fusion

Single canonical views expose different structural regularities to the language model. Rather than selecting one canonicalization globally, we combine multiple canonical views using validation-derived weights.

Let $\mathcal{V}=\{v_1,\dots ,v_K\}$ be the set of canonical views, here Bliss, Pynauty, and BFS profile. For each view $v_k$, the fine-tuned model produces class probabilities $p_k(y\mid G)$. We assign each view a non-negative weight $w_k$ estimated from validation performance. In our experiments, weights are derived from validation negative log-likelihood, so views with better validation calibration receive larger weights. The final prediction is

$$p(y\mid G)={\displaystyle \sum _{k=1}^K}{w}_k{p}_k(y\mid G),{\displaystyle \sum _{k=1}^K}{w}_k=1.$$

The predicted class is then

$$\widehat{y}=arg\underset{y}{max}p(y\mid G).$$

This late-fusion strategy is model-agnostic and does not require modifying the language model architecture. It uses the complementarity of canonical graph texts while keeping each individual training run identical to the single-view setting.

4. Results

Table 1 reports the full 10-fold cross-validation accuracy across five datasets and three small language models. The results show that graph representation has a substantial impact on classification performance. Raw adjacency lists provide a simple baseline, but they are rarely the strongest representation. Edge lists perform similarly to adjacency lists, but do not improve the accuracy–token trade-off. Canonical graph texts generally improve over raw serializations, and the validation-weighted late-fusion strategy obtains the strongest aggregate performance. Figure 1 visualizes the corresponding accuracy–token trade-off across the evaluated graph text representations.

Across all five datasets, late fusion achieves the best average accuracy for all three SLM backbones, as shown in Table 2. Its overall mean accuracy is $0.7579$, compared with $0.6850$ for adjacency lists. This corresponds to an average gain of $+0.0729$. Late fusion also obtains the best average rank, $1.33$, indicating that the improvement is not caused by a single dataset or model.

Among the single canonical views, Pynauty, BFS profile, and Bliss are all competitive. Pynauty gives the best mean single-view accuracy in the aggregate, with an overall score of $0.7331$, followed by BFS profile ($0.7297$) and Bliss ($0.7259$). This suggests that canonicalization itself is useful for graph-as-text classification: stabilizing the node order often makes the serialized graph easier for the language model to exploit. The late-fusion method improves further by combining complementary canonical views rather than committing to one canonicalization method.

The raw edge-list serialization provides a useful sanity check. It is essentially tied with adjacency-list serialization in accuracy ($0.6845$ vs. $0.6850$), but it is longer on average ($615.7$ vs. $511.9$ tokens). Thus, making the graph text more explicit does not by itself improve the representation; the gains come from canonicalization or compact structural encoding.

The compact representations show a different behavior. As shown in Table 3, Graph6 reduces the mean token length by $66.3\%$ relative to adjacency lists, while still improving accuracy from $0.6850$ to $0.6999$. NetLSD gives a stronger accuracy gain, reaching $0.7177$, while reducing token length by $48.1\%$. These results indicate that compact representations can preserve useful structural information while substantially reducing input length. However, compact forms do not match the accuracy of canonical graph texts or late fusion, which suggests a clear accuracy–token trade-off.

The soft-prompt baselines are competitive in some cases but do not provide the strongest overall results. Soft stats MLP improves over adjacency lists on average, while Soft GNN is closer to the raw adjacency baseline. This indicates that learned continuous conditioning can help, but in our setting explicit canonical graph texts remain more reliable.

Overall, the results support three main observations. First, graph serialization is an important modeling choice for SLM-based graph classification. Second, canonical graph texts provide consistent gains over raw graph serializations. Third, compact graph representations offer meaningful token savings, but with some loss in accuracy relative to canonical multi-view fusion.

5. Limitations

This study has several limitations. First, the experiments are restricted to five standard graph classification datasets and three small instruction-tuned language models. Although these datasets cover molecular, biological, and social graph domains, they are still relatively small compared with modern large-scale graph benchmarks. The conclusions should therefore be interpreted as evidence for small and medium graph-classification settings rather than as a universal statement about all graph learning tasks.

Second, canonicalization introduces additional preprocessing cost. Exact canonical labeling with tools such as Bliss or nauty-style methods can be efficient on many practical graphs, but its cost may increase on difficult graph families. This cost is not reflected directly in the token-length trade-off table, which measures language-model input length rather than canonicalization runtime.

Third, late fusion improves aggregate accuracy but requires multiple canonical views. This increases inference cost because the model must process several serialized versions of the same graph. The method is therefore most appropriate when accuracy is more important than single-pass inference efficiency. For deployment settings with strict latency or compute budgets, compact single-view representations such as Graph6 or NetLSD may be more practical.

Fourth, the compact representations studied here are limited to Graph6 and NetLSD. Graph6 is lossless but not naturally linguistic, while NetLSD is compact and invariant but lossy. Other graph descriptors, spectral features, substructure counts, or learned graph tokenizers may provide different accuracy–efficiency trade-offs.

Finally, the study focuses on representation choice rather than language model architecture. We use LoRA adaptation and soft-prompt baselines, but do not redesign the transformer architecture for graph inputs. Combining canonical graph texts with architectures that explicitly model multiple graph views or structural constraints remains an open direction.

6. Future Work

Future work should extend this study in several directions. First, the evaluation should be expanded to larger graph benchmarks and additional domains, including larger molecular datasets, program graphs, citation networks, and heterogeneous graphs. This would test whether the observed benefits of canonical graph texts persist beyond the relatively small benchmark graphs considered here.

Second, canonicalization cost should be analyzed more explicitly. While our results show that canonical graph texts can improve language-model accuracy, a complete deployment-oriented comparison should include preprocessing time, memory use, and inference latency. This is especially important for exact canonical-labeling methods, whose runtime may vary substantially across graph families.

Third, compact graph representations deserve further investigation. Graph6 and NetLSD show that shorter inputs can retain useful predictive signal, but they represent only two points in a larger design space. Future work could study graphlet signatures, WL-based count descriptors, spectral summaries, motif profiles, or learned discrete graph tokenizers. Such representations may provide better trade-offs between interpretability, compression, and classification accuracy.

Fourth, the late-fusion strategy can be developed into more adaptive multi-view models. In this work, view weights are estimated from validation performance and kept fixed at inference time. A natural extension is to learn graph-dependent view weights, allowing the model to choose which canonicalization is most informative for each input graph. Another direction is to distill multi-view predictions into a single-view student model, retaining the benefits of fusion while reducing inference cost.

Finally, future work should connect graph-as-text representations more closely with theoretical questions of permutation invariance. Exact canonicalization provides one route to invariant textual inputs, while compact descriptors provide another. Understanding when these representations preserve the task-relevant information needed by language models remains an important open problem.

7. Conclusions

This paper studied how graph representation affects small language model graph classification. We compared raw adjacency lists, canonical graph texts, compact graph encodings, soft-prompt baselines, and a validation-weighted late-fusion strategy over canonical views. The results show that representation choice is a major factor in graph-as-text learning.

Canonical graph texts generally improve over raw adjacency-list serialization, indicating that reducing arbitrary node-order variation helps small language models exploit graph structure. Compact representations such as Graph6 and NetLSD provide substantial token savings while preserving part of the accuracy gain, revealing a practical accuracy–token trade-off. Finally, validation-weighted late fusion over canonical views achieves the best aggregate performance among the evaluated methods, suggesting that different canonicalizations expose complementary structural information.

Overall, the findings show that graph serialization should be treated as a core modeling decision, not merely as preprocessing. For small language models, canonical and compact graph representations offer simple, model-agnostic ways to improve graph classification accuracy and efficiency.