Table-r1: Self-Supervised and Reinforcement Learning for Program-Based Table Reasoning in Small Language Models

1. Introduction

Table reasoning (TR) [1,2] is a fundamental yet challenging task, requiring structured reasoning over the two-dimensional semantics of tabular data [3]. With the emergence of large language models (LLMs), TR has seen notable progress [4]. Existing LLM-based methods fall into two main paradigms: text-based TR (T-TR), e.g., Table-CoT [5], which treats tables as plain text and generates answers directly; and program-based TR (P-TR), e.g., Binder [6], which generates executable programs to derive answers. Subsequent studies [7,8,9] have further advanced both approaches.1

However, when moving from LLMs (e.g., GPT-4o) to small language models (SLMs, e.g., Qwen-7B) [10], performance drops significantly (Figure 1(c)), primarily due to SLMs’ weaker reasoning and text comprehension abilities. While prior efforts [11,12,13] have fine-tuned SLMs under the T-TR setting, these face inherent limits such as restricted context windows and poor numerical handling [14]. In contrast, P-TR methods naturally mitigate these issues [12,15]2. This motivates a central question: Can we design a P-TR method enabling SLMs to match or even surpass LLM performance?

Pursuing this direction, however, requires overcoming two key challenges of SLMs under the P-TR paradigm, as highlighted in Figure 1(b): (i) poor generalization to heterogeneous layouts (flat, bi-level, hierarchical headers), which impairs accurate referencing of contents via headers as arguments in programs; and (ii) reasoning inconsistency, as evidenced by high pass@16 (73.5%) but low pass@1 (47.5%) in single-shot settings. To address them, we propose Table-r1, a two-stage P-TR method designed for SLMs: In Stage 1, we introduce an innovative self-supervised learning task—Layout Transformation Inference—to enhance layout generalization by training the model to infer structural transformations from a programmatic perspective. This promotes accurate header usage and content referencing. In Stage 2, we adopt and extend Group Relative Policy Optimization (GRPO) [16] into a mix-paradigm variant that prioritizes P-TR while selectively leveraging T-TR as a fallback. The model dynamically switches strategies based on the context and prior reasoning trace, guided by several TR-specific rewards. GRPO then optimizes the model’s policy toward higher-reward completions, enhancing both robustness and adaptability. Extensive experiments across four public TR benchmarks demonstrate that Table-r1 consistently surpasses all prior SLM-based methods and achieves performance on par with LLMs.

The contributions of this paper can be summarized as follows:

• We introduce an innovative self-supervised learning task tailored for the P-TR, designed to improve SLMs’ ability to understand heterogeneous table layouts without manual annotation.
• To the best of our knowledge, we are the first to apply GRPO to TR. We further adapt it into a mix-paradigm GRPO, design task-specific reward functions, and conduct an in-depth analysis of GRPO’s effectiveness in the TR.
• Extensive experiments demonstrate that Table-r1 consistently outperforms all existing SLM-based methods and achieves performance competitive with that of LLMs across multiple TR benchmarks.

2. Related Work

2.1. LM-based Table Reasoning

Recent advances in LM-based TR follow two main paradigms—Text-based TR and Program-based TR—as illustrated in Figure 1(a) and detailed in §Section 3.1.

Text-based Paradigm. In this paradigm, the LLM takes the table and the corresponding question as input and directly generates the final answer. Chen [5] first demonstrated the potential of LLMs in TR by leveraging COT prompting [17] and ICL [18]. Building on this, subsequent works [19,20,21] introduced techniques such as parse-tree decoding, the ReAct [22], and format-aware table transformations to improve reasoning performance. However, such prompt-based methods struggle to generalize to SLMs, whose capabilities are significantly weaker than those of LLMs. Although several studies [7,11,13,23,24,25] have explored instruction tuning to bridge this gap, the performance of SLMs remains limited, primarily due to context length constraints. Truncated tables often omit critical information required for accurate reasoning. Motivated by these limitations, our work takes an alternative direction to unlock the potential of SLMs better, enabling them to excel in TR tasks.

Program-based Paradigm. This paradigm prompts the LLM to generate executable code, typically in Python, based on the provided table and question, and subsequently executes the code to derive the final answer. Binder [6] was the first to introduce the Program-of-Thought framework [14,26] to TR tasks. These methods leverage an external code executor to overcome limitations in context length and numerical reasoning commonly observed in LLMs [27,28]. As a result, P-TR methods have gained substantial traction, with follow-up works [8,29,30,31,32,33] integrating techniques such as ReAct, Tree-of-Thoughts [34], and graph-based modeling to enhance reasoning capabilities further. Notably, some methods [15,35] have adopted P-TR as the primary framework, with text-based paradigms playing a supplementary role to improve robustness and coverage. However, most of these methods remain heavily dependent on the code generation abilities of LLMs, posing significant challenges when adapting to SLMs. Although some initial efforts [12,33] have explored P-TR in the context of small models, a considerable performance gap still exists. In this work, we build upon the P-TR paradigm and propose an innovative method that enables SLMs to achieve performance comparable to that of their larger counterparts.

2.2. Reinforcement Learning for LLM-based Reasoning

Recently, a growing body of research has focused on enhancing the reasoning capabilities of LLMs. The impressive performance of o1 [36] in complex reasoning tasks has spurred the academic community to focus on optimizing models through RL. Notably, DeepSeek-R1 [37] and Kimi-1.5 [38] have both utilized advanced versions of Proximal Policy Optimization [39], resulting in significant improvements in LLM performance on verifiable tasks, such as mathematics and code generation. Among these, the GRPO [16] algorithm employed by DeepSeek-R1 has been widely validated for its effectiveness in boosting the reasoning abilities of SLMs [40,41,42]. Despite these advancements, to date, no work has explicitly applied RL-based methods to the domain of TR.

3. Preliminaries

3.1. Problem Statement

In the TR task, each data sample is formalized as a triplet $\langle t,q,a\rangle$, where $t\in \mathbb{T}$ denotes a semi-structured table, $q\in \mathbb{Q}$ represents a natural language question, and $a\in \mathbb{A}$ is the ground-truth answer derived from table t. A semi-structured table t is composed of a header H and a body section D. The header H encodes schema-level semantics and is designed for readability. Unlike flat database schemas, it may have a hierarchical layout or be split into top and left headers, especially in financial reports [43,44]. The goal of models in TR is to learn a mapping function $\varphi :\mathbb{Q}\times \mathbb{T}\to \mathbb{A}$, which outputs a correct answer $a^{\prime }$ based on the input pair $(q,t)$. In T-TR, the reasoning process is directly handled by a language model $\mathcal{M}$, which predicts the answer in a single step:

$$a^{\prime }=\mathcal{M}(q,t),$$

(1)

where $a^{\prime }$ is the model’s generated response. Alternatively, P-TR paradigm decomposes the TR into two stages: the model $\mathcal{M}$ first produces an intermediate program p, which is then executed by an external executor $\mathcal{E}$ to obtain the final answer:

$$\begin{array}{c}\hfill p=\mathcal{M}(q,t),a^{\prime }=\mathcal{E}(p,t).\end{array}$$

(2)

In our method, the program p corresponds to Python code.

4. Methodology

We propose Table-r1, a two-stage training method for enhancing TR. As illustrated in Figure 2, Stage 1 (§Section 4.1) strengthens the model’s understanding of tabular layouts and semantics; Stage 2 (§Section 4.2) enhances code-based reasoning via reinforcement learning.

4.1. Stage 1: Self-Supervised Learning Tailored for Program-based Table Reasoning

Real-world tables often deviate from standard formats, exhibiting diverse layouts—e.g., flat, hierarchical, bi-directional headers—that hinder semantic parsing and accurate indexing.

To mitigate this issue, we propose a self-supervised task (SSL) task called Layout Transformation Inference (LTI), which trains the model to compare two tables—an original table t and its transformed counterpart $t^{\prime }$—and generate a program ${\tilde{p}}_o$ that reconstructs $t^{\prime }$ from t:

$$t^{\prime }=\mathcal{E}({\tilde{p}}_o,t),$$

(3)

where $\mathcal{E}$ is a program executor. LTI demands fine-grained comparison between t and $t^{\prime }$ to identify layout changes and express them via executable transformations. This encourages the model to distinguish headers from data and use headers as structural anchors. The task is non-trivial—GPT-4o achieves only 45% accuracy in ICL, highlighting the difficulty and potential for improving the model’s understanding ability of layouts without manual annotation.

4.1.1. Training Data Synthesis

Synthetic Instance Construction To automatically generate target tables, we define a set of transformation operations $\mathcal{O}=\{o_1,o_2,\dots ,o_K\}$, where each $o_i$ is a deterministic function modifying the layout of a table t (e.g., row swapping, column deletion; see Appendix D for details). For each t, we apply a random sequence of n operations ($1\le n\le 3$) to generate a transformed table $t^{\prime }$:

$$\begin{array}{c}\hfill p_o=o_n\circ \dots \circ o_1,t^{\prime }=\mathcal{E}(p_o,t),\end{array}$$

(4)

where $p_o$ represents the composition of applied operations and serves as the supervision labels.

LLM-Based Label Rewriting Directly using template-based programs $p_o$ in SSL can introduce distributional mismatch, causing catastrophic forgetting [45]. To mitigate this, we adopt a continual learning strategy based on self-distillation [46]. Specifically, the base model ${\mathcal{M}}_{\mathrm{base}}$ rewrites the raw synthetic program $p_o$ into a semantically equivalent but more fluent version ${\tilde{p}}_o$:

$${\tilde{p}}_o={\mathcal{M}}_{\mathrm{base}}(t,t^{\prime },p_o),$$

(5)

which serves as the final training label.

4.1.2. Fine-Tuning Procedure

We fine-tune the model on synthesized triplets $(t,t^{\prime },\tilde{p}o)$ using the following objective:

$${\mathcal{L}}_{\mathrm{LTI}}={\displaystyle \frac{1}{\left|D\right|}}\sum _{\begin{array}{c}(t,t^{\prime },{\tilde{p}}_o)\in D\\ \mathcal{E}({\tilde{p}}_o,t)=t^{\prime }\end{array}}\left[-logP({\tilde{p}}_o\mid t;t^{\prime };\theta )\right],$$

(6)

where D is the training set and $\theta$ the model parameters. The loss is computed only when ${\tilde{p}}_o$ faithfully transforms t into $t^{\prime }$.

4.2. Stage 2: Reinforcement Learning for Table Reasoning

We enhance code-based reasoning via RL. GRPO [37] is particularly well-suited for SLMs due to its efficiency and strong performance on verifiable tasks, outperforming traditional methods such as PPO [39]. Meanwhile, prior work [15,35] shows that T-TR can complement P-TR under specific contexts. Motivated by these insights, we integrate a mix-paradigm variant of GRPO into Table-r1, enhancing P-TR consistency while allowing dynamic fallback to T-TR when needed. We now detail the components: completion templates (§Section 4.2.1), distillation-based cold start (§Section 4.2.2), reward functions (§Section 4.2.3), and the mix-paradigm GRPO (§Section 4.2.4).

4.2.1. Completion Templates

As shown in Figure 2, we adopt two structured output formats. Both prompt the model to generate intermediate reasoning before deciding on an answer format. If the P-TR template is selected, the answer is enclosed in <code_solution> tags containing Python code; otherwise, a direct answer appears within <answer> tags. This design, inspired by [47], encourages deliberate reasoning and supports flexible strategy selection. Examples are provided in Appendix E.3.

4.2.2. Cold Start with Distilled Data

Inspired by [40], we adopt a distillation-based cold start strategy before RL. We employ LLMs [48] as teacher models to generate reasoning traces and code solutions on the training split, following the prompt format in Appendix E.2. The distilled data is then used to perform SFT on the base model.

4.2.3. Reward Functions

Reward design is central to RL [40]. In GRPO, reward functions evaluate each model-generated completion, guiding optimization toward better outputs. Formally, we define:

$$R\left(c_i\right)=\sum _j{R}_j(c_i,t,a),$$

(7)

where $R_j$ denotes the j-th reward component, and $R\left(c_i\right)$ is the total reward for completion $c_i$.

Beyond format-adherence reward functions (see Table A2 for more details), the following reward functions $R_j$ are introduced to encourage more reliable and interpretable generation:

• Compilation Correctness. Program-based outputs receive the reward only when the code compiles successfully. Text-based outputs are assumed valid by default, which introduces a bias that is counteracted by other reward terms.
• Answer Correctness. For P-TR outputs, we execute the generated code to obtain the answers; for text-based outputs, we use regular expression matching. To refine reward granularity, we compare model outputs with golden answers using similarity metrics. Incorrect but compilable outputs (e.g., print(final answer)) are penalized heavily. We also penalize the use of text-based answers on questions requiring P-TR (e.g., extremum queries over truncated tables).
• Short Code Penalty. To discourage degenerate behaviors (e.g., overly short programs), we impose penalties on code completions that are below a specified length threshold.

4.2.4. Mix-paradigm GRPO

We extend GRPO to a mix-paradigm variant, allowing the model to prioritize P-TR while flexibly falling back to T-TR when appropriate. GRPO optimizes a policy by maximizing the relative advantage of high-reward completions, regularized against a reference policy. Given a query-table pair $\langle q,t\rangle$, the model generates multiple completions $c_i=[s;z^{\prime }]$, where s is a reasoning trace and $z^{\prime }$ is the final answer in a specific format. We define the output space $\mathcal{Z}=\mathrm{program},\mathrm{text}$, where each $z\in \mathcal{Z}$ denotes a distinct answer form—executable code or direct output. The model implicitly selects $z^{\prime }$ during generation, conditioned on s:

$$z^{\prime }=arg\underset{z\in \mathcal{Z}}{max}P(z\mid s;\theta ).$$

(8)

Each $c_i$ is then evaluated using a TR-specific reward function $R_{c_i}$ (defined in §Section 4.2.3). We compute its relative advantage within the batch as:

$$A\left(c_i\right)={\displaystyle \frac{R\left(c_i\right)-{\mu }_R}{{\sigma }_R+ϵ}},$$

(9)

where ${\mu }_R$ and ${\sigma }_R$ are the mean and standard deviation of rewards among the sampled completions $c_i$ in the same batch, and $ϵ$ is a small constant for numerical stability. The policy is updated by maximizing a clipped surrogate objective:

$${\mathcal{L}}_{\mathrm{GRPO}}={\mathbb{E}}_{c_i\sim {\pi }_{\theta }}\left[min\left(r\left(c_i\right)A\left(c_i\right),\mathrm{clip}(r\left(c_i\right),1-ϵ,1+ϵ)A\left(c_i\right)\right)\right]-\lambda ·\mathrm{KL}({\pi }_{\theta }\left|\right|{\pi }_{{\theta }_{\mathrm{ref}}}),$$

(10)

where $r\left(c_i\right)={\pi }_{\theta }\left(c_i\right)/{\pi }_{{\theta }_{\mathrm{ref}}}\left(c_i\right)$ is the likelihood ratio, and $\lambda$ controls the KL regularization strength. This encourages exploration of higher-reward answers while constraining divergence from the reference policy.

5. Experiments

In this section, we first present the datasets and implementation details. We then report experimental results comparing our method against strong baselines, including SOTA, to validate its effectiveness. Additionally, we explore several key findings that arise from our study, which we formulate into the following research questions:

• RQ1: For SLMs, what are the respective strengths and weaknesses of the T-TR and P-TR paradigms in handling real-world TR tasks?
• RQ2: How does each module contribute to the overall performance gains in TR, and in which scenarios does mix-paradigm GRPO demonstrate clear advantages?
• RQ3: What are the respective roles and impacts of SFT and RL in improving the reasoning capability of P-TR models?
• RQ4: To what extent does organizing training data by difficulty levels influence the learning efficiency and final performance of GRPO?

5.1. Experimental Settings

Datasets & Metrics & Implementation. We evaluate on four widely used TR benchmarks. Dataset statistics are summarized in Table 1. For all datasets except AIT-QA, we use the original train-test splits; AIT-QA is randomly split 8:2 based on questions. Following prior work [6,15], we adopt self-consistency (SC) with majority voting, using 5 samples per prediction, and report exact match accuracy as the evaluation metric. Implementation details can be found in Appendix C.

Baselines. We compare against recent TR methods (see Table 2), covering both text-based and program-based paradigms. Additionally, we evaluate the text-based and program-based performance of SOTA LLMs (GPT-4o [51], DeepSeek-v3 [48], DeepSeek-R1 [37]) and base models (Qwen2.5-Coder-7B [52], LLaMA3.1-8B-Instruct [53]) under the same prompt settings with SC=5 as in Table-r1.

5.2. Overall Performance

Table 2 reports results of Table-R1 against all baselines across four TR benchmarks. Our method consistently outperforms all SLMs of comparable scale and matches the performance of top-tier LLMs. Several key observations emerge: (i) For LLMs, P-TR performs better on flattened tables (e.g., WTQ, TabFact), whereas T-TR is more effective on hierarchical tables (e.g., HiTab, AIT-QA); see § Section 5.3 for analysis. (ii) Models like Table-R1 and v3, which operate without external plugins, achieve the best results—highlighting the critical role of the base model’s capacity in TR. That said, proper strategies still provide substantial gains, even for smaller models.

5.3. Error Studies (RQ1)

To analyze the limitations of the two paradigms (RQ1), we manually inspect 200 instance outputs each from WTQ and HiTab using Qwen2.5-7B-Instruct (text-based) and Qwen2.5-Coder-7B-Instruct (program-based). The program-based model is restricted to the first 10 rows due to token limits, while the text-based model processes the entire table (truncated if exceeding 4096 tokens). We follow the classification scheme in [15,58], assigning each prediction to one of six error categories:

• Missing Table Information (MTI): The answer-relevant content is truncated and unavailable to the model.

• Information Extraction Errors (IEE): Caused by structural heterogeneity, manifesting as hallucinations or index misuse (Examples shown in Figure A4 and Figure A5).

• Computation Errors (CMP): Errors stemming from incorrect numerical reasoning.

• Code Syntax Errors (CSE): The generated code fails to compile due to syntactic issues.

• Logical Errors in Code (CLE): The code compiles and runs but yields incorrect results due to flawed reasoning or filtering logic. (An example shown in Figure A6).

• Answer Misalignment (AM): where the output format diverges from the expected answer. CSE and CLE differ from IEE in that they assume syntactic or semantic code validity, but not necessarily accurate indexing.

As shown in Figure 3, computation errors are most frequent in text-based methods, reflecting LLMs’ known weaknesses in arithmetic [26]. These models also struggle with large tables, where high information density increases both extraction errors and the risk of answer truncation due to context limits. The program-based paradigm alleviates these issues but still suffers from syntactic and logical flaws in code generation. In particular, HiTab’s complex hierarchical indexing significantly raises the information extraction errors that occur in the program.

5.4. Ablation Study (RQ2, RQ3)

We conduct ablation studies (RQ2) on four datasets based on two base models to assess each component of Table-r1. As shown in Table 3, all modules enhance overall performance. Key findings include: (1) Cold start provides the largest gains, especially for weaker base models; (2) GRPO consistently boosts performance, with or without cold start; (3) Mix-GRPO yields notable improvements on structurally complex datasets (HiTab, AIT-QA).

To examine mix-paradigm GRPO (RQ2), we analyze output template distributions and observe a strong preference for P-TR (WTQ: 95%, HiTab: 91%), supporting our hypothesis that P-TR better suits SLMs. Case studies further show that, with mix-paradigm enabled, the model selectively switches to text-based templates when contextually appropriate (The strategy is learned during training and tends to switch to T-TR when headers are dense and the question is a simple look-up. See Figure A7).

Table 4 further investigates the effectiveness of the original SSL task—LTI—and the necessity of incorporating continual learning. As shown, using only template-based annotation (w/o rewriting) leads to catastrophic forgetting. In contrast, introducing continual learning significantly improves model performance.

To analyze the impact of RL and SFT on model performance (RQ3), we compare SC=5 and Pass@16 accuracy across five variants: Base, Origin (Base + SSL), Origin + RL, Distillation (Origin + Cold Start), and Distillation + RL. Results in Figure 4 yield three key observations: TabFact exhibits minimal change in potential due to its binary nature, where random sampling yields a high probability of correct answers. All components—SSL, distillation, and GRPO—improve both potential (Pass@16) and capability (SC=5), with SFT showing the most substantial gains. While distillation + GRPO achieves the highest capability, GRPO after distillation reduces potential. Moreover, GRPO still cannot fully convert potential into actual capability.

5.5. Further Analysis (RQ4)

Prior work [40,41] has shown that GRPO is sensitive to training data difficulty. To further investigate this (RQ4), we conduct controlled experiments on WTQ, a benchmark with substantial P-TR challenges. Using execution feedback from a distilled Qwen-7B model, we categorize training examples into three levels: Easy: code compiles and yields correct answers (7.5k examples); Medium: code compiles but yields incorrect answers (3.5k); Hard: code fails to compile (2.3k). Each subset is trained separately for 2200 steps (20 instances per step), logging two reward signals: answer correctness (max 1.5) and compilation success (max 0.75). Test set accuracy is recorded every 100 steps.

Results (Figure 5) show that compilation reward degrades with increasing difficulty, while answer correctness drops sharply even at the medium level. Throughout the process, models trained on sorted subsets underperform those trained on randomly shuffled data (76.5% dev accuracy). We also tested training only on the Easy+Medium subsets, achieving a best result of 75.5%. These findings suggest that while random mixing yields the best overall performance, training on data closer to the model’s current ability can approach optimal results when compute resources are limited.

6. Conclusion

We present Table-r1, a two-stage method that enhances SLM-based TR through an original SSL task and mix-paradigm GRPO. It effectively mitigates layout generalization and reasoning inconsistency issues in P-TR, achieving performance competitive with LLMs. Future work will explore broader RL strategies to advance TR capabilities further.

Appendix E.1. Prompts for T-TR

Figure A1. Prompts for T-TR.

Figure A1. Prompts for T-TR.

Appendix E.2. Prompts for P-TR

Figure A2. Prompts for P-TR.

Figure A2. Prompts for P-TR.

Appendix E.3. Prompts for the Mix-paradigm GRPO

Figure A3. Prompts for the Mix-paradigm GRPO.

Figure A3. Prompts for the Mix-paradigm GRPO.