The Rules of the Game: A Survey of Rubrics for Large Language Models

1. Introduction

The past few years have witnessed a remarkable transformation in the tasks that large language models (LLMs) are expected to handle. The early applications of LLMs focused mainly on tasks with relatively clear objectives and standardized input-output formats, such as question answering [1,2], instruction following [3], and code completion [4]. With the rapid development of LLM capabilities, especially in reasoning [5,6], planning [7], and tool use [8], LLMs have gradually evolved from passive answer generators into more capable AI agents. This evolution enables them to address more complex and highly open-ended tasks, including long-form report generation, professional analysis, software engineering, scientific discovery, multimodal reasoning, and long-horizon agentic tool use [8,9,10,11,12,13,14,15]. In these tasks, the model is no longer only required to produce a short and verifiable answer. Instead, it must understand complex user intents, search for information automatically, reason over multiple steps, interact with external tools or environments, and generate outputs that satisfy diverse task-specific requirements.

Challenges

This transformation in the task landscape makes both evaluation and model training more difficult. For many complex and open-ended tasks [9,16], there is no ground-truth answer, and response quality cannot be reliably assessed only by signals such as exact match, execution success, or binary correctness. In these tasks, high-quality responses need to satisfy task-specific requirements and constraints. For example, in research report generation [17], a model-generated report should comprehensively cover the user question, synthesize evidence from reliable sources, maintain factual consistency, and present the findings in a clear and well-structured manner. In agentic tool-use tasks [18], evaluation goes beyond the final answer: the intermediate trajectory should also satisfy requirements such as subgoal completion, appropriate tool selection, and execution correctness. These requirements create challenges not only for evaluating model outputs, but also for constructing reliable supervision signals for model optimization. Existing reward modeling approaches provide important foundations, but they remain limited in these complex and open-ended tasks. For example, neural reward models [19] offer scalable scalar feedback, yet they lack interpretability and generalizability in these tasks. Reinforcement learning with verifiable rewards (RLVR) [5,20] has achieved strong progress in domains such as mathematics and code generation; however, it is limited to tasks whose answer correctness is verifiable and does not apply to open-ended tasks. LLM-as-a-judge [21] provides a more flexible alternative, but unconstrained judging prompts may lead to unstable, unreliable, or biased assessments. These limitations point to a common missing component: structured and explicit evaluation criteria that specify the requirements a response should satisfy.

Why Rubrics Matter

Rubrics provide a natural way to fill this gap. Originating from educational assessment [22,23], rubrics decompose quality evaluation into a set of explicit criteria, each corresponding to a concrete aspect of the desired output. The lower part of Figure 1 shows concrete examples of rubrics in various tasks. In the context of LLMs, rubrics are closely related to checklists, evaluation dimensions, criteria, and grading guidelines, but they emphasize a more structured representation of quality standards. Rather than asking a judge or reward model to produce a single holistic score, a rubric-based evaluation assesses model outputs along multiple dimensions and then aggregates these fine-grained judgments into an overall assessment. This structure is especially useful for open-ended tasks, where different responses may succeed or fail in different ways. By making evaluation standards explicit, rubrics improve the transparency, controllability, and diagnostic value of model assessment, while also offering a practical interface for transforming human preferences and task requirements into supervision signals of LLM training.

In recent years, rubrics have received increasing attention in the LLM community and have been applied to a broad range of complex tasks. Rubric-based methods have shown strong potential in evaluating and improving model capabilities in scenarios such as deep research report generation [17], medical question answering [16], professional reasoning [11], multimodal generation [24], and agentic task solving [13]. Meanwhile, the target trained models have also evolved from instruction-following LLMs [25,26] to reasoning LLMs [27,28] and tool-using AI agents [18,29]. Given the growing significance of rubrics and the rapid expansion of related studies, a systematic and up-to-date discussion is needed to summarize this emerging research area, which motivates this paper. In this paper, we extensively review recent studies on rubrics and organize them through systematic classification and summarization, as shown in the upper part of Figure 1. Our goal is to help readers better understand what rubrics are and how they can be used for model training and evaluation. Specifically, this paper aims to address the following key questions:

To address question ❶, we first define rubrics in sec:preliminary and compare them with related concepts, including reward models, verifiable rewards, and LLM-as-a-judge. Question ❷ is discussed in sec:rubricsconstruction, where we review four major approaches to rubric construction: direct generation, contrastive generation, iterative refinement, and online or co-evolving generation. Question ❸ is addressed in sec:rubricsfortraining, where we discuss how rubrics are used to train policy models and reward models based on different training mechanisms. Question ❹ is covered in sec:rubricsforevaluation, where we summarize rubric-based evaluation for both general and domain-specific tasks. The papers discussed in Section 3, Section 4 and Section 5 are illustrated in Figure 2. Finally, question ❺ is discussed in sec:discussion, where we identify open challenges and promising directions for future rubric-based research.

Contributions

The contributions of this paper can be summarized as follows:

• Comprehensive overview and unified taxonomy of rubrics for LLMs. This work systematically synthesizes rubric-related research for LLMs and organizes existing studies within a unified taxonomy. By reviewing recent advances in this emerging area, we provide readers with a structured understanding of what rubrics are, how they are constructed, and how they are used for model training and evaluation.
• Systematic taxonomy and in-depth analysis of rubric construction. We systematically summarize existing methods for rubric construction and organize them into four major categories: direct generation, contrastive generation, iterative refinement, and online or co-evolving generation. For each category, we provide an in-depth analysis of its input signals, generation mechanisms, refinement strategies, and representative applications.
• Comprehensive review of rubrics for model training and evaluation. We extensively review how rubrics are applied to model training and evaluation. For model training, we categorize existing methods according to their roles in policy model training and reward model training based on different training mechanisms. For evaluation, we introduce rubric-based evaluation benchmarks for general tasks and domain-specific tasks.
• Discussion of open challenges and future directions. We provide an in-depth discussion of key open challenges in rubric-based research. These discussions highlight the current limitations of rubric-based methods and outline promising directions for future research.

2. Preliminaries: Formalizing Rubrics

2.1. Reward Modeling in Reinforcement Learning

Reinforcement learning (RL) has become a central paradigm for aligning LLMs with desired behaviors. During the RL process, reward modeling is a key component that defines how model outputs are evaluated and optimized. Given an input and a generated response, the reward function measures the quality of the response and provides feedback to guide policy updates. Therefore, the design of the reward signal plays a critical role in determining what behaviors the model ultimately learns.

Reward Models and Verifiable Rewards.

Existing approaches construct reward signals in different ways. One common approach is to train neural reward models [19] that approximate human preferences from annotated data, typically derived from pairwise comparisons. These models assign scalar scores to candidate responses and are widely used to guide policy optimization. Another line of work relies on Reinforcement Learning from Verifiable Rewards (RLVR) [5,20], where reward signals are obtained through objective verification procedures. This paradigm leverages tasks for which solutions may be challenging to generate but can be reliably checked, such as mathematical problem solving or code generation, where correctness can be determined by matching ground-truth answers or executing test cases.

However, both approaches face limitations when applied to open-ended tasks such as deep research report generation [9,10] whose responses are very long and difficult to verify. Reward models often provide coarse scalar rewards, which fail to capture the diverse aspects of open-ended responses and offer limited interpretability. In addition, they require enough annotated preference data for training and are prone to reward hacking [107]. RLVR, on the other hand, is inherently limited to tasks with verifiable outcomes and does not readily extend to open-ended and non-verifiable tasks that lack explicit ground truth and require multi-dimensional evaluation.

LLM-as-a-Judge.

More recently, LLM-as-a-judge [21,108] has emerged as a flexible alternative, where large language models are prompted to directly evaluate the responses. While this approach reduces the need for explicit reward modeling and improves adaptability across tasks, it relies on underspecified judging prompts, where the underlying evaluation criteria are implicitly encoded in the model rather than explicitly specified. As a result, it often produces unreliable and inconsistent judgments [109].

Limitations of Existing Reward Modeling.

Despite their differences, existing reward modeling approaches share several fundamental limitations. First, these approaches typically produce a single scalar reward without explicitly modeling multiple evaluation dimensions (such as factual correctness, coverage, and credibility), making them difficult to generalize to open-ended tasks or non-verifiable domains. Second, the resulting reward signals are often not interpretable, as evaluation criteria are implicitly encoded in model parameters or prompts, making it difficult to understand or control what aspects of the response are being optimized. These limitations highlight a fundamental gap in existing reward modeling: the lack of structured and explicit evaluation criteria. To address this, it is desirable to decompose response evaluation into multiple explicit dimensions. This motivates the use of rubrics, which provide a structured framework that evaluates the response along different dimensions.

2.2. What Are Rubrics?

Definition.

In educational assessment, a rubric is a scoring guide. It tells evaluators what aspects of an answer should be assessed, and what stronger or weaker performance looks like on each aspect. In this way, evaluation standards that might otherwise remain implicit are made explicit, easier to communicate, and more consistent across scoring and feedback [22,23]. With the rise of LLMs, rubrics have received renewed attention. In the LLM setting, they are also often referred to as checklists. In general, the idea is simple: a reviewer (either a human or an LLM judge) examines an output item by item according to a set of evaluation points, and then combines these judgments into an overall score or a structured report [25].

For the sake of unifying later discussion across both evaluation and training, we define a rubric as a set of rubric items:

$$\mathcal{R}={\left\{(d_j,w_j)\right\}}_{j=1}^k,$$

(1)

where $d_j$ is the natural-language description of the j-th rubric item, and $w_j\in \mathbb{R}$ denotes its weight or importance. Let x denote the input or task, and let y denote the model output. For each rubric item, the judge assigns a score

$$c_j(x,y)\in [0,1],$$

(2)

which indicates how well the output satisfies that item.

A simple aggregated rubric score can then be written as

$$S_{\mathcal{R}}(x,y)={\displaystyle \frac{{\sum }_{j=1}^k{w}_j{c}_j(x,y)}{{\sum }_{j=1}^k{w}_j}}.$$

(3)

This normalized form is convenient because it remains comparable across tasks with different numbers of rubric items or different weighting schemes.

This item-by-item scoring scheme is essentially the same structure used when rubrics are turned into reward signals in reinforcement learning (RL) and model evaluation. In RL training, the rubric becomes part of the optimization objective [30,110], while in evaluation, it serves as a scoring function [16]. More broadly, rubrics can be divided into two common types. A holistic rubric assigns one overall score to the output, whereas an analytic rubric breaks evaluation into multiple rubric items and scores them separately. For open-ended LLM tasks, analytic rubrics are often more useful. They make it easier to identify which aspect of an output fails, and they provide more fine-grained signals for both evaluation and training [60,110].

Weighting and Aggregation Methods.

Once multiple rubric items are scored, the next question is how to combine them into a single score. In practice, the most common aggregation strategies can be grouped into three simple types.

The first is direct averaging or summation. In this setting, all rubric items are treated equally, and the final score is obtained by directly summing or averaging the item-level scores:

$$S_{\mathrm{avg}}(x,y)={\displaystyle \frac{1}{k}}{\displaystyle \sum _{j=1}^k}{c}_j(x,y),$$

(4)

or equivalently by using the unnormalized sum

$$S_{\mathrm{sum}}(x,y)={\displaystyle \sum _{j=1}^k}{c}_j(x,y).$$

(5)

This is the simplest and most transparent approach. It is easy to implement and interpret, and is often used when all rubric items are considered equally important.

The second is weighted summation. Here, different rubric items are assigned different levels of importance, and the final score is computed as a weighted combination:

$$S_{\mathcal{R}}(x,y)={\displaystyle \frac{{\sum }_{j=1}^k{w}_j{c}_j(x,y)}{{\sum }_{j=1}^k{w}_j}}.$$

(6)

Compared with direct averaging, weighted summation is more flexible because it allows critical dimensions to contribute more to the final score. This is especially useful when some rubric items, such as safety or task completion, should be emphasized more than others.

The third is implicit aggregation. Instead of first scoring each rubric item separately and then combining them, one may provide the full rubric and the model output to a judge model, and directly ask it to produce a single overall score:

$$S_{\mathrm{imp}}(x,y)=f_{\varphi }(x,y,\mathcal{R}),$$

(7)

where $f_{\varphi }$ denotes the judge model. This approach is often simpler at inference time and may reduce evaluation cost. However, it is less transparent, because the contribution of each rubric item is no longer explicitly observable.

In this paper, we mainly focus on explicit aggregation, including direct averaging and weighted summation, because these forms make the evaluation process easier to inspect and analyze. Implicit aggregation is still important as a practical alternative, especially when efficiency is a primary concern.

2.3. Comparing Rubrics with Other Key Concepts

Rubrics are often discussed together with other related concepts such as LLM-as-a-judge, reward models, and RLVR. However, these concepts operate at different levels. Rubrics mainly specify what standards an output should be judged by. The other concepts are more about who performs the judgment or how that judgment is turned into a training signal. For this reason, rubrics are best understood as an explicit evaluation framework rather than a specific model or learning algorithm.

2.3.1. Rubrics vs. LLM-as-a-Judge

The difference between rubrics and LLM-as-a-judge is the difference between the evaluation standard and the evaluator. LLM-as-a-judge refers to using an LLM to score, compare, or judge outputs [111,112]. A rubric, by contrast, specifies what should be judged: for example, factuality, completeness, style, or safety. An LLM judge may give a single overall score without using a rubric, or it may follow a rubric and score the output item by item. The latter is usually more informative because it shows not only the final score but also where the output succeeds or fails.

2.3.2. Rubrics vs. Reward Models

Rubrics also differ from reward models. A reward model usually outputs a scalar score directly, with the notion of quality implicitly encoded in its parameters. A rubric makes the scoring standard explicit by listing the rubric items and evaluating them one by one. In this sense, a reward model is more like a black-box scorer, while a rubric is more transparent and easier to inspect or edit [60]. Researchers can revise rubric items, adjust their weights, and analyze which dimensions a model performs poorly on. At the same time, the two are not mutually exclusive: rubrics can be used to supervise or construct reward models, which will be discussed in Section 4.2.

2.3.3. Rubrics vs. RLVR

Rubrics relate differently to RLVR (reinforcement learning with verifiable rewards). RLVR is designed for settings where correctness can be checked automatically, such as mathematical problem solving or code generation [113]. These tasks often have a clear target answer, so reward assignment is straightforward. Many open-ended tasks, however, do not have a single correct answer. Writing, dialogue, planning, and style control are typical examples. In such cases, rubrics are useful because they provide structured reward signals through multiple explicit rubric items, instead of relying on one fully verifiable target.

2.3.4. Summary

These concepts can therefore be distinguished in a simple way: rubrics specify what standards to evaluate against; LLM-as-a-judge specifies who performs the evaluation; reward models specify how to output a score directly; and RLVR specifies how rewards can be assigned through automatic verification. The value of rubrics is not that they replace these concepts, but that they provide a clear and interpretable intermediate layer that can support both evaluation and training.

3. Rubrics Construction

The quality of rubrics is the foundation of their downstream applications including model training and task evaluation, which will be discussed in Section 4 and Section 5. Logically prior to both applications, this section focuses on the following question: how are rubrics themselves constructed? We review methods that explicitly propose, decompose, filter, compress, or dynamically update rubrics. A summary of representative methods is provided in Table 1.

We organize existing methods into four paradigms (as illustrated in Figure 3) that form a clear evolutionary trajectory: direct generation (Section 3.2) →contrastive generation (Section 3.3) →iterative refinement (Section 3.4) → online and co-evolving generation (Section 3.5). This trajectory is driven by progress along two coupled dimensions: the conditioning signal fed to the generator (from a bare query, to preference pairs, to full rollout histories) and the quality control mechanism applied after generation (from none, to verification and structural decomposition, to online co-evolution with the policy). Each subsequent paradigm enriches the conditioning signal, strengthens quality control, or both. For example, contrastive generation upgrades the signal from a single response to preference pairs; iterative refinement adds verification and decomposition on top of either signal; and online generation further extends the signal to rollout trajectories while making quality control continuous. Before discussing these four paradigms, we introduce a unified analytical framework (Section 3.1) that makes the comparison concrete.

3.1. Definition of Rubrics Construction

The rubric construction process can be formalized as follows: given an input context $\mathcal{C}$ (which may include a query q, one or more candidate responses, supporting evidence, or response preference pairs), a rubric generator $\mathcal{G}$ produces a set of evaluation criteria $\mathcal{R}=\mathcal{G}\left(\mathcal{C}\right)$, where each criterion $r_k\in \mathcal{R}$ specifies a concrete and assessable quality dimension. To compare methods on equal footing, we characterize each method along three dimensions: what it takes as input (conditioning signal), what it produces (output form), and what properties the produced rubrics are expected to satisfy (quality requirements).

Conditioning Signal.

The richness of the input context $\mathcal{C}$ fundamentally determines which types of rubrics can be generated. The types of input context $\mathcal{C}$ can be roughly divided into the following four categories:

• Query only (q): The rubrics are generated solely from the query along with the task description.
• Query + answer ($q,y$) or query + evidence ($q,\mathcal{E}$): rubrics are generated based on a high-quality answer or supporting evidence.
• Query + preference pair ($q,y^{+},y^{-}$): rubrics are derived by contrasting high-quality and low-quality responses, thereby highlighting discriminative evaluation dimensions.
• Policy rollouts during training: rubrics are derived from, or updated against, the trajectories produced by the policy itself, so that the criteria reflect behaviors actually exhibited during training.

Output Form.

Existing methods produce rubrics in several common forms: binary atomic rubrics where each criterion gives a yes/no judgment [25,30]; hierarchical rubrics where coarse dimensions are split into finer sub-criteria [40,42]; positive and negative rubrics that separately describe desired qualities and typical failure modes [29,60]; nugget-as-rubric, where atomic facts extracted from evidence are used directly as rubric items [41]; and query-agnostic themes or meta-rubrics that apply across instances rather than to a single query [26,38].

Quality Requirements.

A high-quality rubric set must simultaneously satisfy several requirements: discriminativeness demands that rubrics reliably distinguish high-quality from low-quality responses; coverage requires rubrics to span all critical evaluation axes of a task; atomicity and verifiability ensure each criterion is sufficiently fine-grained to be unambiguously assessed; alignment requires rubric directions to be consistent with true quality preferences; redundancy and correlation control prevents overlapping criteria from causing duplicated scoring and noise amplification; and dynamic robustness requires rubrics to remain effective as the evaluated policy model continues to evolve.

Within this framework, improvements in subsequent methods can be understood as addressing three shortcomings of naive direct generation: insufficient conditioning signals: the input does not contain enough information for the generator to identify what really matters; lack of quality control: no mechanism verifies whether the produced rubrics are discriminative, atomic, or non-redundant; static assumptions: the rubric set is fixed once and for all, even though the policy or task distribution it scores keeps changing. Early works that introduced rubric-like evaluation structures [96,114] demonstrated the value of explicit evaluation dimensions but did not systematically investigate how rubrics themselves should be generated. More recent work has started to treat rubric generation as a standalone research problem, with growing attention to its underlying conditioning signals, quality control mechanisms, and structural design.

3.2. Direct Generation

The most fundamental paradigm for rubric construction is direct generation: given a query q (optionally accompanied by a candidate answer y or supporting evidence $\mathcal{E}$), a strong LLM produces a set of evaluation rubrics in a single prompt. This paradigm is enabled by the strong instruction-following and abstract summarization capabilities of modern LLMs, which can infer appropriate evaluation dimensions from a task description and a candidate response. Direct generation offers three key advantages: it requires no preference pairs or pre-trained rubric generators, making cold-start deployment straightforward; it scales readily to new tasks and domains; and it naturally produces instance-specific evaluation criteria tailored to each query.

RaR [30] is a representative work in this paradigm. Given a query and candidate answer, it prompts an LLM to generate multiple self-contained binary rubrics in a single pass, each covering an independent quality dimension with an associated importance weight, and subsequently aggregates rubric scores into a scalar reward for RL training. The key contribution of this work is the formal establishment of instance-specific rubrics as an explicit research object. RLCF [25] takes a complementary approach by extracting checklists directly from instructions, demonstrating that effective rubric generation does not require pairwise preferences. A well-designed prompt over the instruction alone can produce evaluation criteria superior to monolithic reward models. CARMO [31] extends the direct generation paradigm by dynamically generating context-aware criteria for each query to serve downstream reward models, indicating that even within the one-shot setting, context adaptability has become a recognized concern.

Beyond methods that generate rubrics for policy model training, several evaluation benchmarks also adopt the direct generation approach to construct task-specific rubrics. WildBench [32] employs task-specific checklists for evaluation on real user tasks. SedarEval [33] proposes self-adaptive rubrics that match each question with structured scoring and deduction rules, representing an early effort in automated question-specific rubric construction. WritingBench [34] derives evaluation criteria directly from writing instructions. HealthBench [16] and PaperBench [95] rely on human experts to write rubrics: the former uses physician-authored rubrics for medical conversations, while the latter constructs expert-designed hierarchical rubrics for evaluating research replication.

Despite its simplicity and scalability, direct generation exhibits several structural shortcomings that motivate more sophisticated approaches. Coverage depends entirely on what the LLM can enumerate in a single call, with no mechanism for detecting omissions or filling gaps, and the granularity of generated criteria is often inconsistent: some are overly abstract (e.g., “the response should be helpful”) while others are excessively specific. Most critically, generating rubrics from a single response provides no contrastive signal: without simultaneously observing both high-quality and low-quality responses, the model cannot reliably tell whether a rubric genuinely discriminates between response qualities, leading to rubrics that are correct but vacuous. These observations suggest a simple solution: conditioning rubric generation on explicit preference pairs so that the generator can identify the aspects that distinguish good responses from bad ones.

3.3. Contrastive Generation

Relying solely on task instructions to generate evaluation rubrics makes it difficult to identify which dimensions are truly critical for distinguishing response quality. An intuitive solution is to leverage human-annotated preference pairs. Such data is relatively easy to obtain because many deployed applications already provide thumbs-up/down buttons or side-by-side comparison interfaces that continuously collect user preferences at scale, yielding a rich and naturally growing source of contrastive supervision. When the input is extended from $(q,y)$ to $(q,y^{+},y^{-})$, namely a query paired with chosen and rejected responses, the generator can directly target the question “why is response A better than response B?” to extract evaluation dimensions. Pairwise preferences thus provide a natural discriminative signal for rubric generation, fundamentally addressing the discriminativeness deficit identified in direct generation.

Existing contrastive methods vary widely in how they exploit preference signals, from direct prompting to learnable generation models. At the most basic level, direct contrastive prompting feeds a preference pair $(q,y^{+},y^{-})$ to an LLM and asks it to enumerate the dimensions on which $y^{+}$ outperforms $y^{-}$, e.g., factual correctness, depth of reasoning, or instruction adherence; these dimensions are then assembled into a rubric set. CDRRM [36] extends this idea with a two-stage “Contrast-then-Synthesis” pipeline: it first analyzes the main differences between the two responses across multiple aspects, and then summarizes these differences into clearer and more reusable rubrics. This shows that effective rubric construction requires more than simply listing observed differences.

However, generating rubrics in a single call from one preference pair has a clear weakness: the produced rubrics are often tied to the specific differences of that pair and do not generalize to other instances of the same task. Moreover, in the high-reward tail region where quality gaps between responses become subtle, directly generated rubrics often lack sufficient discriminative power [56]. These limitations motivate a second family of methods that adds discriminative filtering as a post-generation quality gate: candidate rubrics are first generated and then evaluated on held-out preference pairs, retaining only those criteria that demonstrably distinguish chosen from rejected responses. OpenRubrics [35] synthesizes rubrics at the per-criterion level from $(q,y^{+},y^{-})$ and then filters them by checking that the kept rubrics give consistent and discriminative scores on held-out pairs. Proxy-GRM [39] closes the loop further: a proxy model uses the generated rubrics to predict preferences on held-out pairs, and prediction accuracy is fed back as a signal to improve the rubric generator.

To improve the generalizability of rubric generation beyond individual instances, a third line of work adopts generator learning, which distills the contrastive rubric generation process into dedicated models or aggregates instance-level results into query-agnostic general principles. For example, MaMs [37] and Auto-Rubric [38] train a specialized rubric generator using preference pairs. This shift from prompt-based extraction to learnable generators is important because it allows rubric generation patterns learned from existing data to be reused across tasks and domains, rather than generating rubrics based on a general-purpose LLM for each new instance.

Contrastive generation thus equips rubric construction with its first principled source of discriminative signal, but two issues remain unsolved and motivate the next paradigm. First, pairwise signals emphasize what differs rather than what matters: the most salient dimension of difference between two responses may not correspond to the most critical evaluation axis of the task. Second, without additional verification, decomposition, and consolidation, the resulting rubrics tend to remain pair-specific rather than forming a coherent, systematic evaluation structure for the task. The next subsection therefore turns to methods that explicitly target these two issues by treating rubric construction as an iterative refinement problem.

3.4. Iterative Refinement

Although direct and contrastive generation provide scalable ways to produce rubrics, the generated criteria are often imperfect in structure and reliability. They may be too coarse-grained to support stable judgment, fail to cover important evaluation aspects, overlap with one another, or remain insufficiently verifiable. Therefore, a line of work treats rubric construction not as a one-shot generation problem, but as an iterative refinement process that improves the quality of an initially generated rubric set before it is used for evaluation or training. These observations have motivated a body of work that focuses on iteratively improving rubric quality with respect to the desiderata introduced in Section 3.1.

Concretely, existing efforts are organized around three complementary quality dimensions: (1) discriminativeness: whether individual rubric items genuinely distinguish response quality; (2) atomicity and coverage: whether rubrics are sufficiently fine-grained and span all critical evaluation axes; and (3) redundancy and correlation control: whether the rubric set is compact, non-redundant, and transferable. The following three subsubsections address each dimension in turn.

3.4.1. Verification-Driven Refinement

A growing consensus in the community holds that rubric quality should not be judged solely by whether rubrics look like evaluation criteria; rather, the critical test is whether they can reliably discriminate between high-quality and low-quality responses in actual comparisons. This recognition has given rise to a propose-evaluate-revise paradigm: candidate rubrics are first generated, then tested by a verifier or judge for discriminative power, and those that fail are iteratively revised.

For example, Auto-Rubric [38] instantiates this loop by testing each candidate rubric on held-out preference pairs to check whether its scores systematically favor chosen over rejected responses; rubrics that fail this check are sent back to the generator and revised, and the generate-verify-revise cycle repeats until the rubric set reaches a target discriminative threshold. (Its downstream compression of the verified rubrics into transferable “Theme-Tips” will be discussed separately in Section 3.4.3.) OptimSyn [46] pushes verification one step further by replacing preference-based checks with downstream training utility: it uses an influence-estimation procedure to quantify each synthetic sample’s contribution to the target model’s objective, and treats this signal as the RL reward for a rubric generator, showing that rubric quality can be grounded in measurable training impact rather than surface-level preference alignment.

The significance of this paradigm is that rubric generation is no longer a one-shot text production task, but rather a generate-and-verify loop with quality control embedded in the construction process itself.

3.4.2. Structural Decomposition

Even when a rubric is discriminative, it can still be unreliable if a single item mixes several quality dimensions together. Coarse-grained rubrics make judge scores less stable and hide important quality differences under overly broad labels. This motivates structural decomposition: recursively splitting coarse rubrics into finer-grained, atomic sub-criteria.

RRD [40] provides the most systematic treatment of this problem. It identifies four fundamental failure modes of naive rubric generation including insufficient coverage, dimension conflation, directional misalignment, and inter-criterion redundancy. Furthermore, they propose a recursive decompose-filter cycle that alternates between expanding rubrics into fine-grained sub-criteria and filtering out redundant or misaligned items. Qworld [115] solves the problem from another angle: performing hierarchical criteria expansion around the implicit evaluation axes of each question, treating high-coverage criterion generation as a systematic exploration of the question space. Search-Gen-V [41] pushes verifiability to the extreme with its nugget-as-rubric approach: instead of expressing rubrics in abstract language, it converts atomic information points (nuggets) from retrieved evidence directly into rubric items, ensuring each can be directly verified. RubricHub [42] implements an automated coarse-to-fine generation pipeline that progressively refines principles into detailed rubrics with increasing discriminative difficulty, producing a large-scale high-quality rubric dataset.

Several works extend structural decomposition to specific domains: Pathak et al. [82] introduces the question-specific rubric generation for code evaluation; DeepResearch Bench II [85] employs a hybrid pipeline combining LLM extraction with expert revision to derive fine-grained rubrics from expert reports; RefGrader [81] automatically derives problem-specific rubrics for mathematical proof scenarios from reference solutions and error analyses; RubricRAG [52] introduces retrieval-augmented rubric generation, anchoring criteria in domain knowledge; and InfiMed-ORBIT [45] converts vague open-ended evaluation standards into verifiable, multi-dimensional fine-grained rubrics for medical consultation and uses them as RL rewards, demonstrating the value of rubric decomposition in high-stakes domains.

3.4.3. De-Duplication and Compression

After large-scale rubric generation, redundancy is inevitable: multiple rubrics may describe the same underlying quality dimension in different words, causing duplicated scoring and noise amplification during aggregation. More fundamentally, rubric generation should not be limited to writing criteria for individual instances, it should also discover transferable evaluation structures from local criteria.

Auto-Rubric [38] addresses this through its query-agnostic compression stage: clustering and distilling verified instance-specific rubrics into reusable “Theme-Tips”. This pipeline embodies the dual objective of rubric generation—producing per-instance evaluation standards while simultaneously learning transferable structures from local criteria. RRD [40] supplements decomposition with relevance weighting and correlation-aware filtering to control redundancy in expanded criterion sets.

An alternative structural induction approach is represented by Sanders et al. [43]. Rather than constructing rubrics from positive quality dimensions, this work induces error taxonomies from reasoning failure trajectories, demonstrating that rubrics can equally be constituted by systematic classification of frequent failure modes. CARO [44] targets a more subtle quality issue: even when rubric sets are free of redundancy, blurred boundaries between dimensions can cause judge confusion. By explicitly optimizing inter-dimension separability, this work demonstrates that high-quality rubric sets require not only comprehensive coverage and fine granularity, but also clear inter-dimension boundaries.

Taken together, these verification, decomposition, and compression techniques suggest that rubric generation is no longer treated as simple text generation, but increasingly as a problem of structural design and refinement. However, these methods still view rubric construction as an offline process: rubrics are created before policy model training and then kept fixed throughout. As the policy improves, some rubric criteria may gradually lose their ability to distinguish strong outputs from weak ones, and a fixed set of rubrics can be more easily exploited through reward hacking. This limitation points to the need for rubrics that can adapt over the policy training.

3.5. Online and Co-evolving Generation

All methods discussed above are offline: rubrics are constructed before training begins and do not change afterwards. The methods reviewed in this subsection drop this assumption and let rubrics co-evolve with the policy or its rollout distribution. We organize them by how the update is performed, which gives three increasingly autonomous categories. (i) Rollout-based heuristic updating (Section 3.5.1) keeps a fixed update rule and refreshes the rubric pool from the latest trajectories; the rubric generator itself is not trained. (ii) Online and alternating optimization (Section 3.5.2) makes the rubric generator a trainable component that is updated jointly with the policy or judge through streaming preferences or alternating RL. (iii) Self-evolving, adversarial, and memory-driven methods (Section 3.5.3) go one step further by actively probing for blind spots and proposing new rubrics for failures the system has not yet covered. The boundary between (i) and (iii) is admittedly soft—a method like DR-Tulu sits closer to (i) because its update rule is fixed—but this axis (who decides what to add to the rubric set: a fixed heuristic, a learned generator, or an adversarial discoverer) gives the cleanest separation we have found.

Online rubric generation is motivated by three main problems. First, static rubrics may become less useful as the policy improves. Criteria that were once able to distinguish strong responses from weak ones may eventually be satisfied by most candidates, making the reward signal weak. Second, fixed rubrics may lead to reward hacking. The policy may learn to match the superficial features of the rubrics without genuinely improving response quality. Third, in long-horizon agentic tasks, some failure modes only appear during policy exploration and are difficult to predict before training.

3.5.1. Rollout-Based Evolving Rubrics

DR-Tulu [29] is a foundational work in this direction. It initializes rubrics from queries and retrieved evidence, then regenerates and updates rubrics after each training round based on the latest policy rollouts. Specifically, it generates positive and negative rubrics based on the most recent trajectories, maintains a rubric buffer, and applies variance-based filtering to retain only those criteria that remain discriminative under the current policy distribution. This design reveals a key insight: for long-horizon agentic tasks, rubric generation must incorporate behavioral patterns newly exposed during training, lest the reward signal rapidly lose its guiding value.

3.5.2. Online and Alternating Optimization of Rubric Generators

A second line of work treats rubric updates not as heuristic steps but as learnable components jointly optimized with the policy or judge. Unlike the rollout-based approach above, here the rubric generator is itself trained, and updates are driven by streaming preferences or alternating RL rather than by replaying recent trajectories.

OnlineRubrics [48] continuously adjusts the rubric set in response to a stream of pairwise comparisons that arrive during training, modeling rubric selection as an online decision problem that progressively refines the set to maximize its explanatory power over the latest observed preferences—this online stream of preferences is what makes it an evolving rather than static method. Rubric-ARM [47] formalizes rubric generation as a latent action within an alternating reinforcement learning framework: the rubric generator and judge are trained in alternation, so the generator’s outputs change with each round of policy/judge updates rather than remaining fixed. Together these two works show that “co-evolving with training” need not rely on rollout replay; it can equally come from online preference streams or alternating optimization.

3.5.3. Self-Evolving, Adversarial, and Memory-Driven Rubrics

The most recent wave of research explores increasingly dynamic rubric adaptation mechanisms. SibylSense [49] maintains a rubric memory bank and employs adversarial probing to discover evaluation blind spots, triggering the generation of new rubrics targeting discovered weaknesses. OpenRS [26] introduces pairwise adaptive rubrics and meta-rubrics—higher-order criteria governing how rubrics are generated and updated—creating a self-organizing rubric ecosystem. RLCER [28] demonstrates self-evolving rubrics in chain-of-thought reasoning scenarios, iteratively refining rubrics based on their actual effectiveness in guiding reasoning policies. AutoRubric-R1V [27] distills process-level rubrics from successful reasoning trajectories in multimodal settings, demonstrating that rubrics can emerge from behavioral exemplars as well as quality contrasts. RLAC [50] dynamically discovers failure-focused rubrics through adversarial critic interaction, positioning rubric generation as a continuous failure mode discovery mechanism.

Online and co-evolving methods thus complete the evolutionary arc traced throughout this section: from single-pass text production, through discriminative extraction and offline structural optimization, to continuous co-adaptation with the training process. The central question has shifted from “how to construct high-quality rubrics offline” to “how to continuously discover quality dimensions and failure modes not yet covered by the current rubric set”. The emergence of negative rubrics, adaptive rubrics, and self-evolving rubrics reflects the community’s recognition that reward hacking and distribution shift are first-class challenges in rubric construction—challenges that static methods, however carefully engineered, cannot fully address.

3.6. Evaluation for Rubrics

After rubrics are constructed, a fundamental question naturally arises: how can the quality of rubrics themselves be quantitatively evaluated? Since downstream model training and evaluation heavily depend on rubric quality, developing reliable evaluation frameworks for rubrics has become an increasingly important research problem. Along this line, several studies have explored this problem from different perspectives.

Qi et al. [51] propose RIFT, a qualitative failure mode taxonomy designed to diagnose structural flaws in rubric composition. By categorizing failures into reliability, content validity, and consequential validity, they provide a principled way to identify issues such as subjectivity, non-atomicity, and missing criteria that are often conflated in downstream signals. Dhole and Agichtein [52] investigate the relationship between intrinsic textual alignment and extrinsic utility. They introduce permutation-invariant metrics like Rubric-BLEU to quantify semantic similarity to expert standards, while demonstrating that a rubric’s true quality must be grounded in its downstream effectiveness—specifically its ability to improve a judge’s discriminative accuracy in preferring high-quality over low-quality responses. Parikh et al. [53] examine the impact of construct clarity and the separation of evaluation dimensions. Their analysis highlights that ambiguously defined or overlapping criteria, such as conflating features between fluency and prosody, directly degrades human inter-rater reliability. Interestingly, automated models perform better on these overlapping dimensions than on the clearly defined “accuracy” construct, as temporal features are computationally easier to capture than segmental quality. This reveals that a rubric’s effective operationalization depends not solely on distinct conceptual boundaries, but also on the computational accessibility of its underlying features. Mittal et al. [54] introduce the TRACE framework to reveal the “implicit weight misalignment” between LLM judges and human developers. By automatically extracting criteria from preference pairs, they quantify how judges systematically deviate from human intuition across different task modalities; for instance, LLMs may overvalue functional logic in code completion while humans prioritize code clarity, indicating that rubric evaluation must verify whether a model’s internal application of criteria aligns with specific task contexts.

4. Rubrics for Model Training

After rubric construction, one of its important application is to support policy model training. Once constructed, rubrics provide structured supervision by decomposing response quality into explicit and interpretable criteria, which can then be transformed into rewards for downstream optimization. Beyond policy optimization, rubrics have also been increasingly adopted in reward model training, where they are used to provide more stable and reliable supervision signals for policy learning. This section introduces these two directions separately: Section 4.1 focuses on policy model training, while Section 4.2 discusses reward model training.

4.1. Policy Model Training

Rubrics have emerged as a principled and interpretable way to define reward signals for reinforcement learning (RL) in policy model training. Instead of relying on opaque reward models or rule-based scoring functions, rubrics explicitly decompose evaluation into a set of human-understandable criteria, enabling fine-grained supervision for complex, open-ended tasks. A standard approach is to use rubrics to score model-generated responses and aggregate these scores into a scalar reward, which is then used to optimize the policy via RL algorithms such as PPO [117] or GRPO [118].

Recently, a growing number of works have begun to explore how to leverage rubrics for more effective policy model training in RL. Broadly, these works can be categorized into two main directions: (1) advanced reward design, and (2) Rubrics as Policy Guidance. The former focuses on addressing the limitations of naive scalar aggregation of multi-dimensional rubric scores, aiming to provide more stable, expressive, and reliable reward signals for policy model training. The latter, in contrast, seeks to move beyond treating rubrics as purely post-hoc evaluation tools, and instead leverages them as prior input or generation guidance during policy training to steer exploration toward higher-quality rollouts, thereby improving both the upper-bound performance of the policy and the convergence efficiency of RL. We first introduce the standard rubric-based RL paradigm and then discuss these two directions in detail. The papers mentioned in this section are summarized in Table 2.

4.1.1. Standard Rubric-based RL

The standard paradigm applies a set of rubrics to evaluate the model outputs and converts them into scalar rewards for RL training. Formally, given an input x and a set of associated rubrics ${\mathcal{R}}_x={\left\{(r_k,w_k)\right\}}_{k=1}^K$, where each $r_k$ denotes an evaluation criterion and $w_k$ its corresponding weight, the quality of a policy-generated rollout O is assessed via a rubric-based scoring function:

$$S(x,O)={\displaystyle \frac{{\sum }_{k=1}^K{w}_k·\mathrm{Judge}(r_k,O)}{{\sum }_{k:w_k>0}{w}_k}}.$$

(8)

Here, $\mathrm{Judge}(r_k,O)$ denotes a scoring function (often implemented by a judge LLM) that evaluates the extent to which the response O satisfies rubric $r_k$. A typical rubric-based RL training pipeline with GRPO is illustrated in Figure 4. Depending on the task and policy model type, rubric-based evaluation can be applied at two different levels: answer-level [25,26,30,50,55,56] and trajectory-level such as training reasoning LLMs [27,28] or tool-calling agents [29,58,116]. The aggregated rubric score $S(x,O)$ can then be used as a scalar reward signal for RL optimization, where standard algorithms such as PPO or GRPO are employed to maximize the expected reward. Alternatively, rubric scores can also be used to construct preference pairs [57], which are then used to optimize the policy model using preference learning algorithms such as DPO [119].

4.1.2. Advanced Reward Design

Reward design plays a central role in RL, as it directly determines the optimization signal that guides policy updates. In standard rubric-based RL, rewards are typically constructed by aggregating multiple rubric scores with fixed or human-designed weights into a single scalar. Such scalar aggregation is often coarse and inflexible, and is prone to reward hacking [120], leading to unstable and suboptimal policy optimization.

To address these issues, recent work has explored more stable and advanced reward aggregation strategies. RBR [59] learns aggregation weights for rubric-based rewards via pairwise preference optimization, where the weights are trained to assign higher scores to preferred responses than less preferred ones, providing a data-driven alternative to fixed heuristic weighting. Rubicon [60] proposes advanced aggregation strategies, such as veto mechanisms, saturation-aware aggregation, and pairwise interaction modeling, moving beyond linear combinations to capture non-linear interdependencies between rubric dimensions. SYNTHAGENT [18] and StitchCUDA [61] further develop structured reward formulations by integrating rubric-based signals with environment-based feedback (e.g., subgoal completion, interaction constraints, and execution requirements) into unified reward functions, along with hard constraints (e.g., veto-style gating) to mitigate reward hacking. RuCL [62] extends reward design with a curriculum-based strategy that dynamically adjusts the weights of rubrics with different difficulty levels throughout training. It stratifies rubrics into easy and hard subsets, where easier rubrics are emphasized in early stages to stabilize learning, while more challenging rubrics are gradually prioritized in later stages. Different from the weighting-based approaches mentioned above, ARL-RR [63] observes that aggregating multiple rubrics into a single scalar reward can cause the policy to lose its awareness of rewards from different rubrics, which may suppress useful learning signals. Instead of compressing rubrics, it optimizes rubrics sequentially, alternating across them during training, and dynamically selects the next rubric via a lightweight search-based strategy based on task performance.

Beyond improving rubric reward aggregation, some work focuses on designing reward signals that better align with the RL algorithms. PAPO [64] identifies that instability can arise from how reward signals are incorporated into RL algorithm GRPO, where joint normalization of outcome and process rewards may obscure useful gradients. It addresses this by decoupling these signals at the advantage level, thereby preserving effective learning signals while avoiding interference between correctness and reasoning quality. RTT [65] further improves the alignment between rubric-based rewards and RL optimization by addressing the sparsity of response-level rewards. It bridges response-level scoring and token-level policy learning by measuring the contribution of individual tokens to rubric rewards, and proposes RTT-GRPO, which combines response-level and token-level advantages within a unified framework. DRO [66] uses rubrics for rollout group-level rejection rather than direct reward aggregation in unverifiable tasks. It applies query-specific binary rubrics as feasibility gates to discard rollout groups that fail to satisfy basic task requirements, preventing GRPO from producing misleading relative advantages among uniformly low-quality rollouts.

4.1.3. Rubrics as Policy Guidance

Reinforcement learning for LLMs often suffers from inefficient exploration in complex tasks, where policies are prone to diversity collapse or become trapped in local optima [121,122]. As a result, high-quality solutions are sparse and difficult to discover through unguided sampling. In contrast, rubrics contain structured and interpretable criteria, providing explicit guidance that can steer exploration toward more promising regions of the solution space.

Building on this intuition, recent work has explored how to leverage rubrics as explicit guidance for improving exploration in reinforcement learning. RuscaRL [67] incorporates rubrics directly into the generation process as structured scaffolds, decomposing complex tasks into a sequence of rubric-aligned sub-steps. By iteratively refining outputs along different evaluation dimensions, it reduces the search space and improves credit assignment, enabling more efficient discovery of high-quality reasoning trajectories. Think-with-Rubrics [68] further internalizes rubrics into the policy’s generation process. Instead of using rubrics only as post-hoc reward criteria, it trains the model to first generate a structured rubric and then produce an answer conditioned on this self-generated rubric. During reinforcement learning, the policy is optimized with both reference-rubric consistency and self-rubric consistency, so that rubrics serve as an explicit intermediate plan that guides response generation before the final answer is produced. HeRL [69] leverages rubrics as feedback signals by converting unmet rubrics into hindsight experience, which is injected as in-context guidance to improve model outputs. This feedback-driven mechanism provides supervision on how to improve a given response and helps explore higher-quality responses beyond the current policy distribution. These approaches demonstrate that rubrics can serve not only as evaluation signals but also as structured guidance for policy optimization, transforming exploration from unguided sampling into a more directed and efficient process.

4.2. Reward Model Training

Beyond their role in policy model training, rubrics have also been increasingly adopted in reward model training, which is essential for providing scalable, stable, and cost-effective reward signals compared to directly querying LLMs during policy optimization, and also enables broader applications such as offline evaluation, filtering, and ranking. Existing studies demonstrate that rubrics can support reward model training from multiple perspectives, as illustrated in Figure 5. Specifically, they have been applied to improve the interpretability of reward models, to provide dense and informative reward signals during optimization, and to facilitate the construction of high-quality training data. These three directions highlight the diverse roles that rubrics can play in different aspects of reward model training. Representative papers mentioned in this section are shown in Table 3.

4.2.1. Rubrics for Interpretability

Traditional reward models typically produce a single scalar score as output, offering limited transparency into the underlying evaluation process. In such settings, the evaluation criteria used by the model remain implicit, making it difficult to determine which aspects of a response contribute to the final score or whether the model follows the intended evaluation standards.

Introducing explicit rubrics into reward modeling transforms the evaluation process into a structured procedure guided by predefined rubrics. Instead of relying on latent evaluation patterns, the reward model is encouraged to evaluate responses according to clearly specified rubrics, either by generating rubric-grounded reasoning to support preference judgments or by assigning structured scores to individual rubric dimensions. This structured use of rubrics improves interpretability by exposing the rationale behind preference judgments and clarifying how different evaluation criteria contribute to the final reward.

Rubric-grounded Reasoning.

In this line of work, rubrics are incorporated into reward models by requiring the model to generate rubric-grounded reasoning before producing the final preference judgment. Rubrics are provided as part of the input, guiding the model to compare candidate responses across multiple rubric dimensions and produce structured reasoning prior to outputting a final decision.

Most methods in this line rely on supervised fine-tuning using rubric-conditioned data generated by stronger teacher models, where the supervision signals typically include both preference labels and teacher-generated reasoning aligned with the rubrics. Representative works adopt this paradigm with different focuses. R3 [70] trains a reward model capable of performing interpretable scoring across diverse rubric settings, enabling consistent reasoning under heterogeneous rubrics. mR3 [71] extends this framework to multilingual settings by leveraging large-scale multilingual data, allowing rubric-grounded evaluation across different languages and rubric conditions. CDPRM [36] first deeply contrasts preference data to distill customized rubrics, and then trains reward models to generate reasoning trajectories and final preference judgments strictly grounded in these rubrics.

Beyond supervised fine-tuning, several studies further use reinforcement learning (RL) to refine rubric-grounded preference prediction. RRM [72] defines the RL reward function based on the accuracy of preference judgments. Building upon this design, OMNI-RRM [73] further incorporates format-aware rewards that encourage generated reasoning to reference all provided rubrics and include explicit comparative expressions. Going a step further, C2 [74] requires the reward model to first output a judgment of whether provided rubrics are helpful or misleading, and incorporates the correctness of this judgment into the reward signal. This incentivizes the model to learn rubric utility assessment, ultimately enabling inference-time gating that discards misleading guidelines in favor of rubric-free reasoning. DeltaRubric [76] further extends this by jointly optimizing an instance-specific planner and a grounded verifier through multi-role RL, utilizing a relative improvement reward to prioritize the generation of rubrics that successfully flip baseline judgment errors.

Rubric-Dimension Scoring.

In contrast to rubric-grounded reasoning, another line of work incorporates rubrics by assigning separate scalar scores to individual rubric dimensions and aggregating these scores into a final reward. In this paradigm, reward models are typically built upon an LLM backbone, complemented by lightweight prediction modules such as linear layers. This design enhances interpretability by making the contribution of each rubric dimension observable and explicitly revealing how different dimensions are aggregated into the final reward signal.

Representative works adopt different mechanisms to implement rubric-dimension scoring. ArmoRM [75] trains a multi-objective reward model that predicts absolute scores for multiple rubric dimensions and employs a mixture-of-experts gating mechanism to dynamically assign weights to different dimensions based on the input context. It jointly trains dimension-level scores using mean squared error (MSE) loss while optimizing the aggregated reward with Bradley–Terry (BT) loss. Critic Rubrics [123] transforms previously unused interaction trajectories into structured supervision signals by extracting multiple rubric-aligned behavioral features and jointly predicting these features alongside sparse human feedback. It trains reward models to predict dimension-level scores using regression objectives such as MSE loss. MR-RML [77] introduces a geometric projection-based formulation in which both responses and rubric dimensions are represented as vectors, and dimension scores are computed based on projection lengths along rubric directions. The aggregated reward is then optimized using preference-based objectives such as BT loss.

4.2.2. Rubrics for Reward Signals

Beyond their use for interpretability, rubrics can also be incorporated into the reward model optimization process to construct fine-grained reward signals during reinforcement learning. Traditional reinforcement learning for reward models often relies on relatively coarse supervision, such as binary correctness labels or preference agreement between responses. While these signals are effective for guiding overall preference alignment, they provide limited feedback on specific evaluation dimensions, making it difficult to enforce detailed evaluation standards or capture nuanced differences between responses. By introducing rubrics as structured reference units, reinforcement learning objectives can incorporate more informative supervision that reflects how well a model’s response aligns with predefined evaluation criteria.

Representative works explore different strategies to incorporate rubric-based signals into reinforcement learning. METAJUDGE [78] introduces atomic reasoning alignment, where human annotations are decomposed into atomic reasoning units that serve as rubric-level references. During reinforcement learning, the reward function evaluates the alignment between atomic reasons generated by the reward model and those extracted from human annotations, using this alignment score as part of the optimization signal to constrain reasoning quality and encourage consistency with rubric-defined criteria. Proxy-GRM [39] adopts a different strategy by requiring the reward model to first generate rubrics before performing analysis and producing the final judgment. To further guide rubric generation quality, the framework introduces an auxiliary proxy model that independently evaluates the generated rubrics, and incorporates its assessment as an additional reward signal during reinforcement learning. This design encourages the model to produce high-quality rubrics that can support reliable downstream evaluation.

4.2.3. Rubrics for Data Construction

Rubrics also play an important role in training data construction by guiding the creation of high-quality and informative data for reward model learning. Conventional preference datasets often contain superficial cues, such as response length, formatting style, or repeated phrasing, which reward models may exploit instead of learning to assess the substantive quality of responses. By incorporating rubrics into the data generation process, it becomes possible to construct training data that emphasizes the core dimensions of response quality, reduces reliance on surface-level heuristics, and mitigates the risk of reward hacking.

A representative method in this line is CROME [79], which automatically extracts causal rubrics that determine answer quality and generates both contrastive samples that modify only the core quality and tie samples that alter only superficial features. This intervention forces the reward model to base its evaluations on the true causal criteria rather than surface cues, improving alignment and robustness.

5. Rubrics for Evaluation

Previous sections examined how rubrics are constructed and used for model training; this section focuses on how they function as operational interfaces for model evaluation. In open-ended settings, rubric-driven evaluation translates underspecified notions (e.g., helpfulness, faithfulness, safety, and process soundness) into auditable criteria that support both scoring and diagnosis. Table 4 and Table 5 summarize representative benchmarks, and the discussion is organized by evaluation scope. We organize our discussion into two progressive dimensions: (1) general-task evaluation and (2) domain-specific task evaluation.

5.1. Rubrics for General Tasks Evaluation

General-task evaluation asks whether model outputs meet broad user expectations in settings where no single ground-truth answer exists. In this scenario, rubrics act as explicit evaluation contracts: they make implicit criteria visible, impose a uniform level of granularity on judgments, and leave an auditable trail that supports both scoring and error analysis. We categorize rubric-based general-task evaluation into four categories according to the underlying LLM capabilities being assessed:

5.1.1. Reasoning Capability Evaluation

Rubric-driven reasoning evaluation shifts emphasis from outcome-only correctness toward explicit assessment of how a model reasons. In mathematical and formal settings, MathCheck [80], SedarEval [33], and RefGrader [81] decompose reasoning into criterion-level subskills, including task understanding, intermediate validity, and final answer quality; this design enables precise failure localization. A similar paradigm appears in code generation and technical reasoning, where RubricCode [82] and TRACE [54] evaluate implementation logic, error handling, and rationale consistency beyond binary pass/fail signals. In professional and normative reasoning contexts, ProfBench [11], $1M-Bench [12], and MoReBench [83] adopt multidimensional criteria that jointly capture domain validity, analytical rigor, and value-sensitive consistency. Complementary rubric-construction work, including Qworld [115] and Search-Gen-V [41], further indicates that question-conditioned and verifiable criteria improve discriminative performance. Collectively, these studies suggest that robust reasoning evaluation depends on explicit separation between process quality and outcome quality.

5.1.2. Deep Research and Open-Ended Generation Evaluation

In long-form generation and deep-research tasks, quality is jointly determined by coverage, factual grounding, synthesis depth, and report organization. Recent work therefore focuses on compositional rubric design. For general long-form generation, HelloBench [84] and WritingBench [34] adopt hierarchical rubrics that evaluate content adequacy and discourse-level presentation along separate dimensions. In deep-research evaluation, DeepResearch Bench [17], DeepResearchBenchII [85], DEER [86], and ResearchRubrics [87] operationalize evidence-oriented criteria for recall, analytical soundness, citation-grounded argumentation, and report-level utility. The same logic extends beyond text-only outputs: MiroEval [124] adds process-and-outcome co-assessment in multimodal research scenarios, while PencilsDown [88] demonstrates rubric-style itemization in retrieval-grounded generation. At the infrastructure layer, Autorubric [38] and RubricHub [42] show that scalable verification, normalization, and compression of rubric items are central to reliable open-ended evaluation.

5.1.3. General Agent Capability Evaluation

As evaluation targets shift from static responses to interactive execution, rubric design moves from answer-level dimensions to trajectory-level diagnostics. Capability-oriented benchmarks such as AgentBoard [13] and AdaRubric [89] characterize agent behavior through explicit dimensions (e.g.planning, memory, and grounding), improving interpretability in cross-agent comparison. Tool-use benchmarks, including TRAJECT-Bench [90] and MCP-Universe [91], prioritize executable criteria over tool selection, argument validity, and dependency/order constraints in realistic ecosystems. Multi-turn assistant settings (MultiChallenge [92], Scribe [93], AstraBench [94]) extend this with process-oriented criteria for sub-goal completion, conversational consistency, and interaction reliability. Evidence from long-horizon tasks, including PaperBench [95] and DR-Tulu-style [29] evaluations, indicates that final success can mask substantial process failures, reinforcing the need for rubric designs that preserve intermediate diagnostic signals.

5.1.4. Alignment Evaluation

Alignment-oriented evaluation uses rubrics to unify heterogeneous constraints from user intent, system instruction, and safety policy within a common scoring interface. Benchmarks such as FLASK [96], InFoBench [98], AdvancedIF [99], and WildBench [32] decompose alignment into checkable dimensions over instruction adherence, utility, and response quality, improving comparability across diverse prompts and interaction settings. On the evaluator side, rubric-conditioned judging in G-Eval [97], Prometheus [114], and MT-Bench/Chatbot-Arena-style [112] protocols yields more explicit rationales and better calibration than unconstrained holistic judgments. Meta-evaluation studies, including RubricEval [125], RubricBench [126], and JudgeBench [100], show that reliable rubric execution by judges remains a bottleneck, especially on hard and fine-grained items. In safety-sensitive settings, StrongREJECT [101] further indicates that robust auditing increasingly depends on explicit, verifiable policy-oriented dimensions rather than latent preference proxies.

Across these four categories, a consistent trend emerges: evaluation criteria are becoming more explicit, process-aware, and scenario-adaptive. This shift improves score reliability and diagnostic granularity in open-ended settings, but also reveals the boundaries of general-task rubrics: they are designed for transfer and cannot fully encode domain-specific constraints, professional standards, or environment-grounded verification requirements. This limitation motivates the next subsection, which examines rubric specialization for concrete downstream domains.

5.2. Rubrics for Domain-Specific Tasks Evaluation

While general-task rubrics provide transferable evaluation interfaces, downstream applications impose domain-specific constraints that cannot be fully captured merely by broad criteria. Domain-specific task evaluation therefore emphasizes rubric specialization under concrete knowledge boundaries, safety requirements, and environmental feedback. We organize this part into two complementary views: rubric designs for intermediate trajectories and rubric designs for final outputs.

5.2.1. Rubrics for Intermediate Trajectories

As LLMs evolve into agentic systems, the evaluation paradigm is shifting from exclusively assessing static final answers to evaluating dynamic, intermediate reasoning trajectories. Trajectories in domain-specific downstream applications are highly dependent on domain-specific environments. Therefore, evaluating these complex processes requires customized, instance-specific rubrics capable of tracking whether the LLM’s intermediate reasoning aligns with unique environmental feedback and professional standards.

Recent benchmarks have increasingly adopted a strategy of decomposing complex execution processes into verifiable and fine-grained components. In the domain of information seeking, GISA [132] provides complete human search trajectories for every query, recording search queries issued, search engine results, and click-through behaviors with timestamps. While GISA does not use trajectories as formal evaluation rubrics, its analysis reveals that alignment with human search strategies positively correlates with task performance, suggesting that process-level trajectory data can serve as a meaningful signal for diagnosing agent weaknesses. In the context of autonomous tasks like research paper replication, PaperBench [95] decomposes the task pipeline into several modules, including code development and execution. The associated rubrics are designed to pinpoint exact locations of failure and provide partial credit even if the agent is not able to finish the final task. Beyond engineering tasks, MoReBench [83] focuses on moral reasoning tasks. Its core claim is that the logical completeness in the reasoning process is more important than the final moral decision, because moral dilemmas may not have objectively correct answers. Hence, it mainly focuses on auditing the internal thinking traces based on expert-written rubrics covering moral factor identification, process clarity, logical coherence, and outcome helpfulness.

For scenarios where intermediate trajectories cannot be directly obtained (e.g., black-box evaluated agents), several studies instruct the models to present intermediate reasoning steps within the final outputs. ProfBench [11] studies professional domains like PhD-level questions in physics and chemistry, as well as MBA-level issues. Its rubrics explicitly require the model to demonstrate intermediate calculation and reasoning, allowing the evaluation framework to further assess whether the information in the grounded documents is accurately extracted. Similarly, Pathak et al. [82] study code quality evaluation by following a point-wise, question-specific rubric approach to verify logical trajectories in each code implementation step. For tool-augmented agents, Scribe [93] standardizes intermediate evaluation by routing sub-goals to predefined skill prototypes. This constrained verification approach equips LLM judges with skill-conditioned rubrics to directly assess sub-goal completeness, execution flaws, logical gaps, and common traps. MedMT-Bench [102] uses instance-level rubrics to evaluate long multi-turn medical dialogues. It simulates the full process of medical consultation, diagnosis, and follow-up, and checks whether models can remember earlier information, avoid being misled by irrelevant context, follow medical safety rules, ask clarifying questions when user instructions are ambiguous, and respond to multiple user requests in one turn. These requirements are converted into atomic test points, so that LLM judges can evaluate complex medical dialogues through fine-grained yes-or-no checks.

5.2.2. Rubrics for Final Outputs

While trajectory analysis provides useful information about the execution process, final outputs remain the main results shown to users. Therefore, output-oriented rubrics are still important for evaluating what users actually receive. Instead of assessing correctness alone, these output-oriented rubrics evaluate output quality from several related dimensions. Existing practices can be summarized into four dimensions:

Content Factuality. Content factuality serves as a foundational dimension of final-output evaluation. It is crucial to evaluate whether the final outputs are not only correct and comprehensive but also strictly grounded in the provided or retrieved context.

For example, in Retrieval-Augmented Generation (RAG) systems, PencilsDown [88] explicitly evaluates this grounding through query-derived rubrics that measure information relevance and factual completeness. Contemporary benchmarks have shifted from holistic correctness to granular, context-aware verification. Scaling this principle to deep research agents, DeepResearchBench [17] introduces the FACT framework to evaluate deep research agents. It transforms holistic correctness into a verifiable metric by checking whether generated statements are accurately supported by their cited URLs, thereby quantifying citation trustworthiness and factual abundance in long-form research reports. However, since cited sources themselves may be inaccurate, DeepResearch Bench II [85] shifts focus toward expert-derived ground truth. It employs thousands of atomic, binary rubrics to rigorously verify specific factual and numerical claims, effectively rejecting seemingly correct hallucinations. This progression marks a shift from verifying citation consistency to enforcing granular, expert-validated accuracy. In high-stakes scenarios, HealthBench [16] utilizes physician-authored rubrics to specifically evaluate whether the model misses critical medical safety warnings. Expanding into complex professional domains, PRBench [103] leverages expert-curated criteria to evaluate not merely factual statement accuracy, but the precise application of facts to legal and financial rules. In context-aware and personalized environments, AstraBench [94] evaluates factuality through the lens of IR Recall and “No Hallucination” scores across complex and context-aware personal assistant tasks, such as cross-app information retrieval and temporal scheduling, ensuring that agentic responses align with the user’s evolving personal context. Furthermore, across domain-specific tasks like querying certain APIs (for financial data) and executing code interpreters, Scribe [93] incorporates targeted penalty rubrics to audit for errors like hallucinated outputs. This effectively transforms factual verification from a surface-level check into a diagnostic mechanism for identifying underlying reasoning failures.

Factuality has also been extended beyond text to multi-modal scenarios. For instance, TechImage-Bench [24] uses structural rubrics to check fine-grained entity presence, spatial relations, and semantic accuracy in technical scientific diagrams. Likewise, in long-video understanding, LongShotBench [104] uses graded rubrics to penalize temporal inconsistencies, entity mismatches, and multi-modal hallucinations, helping ensure that model outputs stay faithful to complex visual evidence.

Safety Auditing. Safety auditing serves as the critical counterpart to factuality. While factuality focuses on ensuring the presence of accurate and grounded information, safety auditing strictly enforces the absence of harmful, non-compliant, or high-risk content. In high-stakes environments, even a factually grounded output can lead to severe failures if it violates safety constraints or industry regulations.

Accordingly, recent benchmarks have reframed output evaluation into rigorous safety auditing by introducing severe penalty mechanisms. For instance, in the medical domain, HealthBench [16] and RubricRAG [52] use explicit negative rubrics to penalize unsafe behaviors, such as hallucinated clinical claims, dangerous triage advice, or failure to identify emergency “red flags”. MoReBench [83] further evaluates the harmlessness of final outcomes in pluralistic moral dilemmas. Beyond evaluating the direct outputs of models, comprehensive safety auditing also extends to the foundational artifacts of AI systems. For example, DataRubrics [128] applies automated quality rubrics to assess the accountability, ethical documentation, and transparency of datasets, mitigating systemic downstream risks by enforcing rigorous compliance standards at the source.

Professional Presentation and Structural Coherence. While both content factuality and safety auditing constrain the internal content, presentation and coherence determine how effectively that content is consumed and trusted by the user. In real-world deployments, outputs must satisfy domain-specific norms for organization, formatting, and tone. DEER [86] uses expert-developed rubrics to assess report-level coherence and readability. ProfBench [11] similarly grades conciseness, structure, and professional tone. $1M-Bench [12] includes verbalization quality as a core dimension for industry-style deliverables. Crucially, evaluating professional presentation also involves assessing the naturalness of these structures. For instance, LREAD [127] demonstrates that while LLMs excel at macro-structural coherence, they frequently produce overly regularized outputs characterized by structural rigidity, mechanical consistency, and subtle “translationese” that violate authentic stylistic norms. Beyond text, PresentBench [105] extends these ideas to multimodal artifacts through checklists for layout and design consistency. Similarly in the audio modality, recent work establishes rubric-guided frameworks for SpeechLLMs, extending the concept of presentation coherence to acoustic delivery [53]. Specifically, this maps structural coherence to fluency (evaluating the temporal smoothness and uninterrupted coherence of speech) and stylistic naturalness to prosody (capturing the expressiveness and natural rhythm of intonation).

Practical Utility and Actionability. Practical utility and actionability assess whether the generated artifact successfully resolves the user’s underlying problem under realistic constraints. Beyond the internal constraints of content factuality and safety auditing, utility- and actionability-oriented rubrics evaluate whether outputs are genuinely usable and whether they help complete the user’s intended goal.

For instance, in deep research tasks, DEER [86] includes “request fulfillment” criteria to verify that the final report comprehensively addresses every specific question raised by the user without evasion. Beyond mere coverage, utility demands actionable guidance. In morally complex settings, MoReBench [83] measures “outcome helpfulness,” rewarding agents that provide clear, actionable paths rather than purely abstract philosophical discussions. This demand for actionability is even more critical in specialized professional domains. PRBench [103] utilizes expert-curated rubrics to strictly evaluate the “practical utility” and “procedural correctness” of legal and financial advice, ensuring the generated deliverables are not merely theoretically sound, but practically viable and executable for real-world clients. Furthermore, in context-aware personal assistant scenarios, AstraBench [94] measures end-to-end “task completion” and “conversation effectiveness,” evaluating whether the agent can orchestrate state changes to achieve the user’s goal. Ultimately, the goal is to ensure the reliability of context-aware assistants in real-world use. To bridge the gap between predefined rubrics and diverse human expectations, LLM-RUBRIC [106] calibrates multidimensional evaluations using a personalized calibration network, modeling judge-specific subjective preferences to accurately predict the overall user satisfaction of the final artifact.

In summary, domain-specific task evaluation requires a shift toward customized, domain-aware rubrics. This process must systematically audit dynamic intermediate reasoning trajectories while comprehensively assessing final outputs across four core dimensions. Specifically, it must evaluate content factuality, conduct safety auditing to prevent harmful or non-compliant outputs, enforce professional presentation to maintain structural coherence, and verify practical utility to guarantee actionable real-world execution. Ultimately, this specialized paradigm ensures that LLMs generate artifacts that are not merely safe and theoretically sound, but fully aligned with rigorous professional standards and user-centric goals.

6. Open Questions and Discussion

Although rubric-based methods have rapidly emerged as a promising interface for specifying, evaluating, and optimizing LLM behavior, many foundational challenges remain unresolved. Unlike conventional evaluation protocols or scalar reward models, rubrics expose explicit criteria that can guide both judgment and training. This transparency improves interpretability, but it also introduces new challenges related to robustness, generalization, personalization, safety, and evaluation reliability. In this section, we will discuss these challenges in detail.

6.1. Robust Rubric Design against Reward Hacking

Reward hacking remains one of the most common failure modes in reinforcement learning, and rubric-based rewards do not escape this problem [120]. Compared with reference-based rewards or task-specific verifiable signals, rubrics expand the space of credit assignment by exposing multiple independently checkable criteria. This broader coverage is valuable for open-ended tasks, but it also enlarges the surface through which a policy can find shortcuts that satisfy rubric wording without genuinely improving response quality. Therefore, a central open question is how to design robust rubrics that provide rich supervision while reducing the risk of being exploited during optimization.

Granularity and Scope.

Existing rubric construction methods are still largely developed for general LLM generation settings, where rubrics mainly evaluate broad response-level qualities such as fluency, coherence, helpfulness, harmlessness, and overall reasonableness. Accordingly, many optimization and evaluation protocols are closer to general reward-model benchmarks, where the goal is to judge whether one response is preferable to another under relatively generic standards. However, as LLMs are increasingly used in more concrete application scenarios, especially agent-related tasks, the required rubric granularity becomes much more fine-grained. In these settings, evaluation is no longer limited to whether the final response looks reasonable, but also concerns whether the agent retrieves the right evidence, follows constraints, executes intermediate steps correctly, and integrates information into a task-completing output.

This shift raises new challenges for rubric design. For example, in search-agent evaluation, rubrics may need to assess the correctness and relevance of each retrieved evidence piece, whether necessary facts are covered, whether temporal or numerical constraints are satisfied, whether claims are properly supported by sources, and whether the final answer avoids unsupported synthesis. These criteria are more specific than general response-quality dimensions, and their design and weighting can directly affect agent training and final behavior. If rubrics overemphasize factual coverage, agents may retrieve and repeat many facts without producing a useful synthesis; if they overemphasize concise final answers, agents may ignore important evidence or constraints. However, existing rubric construction methods still lack systematic studies of such scenario-specific design choices. Future work should therefore investigate how rubric granularity and scope should vary across application settings, and how to balance general quality dimensions with evidence-level, constraint-level, and process-level criteria in agent evaluation and training.

Criterion Coupling and Aggregation.

Another important issue is the coupling among rubric criteria. Existing studies have moved beyond coarse-grained overall preference scores toward more fine-grained, multidimensional evaluation. For example, FLASK [96] decomposes overall evaluation into different alignment skills and emphasizes that different task instances may require different compositions of capabilities, while Prometheus [114] supports fine-grained evaluation of long-form outputs with customized score rubrics and further demonstrates the potential of rubric-guided evaluators as reward models. These studies suggest that rubric criteria should not be treated as isolated checklist items; rather, they jointly define the evaluative structure of model outputs. For instance, in complex planning tasks, criteria such as satisfying explicit constraints, maintaining cross-turn state consistency, producing executable answers, and avoiding unwarranted assumptions are often tightly interdependent. A failure in state tracking may directly lead to constraint violations, while an oversimplified response may simultaneously reduce completeness and executability.

This coupling is particularly important for reinforcement learning, where the reward signal is not merely a linear sum of individual rubric scores but a structured signal shaped by multiple interacting objectives. MORLAIF [133] decomposes complex preferences into multiple principle-specific preference models, such as toxicity, factuality, and sycophancy, and combines them through scalarization to form reinforcement learning signals. This suggests that rubric criteria may be synergistic or conflicting: some criteria jointly define necessary conditions for a high-quality response, whereas others introduce trade-offs. If such coupling is ignored, RL training may encourage the model to optimize locally easy-to-satisfy criteria while sacrificing global task success, leading to reward hacking. Future work should therefore investigate how to learn dependency graphs among criteria, design nonlinear or hierarchical reward aggregation functions, distinguish hard constraints from soft preferences, and analyze how different rubric compositions affect generalization, training stability, and interpretability.

Training-Adaptive Rubric Evolution.

Static rubrics, no matter how carefully engineered offline, are especially vulnerable during RL training. A rubric that is effective for judging the initial policy may become insufficient once the policy has learned to optimize against it. This creates a dynamic form of reward hacking: the model does not merely exploit a fixed reward function, but gradually discovers weaknesses in the rubric as training proceeds.

A natural mitigation, exemplified by works such as DR-Tulu [29], OnlineRubrics [48], SibylSense [49], and RLAC [50], is to let rubrics co-evolve with the policy. Under this perspective, rubrics are not static evaluation artifacts, but adaptive supervision mechanisms that can incorporate newly discovered failure modes into the criterion set. Such self-evolving rubrics may make the evaluation contract progressively stricter as the model improves, thereby reducing the chance that the policy repeatedly exploits outdated criteria. However, this direction also raises unresolved algorithmic questions: how often should rubrics be updated, how can update frequency be balanced against optimization stability, how can the rubric set avoid overfitting to the current policy distribution, and how to characterize the convergence of a joint policy–rubric optimization process. Designing principled training-adaptive rubric evolution mechanisms, rather than relying on ad-hoc heuristic updates, remains a critical open problem.

6.2. Generalization of Rubric-Based Reward Models

A complementary challenge concerns the generalization of rubric-based reward models. Rubrics have increasingly been embedded into reward modeling pipelines, either as explicit criteria used by judge models or as structured supervision for training dedicated reward models. Recent attempts—ranging from rubric-grounded reasoning models such as R3 [70] and CDPRM [36] to dimension-decomposed scorers such as ArmoRM [75] and MR-RML [77]—show that rubrics can improve the controllability and interpretability of reward modeling. Instead of producing only an opaque scalar score, such models can decompose evaluation into dimensions, expose intermediate judgments, or aggregate rubric-level scores into a final reward. This makes them attractive as reusable reward sources for policy optimization.

However, existing rubric-based reward models are still far from generally reliable. Fine-grained rubrics are often constructed for specific domains, tasks, or benchmarks, which limits their transferability to settings with different objectives, output formats, or evaluation standards. A reward model trained on rubric supervision from one domain may overfit to the surface form of those rubrics, the distribution of training responses, or the scoring habits of the judge used to generate supervision. When applied to new tasks, such a model may fail to adapt to new rubrics, misinterpret domain-specific requirements, or assign high scores to responses that satisfy familiar criteria while missing task-critical dimensions. This problem is especially severe in specialized domains such as medicine, law, finance, and scientific reasoning, where evaluation standards depend on expert knowledge and cannot be fully captured by generic helpfulness or correctness criteria.

The key open challenge is therefore to train rubric-based reward models that can generalize across tasks and domains while remaining faithful to the intended evaluation criteria. This involves several intertwined questions. First, what kind of rubric supervision is needed for cross-domain transfer: broad generic criteria, domain-specialized criteria, or compositional mixtures of both? Second, how should rubric information be represented inside the reward model: as natural-language criteria, dimension-level vectors, structured graphs, or latent preference factors? Third, how can we evaluate the reliability of a rubric-based reward model when no gold rubric exists for a new task? Recent meta-evaluation efforts such as RubricEval [125], RubricBench [126], and JudgeBench [100] mark important first steps, but a principled understanding of rubric reward model generalization is still missing.

One promising direction is to enable efficient adaptation with limited new supervision. Instead of requiring large-scale rubric annotations for every new domain, a rubric-based reward model could be recalibrated using a small number of task-specific rubric annotations or expert-written criteria. Such adaptation may help the model adjust to domain-specific standards, output formats, and failure modes while retaining general evaluation capabilities learned from broad rubric supervision. This is particularly important for specialized domains such as medicine, law, finance, and scientific reasoning, where fully annotating fine-grained rubrics is expensive but small amounts of expert feedback may be available.

Another important direction is to design modular or compositional rubric representations. Many evaluation dimensions, such as factuality, completeness, safety, evidence grounding, instruction following, and practical utility, recur across tasks but appear in different combinations and with different weights. If reward models can learn reusable rubric modules, then new task-specific rubrics may be constructed by selecting, recombining, and recalibrating these shared dimensions rather than retraining the model from scratch. This would allow reward models to transfer partially across tasks: general dimensions can be reused, while domain-specific dimensions can be added or adjusted with limited supervision. Such compositionality may also improve interpretability, because the reward model’s behavior can be analyzed in terms of which rubric modules are activated and how they are weighted in a given task.

6.3. Bias in Rubric-Based Evaluation

Although rubric-based evaluation improves the interpretability of LLM evaluation, it does not guarantee unbiased judgments. Bias may be introduced by the rubric content, by the judge models that apply the rubrics, and by the human experts who define or validate the criteria. Understanding these sources of bias is important for making rubric-based evaluation more reliable and comparable across tasks, models, and evaluators.

Bias from Rubric Design.

Bias in rubric-based evaluation can arise from how rubrics are written or presented to the judge model. On the one hand, LLM judges can be sensitive to the phrasing, ordering, and formatting of rubric criteria. Even when different expressions look equivalent to human evaluators, they may lead LLM judges to generate different judgments [110]. On the other hand, rubric-based scoring may also suffer from position bias when multiple score options are presented [134]. For example, presenting the score levels as “Poor–Fair–Good–Excellent” may produce a different score distribution from presenting them in the reverse order, even though the available options are semantically equivalent. Therefore, rubric design should not rely only on intuitive writing, but should involve systematic prompt design and empirical validation.

Bias from Judge-Model.

Bias can also come from the judge model that applies the rubrics. First, different LLM judges may assign different scores to the same response under the same rubric, due to differences in model priors, training data, and understanding of rubrics. This makes rubric-based evaluation dependent not only on the rubric itself, but also on the selected judge model. Second, LLM judges may exhibit self-preference bias [135], where they tend to give higher scores to outputs generated by themselves or models from the same family. This bias can persist even under seemingly objective rubrics and may yield unreliable model comparisons. Third, LLM judges may differ from human evaluators in how they weight rubric dimensions. Even when humans and LLMs use the same rubrics, they may focus on different aspects of response quality [54]. For example, in code evaluation, LLM judges may emphasize functional correctness or detailed explanations, while human developers may care more about context-aware solutions, maintainability, or whether the code naturally fits into the existing workflow. These observations suggest that, beyond improving rubric design, developing more reliable evaluation protocols and reducing bias across different LLM judges are important directions for future work.

Human-Expert Disagreement and Pluralistic Bias.

A third source of bias comes from the human side of rubric construction and validation. Many rubric-based evaluations implicitly assume the existence of a single “golden standard” or unified ground truth. However, in highly specialized downstream domains, such as medicine, law, education, moral reasoning, and policy analysis, experts may reasonably disagree about what constitutes a high-quality answer. Different clinical schools may prioritize different diagnostic considerations; different legal frameworks may lead to different interpretations; and different educational philosophies may value different forms of explanation or scaffolding. In such cases, rubrics written only by one human expert may encode a legitimate but partial perspective, while appearing to be an objective standard.

This suggests that future rubric-based evaluation should move beyond the single-standard assumption and consider multi-perspective or pluralistic rubrics. Drawing inspiration from benchmarks such as MoReBench [83], which emphasizes logical completeness and process coherence in pluralistic moral dilemmas, future rubrics may evaluate whether an output is internally coherent within a given theoretical or professional framework, rather than forcing all responses into a single preference ordering. More broadly, rubric-based evaluation may need mechanisms for representing expert disagreement, reporting score uncertainty, and distinguishing factual errors from perspective-dependent judgments. Developing such pluralistic rubric architectures is essential for robust domain-specific evaluation, especially in settings where professional diversity and normative disagreement are unavoidable.

6.4. Personalized Rubrics

Most existing rubric-based methods treat rubrics as general evaluation criteria shared across users. However, in many open-ended tasks, especially creative writing, recommendation, education, and assistant-style interaction, response quality is highly user-dependent. Different users may prefer different tones, levels of detail, writing styles, reasoning patterns, interaction goals, and even different trade-offs between accuracy, creativity, conciseness, and emotional resonance. Therefore, an important open question is how to construct personalized rubrics that explicitly capture individual preferences while remaining interpretable, reliable, and safe.

PREFINE [136] provides a useful example in personalized story generation. Instead of relying on direct user feedback or parameter updates, it constructs a pseudo-user agent from user history and generates user-specific rubrics to guide critique and refinement. This demonstrates that rubrics can serve not only as general evaluation standards, but also as an explicit interface for representing personalized preferences. In such a framework, user-specific rubrics can make personalization more transparent: rather than simply producing a personalized output, the system can expose what aspects of the user’s preference it is optimizing for, such as preferred narrative style, emotional tone, character development, or thematic focus.

Despite this promise, personalized rubrics remain difficult to build and evaluate. User preferences are often implicit, sparse, noisy, and context-dependent, making it unclear how much user history is needed to infer faithful rubrics. Moreover, preferences may change over time or vary across tasks, so a static user profile may quickly become outdated. Another risk is superficial personalization: a system may overfit to easily observable signals such as wording, genre, or length, while missing deeper preferences such as factual rigor, emotional nuance, pedagogical value, or practical usefulness. This is particularly problematic when personalized rubrics are used not only for evaluation but also for training, because models may learn to imitate shallow preference markers rather than genuinely adapt to user intent.

A further challenge lies in the tension between personalization and general quality or safety standards. Optimizing a user-specific rubric should not lead the model to reinforce biased, unsafe, manipulative, or low-quality preferences. Future work therefore needs mechanisms to validate whether personalized rubrics truly reflect user preferences, remain stable across contexts, and are compatible with broader alignment constraints. Promising directions include separating user-specific preferences from universal safety requirements, auditing personalized rubrics for harmful or biased criteria, modeling preference uncertainty, and allowing users to inspect or revise the rubrics that represent them.

6.5. Safety of Rubrics

As rubrics become widely used in LLM evaluation and training, their safety also becomes an important open question. Existing work often treats rubrics as transparent and trustworthy evaluation instructions, but recent evidence suggests that rubrics themselves can become an attack surface. Ding et al. [137] identify Rubric-Induced Preference Drift (RIPD), where seemingly natural and benchmark-compliant rubric edits can systematically shift the preferences of LLM judges on target domains. This is especially concerning because such edits may not significantly reduce aggregate benchmark performance, yet they can still change the judge’s preference direction and affect downstream alignment results.

This problem suggests that rubric safety should be studied beyond traditional evaluator robustness. A malicious or poorly designed rubric may subtly change criterion weights, introduce biased wording, emphasize some dimensions while downplaying others, or reshape the judge’s decision boundary while still appearing reasonable to human inspection. For example, a rubric edit may preserve the same high-level topic but make the judge more tolerant of unsupported claims, more favorable toward a particular style, or less sensitive to safety-relevant omissions. Because natural-language rubrics are flexible and often manually refined, such preference shifts can be difficult to detect through standard benchmark validation alone.

The risk becomes more serious when rubric-based judgments are used as preference labels or reward signals. In this case, rubric-induced bias may propagate into policy model training and produce persistent behavioral drift. A judge that is biased by a rubric may generate biased preference data; a policy optimized on that data may then learn the shifted preference; and future evaluation may fail to detect the drift if it relies on similarly vulnerable rubric-based judges. Therefore, rubric safety should be understood as a pipeline-level problem rather than merely a prompt-level problem.

Future work needs mechanisms for detecting unsafe rubric edits, auditing preference shifts across domains, and validating rubric invariance under paraphrasing or refinement. One promising direction is to evaluate rubrics not only by aggregate benchmark scores but also by their directional effects on targeted preference axes. Another direction is to develop adversarial tests that perturb rubric wording, ordering, and emphasis to measure whether judge behavior remains stable. Ensuring the safety of rubrics is essential for making rubric-based evaluation and training reliable in high-stakes alignment pipelines.

7. Conclusions

As LLMs continue to move toward open-ended, high-stakes, and agentic applications, rubrics offer a practical way to express complex quality standards as explicit, interpretable, and operational criteria. This paper systematically reviewed rubric-based research from three perspectives: how rubrics are constructed, how they support the training of policy models and reward models, and how they are used for task evaluation across general and domain-specific tasks. Although recent studies have shown the value of rubrics in improving model training and evaluation, many challenges remain, such as reward hacking in rubric-based RL, generalization of rubric-based reward models, and bias in rubric-based evaluation. Future work should therefore develop more reliable, adaptive, and trustworthy rubric systems that can better connect human expectations, task requirements, and model behavior.