Towards Pluralistic Alignment of LLMs: A Comprehensive Survey

1. Introduction

As large language models (LLMs) exert growing influence, ensuring their beneficial and safe use has become increasingly important (Askell et al. 2021; Ji et al. 2023a; Minaee et al. 2024; Shen et al. 2023; Wang et al. 2024c). Accordingly, alignment has become central to responsible AI (Gabriel 2020; Ji et al. 2023a; Ouyang et al. 2022; Shen et al. 2023), which is the process of ensuring that AI model exhibits behaviour consistent with human values and intentions (Ji et al. 2023a; Shen et al. 2023). This has driven widespread adoption of aligned LLMs, mitigating issues such as privacy violations, toxicity, and misinformation observed in earlier systems (Bubeck et al. 2023; Carlini et al. 2023; Ji et al. 2023b; Weidinger et al. 2021). Despite these advances, alignment methods still face fundamental limitations. In particular, they tend to capture monolithic (or average) human values (Feng et al. 2024a; Shetty et al. 2025; Sorensen et al. 2024b), oversimplifying the alignment problem by failing to represent pluralistic human values (Conitzer et al. 2024; Sorensen et al. 2024b; Stanczak et al. 2025). Human values and requirements vary widely, not only across individuals, but also for the same individuals across different contexts (Bilsky et al. 2011).

To address this limitation, emerging research in pluralistic alignment (Conitzer et al. 2024; Sorensen et al. 2024b; Stanczak et al. 2025) examined how systems can be designed and deployed to reflect diverse human needs and values. Sorensen et al. (2024b) proposed a theoretical framework defining three modes how models could be pluralistic: Overton, Steerable, and Distributional. As illustrated in fig:plural-align-overview, consider the query: “Should the country drastically reduce immigration levels?” Under the Overton mode, the model presents all reasonable answers, covering a range of perspectives based on political views, moral frameworks, In Steerable mode, the model generates a response conditioned on a specified attribute, for instance, opposing the proposal from a far-left perspective or supporting restrictive immigration policies from a far-right perspective. Finally, in the Distributional mode, responses reflect population-level preferences. If 80% of the population opposes the proposal and 20% supports it, the model should respond “No” 80% of the time and “Yes” 20% of the time.

Figure 1. An overview of three pluralistic alignment modes (Overton, Steerable, and Distributional) for an example query: “Should government have strict policies to reduce immigrant levels?” Overton covers different perspectives, whereas Steerable responds as per attribute conditioned, and Distributional output match reflecting population preference distribution.

Figure 1. An overview of three pluralistic alignment modes (Overton, Steerable, and Distributional) for an example query: “Should government have strict policies to reduce immigrant levels?” Overton covers different perspectives, whereas Steerable responds as per attribute conditioned, and Distributional output match reflecting population preference distribution.

Over the past few years, pluralistic alignment has attracted significant attention, including numerous papers (Castricato et al. 2025; Chen et al. 2024a; Feng et al. 2024a; Shetty et al. 2025; Sorensen et al. 2024b, inter alia) and dedicated workshops at major AI and ML conferences (Shen et al. 2025; Terekhov et al. 2024). However, rapid progress has led to limited clarity about the field’s current state, open gaps, and future directions. Existing work has demonstrated the potential for achieving pluralism in constrained settings and modes. Our goal is to provide a systematic and comprehensive guide that helps researchers identify appropriate tools and techniques and contribute to this rapidly evolving area. We are also the first to examine pluralism beyond LLMs. These definitions are generalisable to broader AI systems, and we include other modalities such as images, audio, and agents.

To this end, our survey presents an overview of the current landscape of pluralistic alignment. For each alignment mode (defined in Section 3), we provide a comprehensive overview of existing methods (Section 4) and evaluation approaches (Section 5), together with current shortcomings and directions for future research (Section 6). We aim to provide a clear entry point for researchers and facilitate further work on pluralistic LLM systems. We believe this survey is timely, offering a bird’s-eye view of the literature while supporting continued progress in pluralistic alignment.1

The key contributions of this survey are as follows:

• To the best of our knowledge, this is the first survey dedicated exclusively to pluralistic alignment, providing a bird’s-eye view of three modes—Overton, Steerable, and Distributional—which are often studied in isolation. We consolidate existing research and identify pluralistic methodologies, providing a practical roadmap for researchers.
• We provide a detailed review of pluralistic alignment techniques, together with relevant datasets and evaluation metrics, helping researchers make informed methodological choices. For each category, we synthesise the literature and highlight open challenges and key takeaways.
• We examine understudied aspects of pluralistic alignment and its connections to adjacent fields, outlining promising directions for future work, including multi-turn interactions, agentic AI, temporal and geo-alignment, mechanistic interpretability, and safety risks. This survey aims to advance the development of generalist models that reflect pluralism and enable safe and responsible deployment.

2. Related Surveys and Key Differences

As AI systems are deployed to broader populations, prevailing preference-based alignment methods (Ouyang et al. 2022; Rafailov et al. 2023) that model average opinions risk treating value diversity as noise (Aroyo et al. 2023; Siththaranjan et al. 2024), contributing to bias (Benkler et al. 2023), fairness concerns (Gallegos et al. 2024), and under-representation (Ovalle et al. 2025). Equally concerning is that models may be aligned to reflect the biased preferences of their creators (Bender et al. 2021; Gabriel 2020; Weidinger et al. 2021, ), thereby supporting particular agendas rather than supporting the full spectrum of human values. Pluralistic alignment instead seeks to model this diversity directly. Despite its growing relevance, no prior survey provides a comprehensive overview of pluralistic alignment. Existing reviews focus on general alignment (Cao et al. 2024; Ji et al. 2023a; Shen et al. 2023), multimodal alignment (Shu et al. 2025; Yu et al. 2025), reward design (Ji et al. 2025), cultural alignment (Pawar et al. 2025a), or personalisation (Xie et al. 2025b; Zhang et al. 2024). We address this gap by presenting a systematic overview of alignment through the lens of supporting pluralism: consolidating current methods, clarifying distinctions, and outlining open challenges (shown in blue boxes at the end of each subsection), also connecting pluralistic alignment to broader LLM research.

3. Pluralistic Alignment

A modern democratic society is characterised not simply by a pluralism of comprehensive religious, philosophical, and moral doctrines but by a pluralism of incompatible yet reasonable comprehensive doctrines.

—Rawls (1971)

We define a pluralistic model as one that serves different users and adapts to their varied queries across various contexts. It is paramount to have a single foundation model that is pluralistic (Bai et al. 2022b; Gordon et al. 2022; Sorensen et al. 2024b). We study pluralistic behaviour in LLMs using a framework from Sorensen et al. (2024b), which we describe next, along with motivation. Then, we formally define each part of the framework in Section 3.2.

3.1. Sorensen’s Framework

Figure 1 illustrates all three modes for an example query in a pluralistic AI system. The motivation behind these modes are as follows:

• Overton alignment expects the model’s output to cover all feasible responses. Usually, there are many reasonable answers to a question (Min et al. 2020; Scherrer et al. 2023), and outputting a single reasonable answer promotes bias and limits representation. An Overton-pluralistic model instead covers all reasonable answers. The most relevant application of Overton pluralism is advice giving (Jakesch et al. 2023; Krügel et al. 2023), where responses should cover all perspectives and opinions.
• Steerable alignment allows the model to successfully orient itself to a particular attribute (political view, religion, culture, among others). Customisation is key in steerable alignment where users often want to personalise models towards characteristic properties and perspectives (Chen et al. 2024c; Li et al. 2023; Ma et al. 2024).
• Distributional alignment steers the model’s response to match the underlying population distribution. Distributional alignment applies in scenarios where the model must reflect the views of a population. Example applications include social simulations (Park et al. 2022; Törnberg et al. 2023), autonomous vehicles (Cui et al. 2024), and related tasks.

3.2. Problem Formulation

To formalise the definition of pluralistic alignment, throughout we assume an AI system $ℳ$, that receives a query x and generates a response y.

• Overton.

We define the set of reasonable answers $ℛ$ for a given query x as those responses that reflect an interpretation or solution that the majority of the population would agree upon. However, additional filters (such as safety, fairness, etc.) are applicable. We denote the set of all such reasonable answers by the Overton window $W\left(x\right)$ (OED 2026):

$$W\left(x\right)=\{y\in \mathcal{Y}\mid (x,y)\in \mathcal{R}\}.$$

A response set $\left\{y\right\}$ is Overton-pluralistic for a given query x if it consists of all reasonable answers (i.e., $W\left(x\right)$). Similarly, a model $ℳ$ is Overton-pluralistic if for a given input x,

$$\mathcal{M}\left(x\right)=W\left(x\right).$$

• Steerable.

This is the case where the model must faithfully align (or steer) a response y to a given query x towards a specified attribute or perspective. Steering attributes 𝒜 denote the set of such attributes, including values, perspectives, populations, frameworks, and others.

We say that a response $y_{x,a}$ reflects an attribute $a\in$ 𝒜 if the response y to query x is consistent with a. Accordingly, a model $ℳ$ is steerable-pluralistic with respect to attributes 𝒜 if, for every input $x:$

$$\forall a\in \mathcal{A},\mathcal{M}(x,a)=y_{x,a}.$$

• Distributional.

Here, we define a reference population $𝒢$ as the set of people that a distributionally-pluralistic model aims to represent. A distributionally aligned model $ℳ$ then responds y to a given query x according to the reference population distribution. In other words, $ℳ$ is well calibrated over reasonable answers y with respect to the reference population.

$$P_{\mathcal{M}}(y\mid x)=P_{\mathcal{g}}(y\mid x)$$

4. Pluralistic Alignment Methodologies

Given Sorensen’s framework, we categorise methodologies introduced in prior work and provide avenues for future research. Figure 2 organises existing literature as per pluralistic alignment modes supported.

4.1. Overton

• Training-Free.

ReConcile (Chen et al. 2024d) is one of the initial works supporting Overton mode. It is an inference-time-based multi-agent method involving collaborative discussion (multiple rounds) among LLMs, leveraging diversity across different models to synthesise a response with a higher Overton window coverage. PICACO (Jiang et al. 2025a) is the most recent in-context alignment (ICA) method for Overton mode (or value coverage) working solely on inference time without any training. At its core, it is a prompting-based technique trying to elicit diverse responses embedded in the LLM (Cheng et al. 2024; Ganguli et al. 2023). However, unlike prior ICA approaches, it does not need multiple LLMs as in OPRO (Yang et al. 2024a) for iterative prompt optimisation, sophisticated time-consuming human-crafted meta-instructions as in URIAL (Lin et al. 2024), or the strong LLM extension of URIAL in URIAL+Sum (Lake et al. 2025). PICACO maximises the total correlation between target values—comprising Helpful and Harmless (Bai et al. 2022a; Weidinger et al. 2021) and the Schwartz Value Theory (Schwartz 2012)—and generated responses using a feedback loop to construct effective prompts. This also combats the instruction bottleneck, LLMs struggle to comprehend multiple values in a single prompt (Chen et al. 2024e; Jiang et al. 2024). Including the well-known prompting issues (Chakraborty et al. 2024; Sahoo et al. 2024; Zhao et al. 2024), this work only evaluates a limited set of values and uses a potentially biased LLM scoring for value coverage.

• Others.

Feng et al. (2025) propose a policy-based work primarily focused on Overton mode. It is grounded in a real-world 15-week observational study of LLM policymaking workshops in an industrial AI lab. They advocate that LLM policy prototyping could be an interesting avenue for pluralistic alignment because policy prototyping encourages discussion with contrasting preferences and opinions. Similarly, Collective Constitutional AI (CCAI) (Huang et al. 2024) proposes a multi-stage process that sources diverse rules from the public and incorporates them in LLM development. This builds on the foundations of Constitutional AI and can be seen as an extension of reinforcement learning from human feedback (RLHF) (Ouyang et al. 2022), promoting that the responsibility of LLM should not rest only with developers.

4.2. Steerable

• Prompting-Based.

LLMs exhibit a degree of steerable pluralism via prompting (Sorensen et al. 2024b). Initial works achieved steerability by simply setting different contexts in the system roles of LLMs in a zero-shot manner. One simple approach is “role-playing”, where LLMs simulate a person from a particular culture, i.e., CultureDom (Wang et al. 2024d), Culture-Gen (Li et al. 2024d), and HofstedesCAT (Masoud et al. 2025), or demographics, i.e., NormAd (Rao et al. 2025) and SythPersona (Kwok et al. 2024). Similarly, methods that include user history in prompts lead to better steerability, e.g., Persona (Castricato et al. 2025).

The relevance of ICA methods for Overton (and in general) alignment (Section 4.1) has also been studied for steerability. One of the first works showed that a simple few-shot personality prompting (${\mathrm{P}}^2$) (Jiang et al. 2023) using Big Five traits was effective in inducing personality. Adams et al. (2025) improved ICA for the steerable mode. It proposed ICL-SPM, a few-shot comparative regression-based approach. It uses in-context prompting for improved steerable accuracy, with chain-of-thought reasoning for explainability, and LLM-as-Judge for reducing bias in candidate selection. Overall, as in ICA methods, no fine-tuning or model merging is needed. Expectedly, ICL-SPM is shown to improve on Prompt-SPM (Hu et al. 2024) (a zero-shot prompt-based alignment method) for MIC (Ziems et al. 2022) and HelpSteer2 (Wang et al. 2024e) benchmarks. Spica (Chen et al. 2025) is another in-context alignment technique for the steerable mode. It accounts for group-level differences during example retrieval for in-context prompting, rather than only similarity. They show improved steerability on ValuePrism (Sorensen et al. 2024a) and PRISM (Kirk et al. 2024a) datasets. However, the study is limited to only four demographic groups and lacks strong steerable baselines.

• Training-Based.

The culture and personality sub-dimension of pluralistic alignment (AlKhamissi et al. 2024) has been of interest in the community, especially their adaptability (or steerability). There have been several works (Bose et al. 2025; Li et al. 2024a; Li et al. 2024b; Shi et al. 2024; Xu et al. 2025; Zhu et al. 2025, inter alia) collecting various cultural or personality data (with the hope of more representation) and then fine-tuning LLMs to achieve better steerability. Taking a different route, Guo et al. (2025) proposes COUPLE, an inference-time causal-based method, improving on previous causal-based work ValueCausalGraph (Kang et al. 2025). The paper argues that existing solutions consider all values to be independent and/or equally weighted, failing to capture their causal interdependencies between them. COUPLE builds a causal graph between values and model response using an LLM, then applies counterfactual reasoning to generate outputs aligned with a specific given value. They demonstrate better performance compared to existing prompting and tuning methods, but it is limited to the culture sub-dimension of pluralism and need a strong reasoning model.

• Reinforcement Learning Based.

In need of controllable LLMs, previous work has explored fine-tuning for multi-objective using scalar feedback for user-steerable LLMs as an alternative to RLHF or DPO (single-objective). SteerLM (Dong et al. 2023) was one such prominent supervised fine-tuning method. This method conditions its generation to conform to a set of attributes. The reward model in RLHF is replaced with an attribute prediction model, which predicts the human preference for different attributes. This work was further improved and extended using better multi-attribute datasets, HelpSteer (Wang et al. 2024f) and HelpSteer2 (Wang et al. 2024e). SteerLM has the same limitation of reward mispecification as in RLHF due to the use of reward models. This was rectified in several follow-up works (Guo et al. 2024; Manyika 2024; Wang et al. 2024a; Wang et al. 2024b, inter alia). Similarly, there have been several position works (Harland et al. 2024; Vamplew et al. 2024; Xiong and Singh 2025) sketching out the connection between multi-objective reinforcement learning and steerable pluralistic alignment.

4.3. Distributional

Santurkar et al. (2023) were among the first to study distributional alignment, contributing OpinionQA, an instrumental survey-based benchmark for evaluating pluralistic alignment. Expectedly, they established notable misalignment between the views reflected in LLMs and those of US demographic groups. Meister et al. (2025) empirically study the distributional mode on GlobalOpinionQA, OpinionQA, and their new dataset, NYTBookOpinions, that expands measurements beyond political and cultural values. Results reveal systematic weaknesses in LLMs as demographic simulacra, i.e, describing human distributions better than sampling them. This work also coined the term: “distributional alignment”, laying out the evaluation protocols and datasets.

• Prompting-Based.

Xie et al. (2025a) designs a prompt mixture modelling solution (PromptMixture) for distributional alignment, focusing on social simulation (or distributional mode). It comprises a pool of prompts, each representing a subpopulation, and then learns an optimal mixture for these prompts (Park et al. 2022). Finally, prompts are sampled according to the weights to match the underlying human distribution, simulating the actual population. The method offers improvements over existing similar social simulation techniques PersonaHub (Xu et al. 2025) and OASIS (Yang et al. 2024b). PersonaHub sets a persona as a system prompt sampled from a pool of personas, while OASIS uses demographic-inspired personas from the Big Five dataset. Likewise, Kambhatla et al. (2025) studies distributional calibration of LLMs for different social science surveys. Simple supervised calibration of probabilities generated by LLMs improve distributional alignment. The LLMs tested for different variations of socio-demographic prompts (e.g., SimLLMCultureDist (Cao et al. 2025), SelfAlign (Choenni and Shutova 2024)), showing effective calibration. A core issue is that heavy reliance on demographics and heuristics for persona sampling could lead to oversimplification and under-representation (Beck et al. 2024; Cheng et al. 2023a; Cheng et al. 2023b).

• Training-Based.

PAL (Chen et al. 2024a) is a training-time framework for pluralistic alignment, capturing and preserving heterogeneity, addressing the core limitation of RLHF (single, universal preference). It introduces the lens of the ideal point model and metric learning, using a ground-up mixture modelling approach. It inherently supports distributional and steerable modes in this order. Likewise, MallowsPO (Chen et al. 2024b) leverages dispersion-based modelling, and DPRM (Li et al. 2024c) uses multi-attribute annotations along with optimal-transport-based loss to better align with diverse human preferences. MixDPO (Imai et al. 2026) is a generalisation of DPO, a prominent RLHF technique that optimises models by directly comparing preferred and less-preferred outputs in preference data. In DPO and previous approaches, all human preferences have uniform strength, leading to average (or monolithic) human preference learning. In contrast, MixDPO has variable preference strength (based on individual and context), which is more suitable for pluralistic alignment. They demonstrate that MixDPO leads to improvement compared to DPO. Although it lacks evaluation of the modes of pluralistic alignment specifically, it is most applicable to the distributional mode because it captures heterogeneous preference strength. Halpern et al. (2025) proposes an ensemble of rewards heuristic, FSAM, instead of a single reward model to better represent pluralism. It treats annotator disagreement as soft labels instead of noise. They learn this diverse human preference distribution via pairwise calibration, where the proportion of rewards models preferring one preference captures the fraction of annotators (or population) with that preference. The work demonstrated its effectiveness both theoretically and empirically. However, all these works need further empirical investigation on standard distributional benchmarks.

4.4. All Modes

• Training-Based.

ModPlural (Feng et al. 2024b) was one of the foundational works tackling all three modes of pluralistic alignment. It is a model-collaboration technique where the main LLM works in collaboration with a set of specialised smaller LLMs to achieve pluralism. They show for different types, sizes, and architectures of main LLMs empirically better compared to standard baselines: off-the-shelf model, pluralistic prompting, and mixture of experts (pool of specialised LLMs). As per the mode of alignment, collaboration is conducted. For Overton, the main LLM acts as a summariser for specialised LLM responses and compiles a final representative (higher Overton window coverage) answer. In Steerable, one relevant response from the pool is chosen, and the main LLM is steered to output conditioned on this chosen response. Finally, for distributional, the process is similar, where the main LLM generates conditioned on each pool of responses. The point to note is that instead of focusing solely on text, it aggregates multiple answer probability distributions resembling the underlying population preference distribution. In the paper, they evaluate two sets of specialised LLMs, perspectives-informed across left/centre/right for social media texts and community-based covering different continents. However, the specialised LLMs are basically fine-tuned 7B LLMs on custom datasets. The main selling point of ModPlural was due to the decoding-time collaboration; a new specialised LLM could be consulted or patched as required. Still, the requirement of special datasets and computationally heavy fine-tuning bottlenecks remains a challenge. There have been subsequent works trying to replace the pool of specialised LLMs with training-free alternatives, we discuss cover next.

• Training-Free.

Follow-up work, Ethos (Zhong et al. 2025), replaced this fine-tuned pool of LLMs with a pool of personas appropriate for the query. The core tenet is to elicit different answers from the same secondary LLM. Though scalable and training-free, the known issues of prompting (Chakraborty et al. 2024; Sahoo et al. 2024; Zhao et al. 2024) and its brittleness still persist. A recent work, VISPA (Zheng et al. 2026), improves on Ethos and ModPlural with state-of-the-art performance on Vital (Shetty et al. 2025) and general pluralistic alignment benchmarks (from Feng et al. (2024a). VISPA is inspired by the success of activation steering for capturing different concepts and values, and better-controlled generation compared to prompting. The method is not heavily impacted by the choice of different activation steering instantiations. Therefore, with further improved methods in activation steering, it is a potential scalable and interpretable pathway to pluralistic alignment. This work also takes the first step for combating representation and bias issues by incorporating a comprehensive set of values considered for internal steering, including non-WEIRD (Western, Educated, Industrialised, Rich, and Democratic).

5. Evaluation of Pluralistic Alignment

5.1. Datasets

In this section, we cover datasets in the pluralistic alignment literature. We also go beyond and cover relevant potential datasets from related literature. We predominantly cover benchmark datasets across different dimensions, which might be of interest to readers, like modes, languages supported, modalities, turn settings covered, and others. A detailed table Appennndix A1 covers all datasets and metadata, providing a bird’s-eye view. To be consistent with previous discussion on methodologies, we cover benchmarks as per alignment modes.

• Overton.

An Overton benchmark should comprise a scenario/situation and corresponding applicable and reasonable answers as defined in Section 3.2. ValuePrism (Sorensen et al. 2024a) was the first Overton-specific dataset that consists of a scenario and different values applicable to it. These different values applicable to a scenario represent the Overton window. It is a large-scale dataset formed from a human online platform, Delphi (Jiang et al. 2025b), with diligent dataset preparation. However, it only encompasses value pluralism, and this is not conclusive of all aspects of pluralism.

• Steerable.

A steerable benchmark consists of situations that contain a question and a list of responses, where each response is labelled with a set of attributes (this can be values, perspectives, or properties of interest). And, we expect the model to steer towards a specific response for a given attribute and question. Therefore, surveys (Durmus et al. 2024; Inglehart et al. 2000; Santurkar et al. 2023) are relevant because they document different preferred answers (as per demographics, country, culture, and other attributes) for the same question. Similarly, the literature has numerous datasets on moral dilemmas (Chiu et al. 2025; Ren et al. 2024; Scherrer et al. 2023; Sorensen et al. 2024a), conditioning different and often conflicting values, making them suitable as steerable datasets. Recently, chatbot preferred response datasets (Kirk et al. 2024a; Zhang et al. 2025a) with rich user information have been contributed. As in other fields of current research, we have several synthetic datasets contributed. Castricato et al. (2025) introduced a large-scale synthetic dataset Persona to evaluate steerability across diverse user preferences using LLM-as-Judge. Recently, in Jiang et al. (2025c), they propose the IndieValueCatalog dataset, a transformed version of World Values Survey (WVS) to study individual steerability.

• Distributional.

For distributional-pluralistic system evaluation, we need a dataset with a query and a suite of answers along with a target distribution over these answers (usually collected by many annotators). Again, existing survey data is ideal for such an evaluation (Durmus et al. 2024; Jiang et al. 2023; Santurkar et al. 2023). The works here leverage a multiple-choice dataset and try to capture whether the model distribution matches the underlying golden distributions on limited buckets of multiple-choices. GlobalOpinionQA is a prominent cross-national survey capturing opinions on global matters. Whereas the Machine Personality Inventory (MPI) is a collection of 120 questions measuring human personality traits, these are some prominent datasets used for distributional alignment evaluation. Similarly, there are also several training datasets comprising answer preference pairs from traditional alignment literature applicable in distributional alignment (see Appendix Table A1), both for evaluation and training.

• All Modes.

Feng et al. (2024b) proposed first benchmark covering all thee modes pluralistic alignment. They also re-purposed existing datasets, contributing a benchmark using ValueKaleido for Overton and Steerable; OpinionQA for Steerable; MoralChoice and GlobalOpinionQA for Distributional mode evaluation. Vital (Shetty et al. 2025) has taken the first steps, being the first comprehensive pluralistic alignment benchmark. And, subsequent works (Zheng et al. 2026; Zhong et al. 2025) have built on this benchmark and contributed improved pluralistic alignment methods. Similarly, a few culture-focused datasets: CulturalPalette (Yuan et al. 2024) and CulturalKaleido (Banerjee et al. 2025) have been contributed. Although these works do not evaluate all three modes, given the nature of the datasets, there is scope to support all three modes. However, they are limited to cultural aspects. Similarly, Vital is healthcare-focused.

5.2. Evaluation Metrics

Evaluation is important for alignment research, especially for the development of empirical alignment methods. We survey metrics that have been developed and used to evaluate pluralistic alignment. Fundamentally, these metrics are specific to alignment mode being evaluated. Other than standard text generation metrics (such as BertScore) or LLM-as-Judge evaluation, the most prominent and best Overton window coverage evaluation was defined in Feng et al. (2024b). It proposes a metric using natural language inference (NLI) model entailing whether a set of golden values is covered in the response. Auxiliary, they recommend pairwise win-rate evaluation among two responses (which response seems more pluralistic) using human and LLM annotators, though not scalable. The same was adopted in several subsequent works (Zheng et al. 2026; Zhong et al. 2025). Recently, Poole-Dayan et al. (2025) proposed the first framework to measure Overton pluralism, OvertonScore. They formalise it as a set-coverage metric with each value weighted as per the fraction of survey participants in support of it (instead of equal weights). Additionally, they propose a standard benchmark and demonstrate scope for further improvement for Overton coverage in current LLMs (achieving ≈0.4 of the maximum 1.0). More importantly, there is a strong correlation between OvertonScore and human judgement, providing a scalable and reliable metric for Overton evaluation and future system progress. The steerability of models is usually measured by the accuracy of such correct steering in the LLMs. Similarly, for Distributional alignment mode the model distribution with the target human distribution using the Jensen-Shannon distance, where lower values mean higher distribution similarity. This distance is averaged for each sample in the benchmark. Similarly, some works have used Earth-mover distance (or Wasserstein distance) for measuring similarity between distributions.

6. Discussion and Future Directions

While the research community has made substantial strides, pluralistic alignment is far from a solved problem. There are many under-explored and promising research directions in pluralistic alignment. In this section, we outline directions for future research in addition to highlighted points above (denoted by pointers).

• Dataset Challenges.

Most existing work assumes a single-turn setting with static user representations. However, in a practical setup, user representations should evolve across multiple interactions (i.e., multi-turn) (Chakraborty et al. 2024; Durmus et al. 2024; Klassen et al. 2024a). For example, a user’s stance on a topic may shift after exposure to new information, or certain value preferences may change depending on prior interactions (Berinsky 2017; Grimm 2010). As a first step, future work should prepare a dataset for evaluating pluralism in multi-turn settings, followed by methods to handle it. A comprehensive multi-modal benchmark remains challenging, similar to traditional alignment (Wu et al. 2023; Yin et al. 2024b). Moreover, pluralistic alignment must account for diverse perspectives across cultures and languages. However, as shown in Table Appendix Table A1, current datasets are predominantly English and Western-centric. Advancing this field requires focused efforts on better data representation.

• Better Pluralistic Methods.

In addition to the takeaways from Section 4. Current works treat alignment as a static solution; however, human preferences and values evolve over time. We suggest recent approaches evaluating fairness over time could be applied to temporal pluralism (Klassen et al. 2024b). There is also a lack of work ensuring pluralistic alignment in the context of agentic AI. Alamdari et al. (2024) sketches a connection in agents learning policies while being considerate of others in the environment and the future well-being. However, the paper lacks empirical demonstrations, and further work is needed to study the effectiveness of such alignment policies, ensuring collective social welfare while achieving their goals. Considering that AI agents are deployed in the wild, a pluralistic geo-alignment is also important where agents’ actions vary depending on geography, culture, legality, systems, and evolve over time. Such spatio-temporal alignment is advocated in Janowicz et al. (2025), and there is a need for future work on all fronts.

• Advancing Pluralistic Evaluation.

We highlight the need for multi-dimensional benchmarks (e.g., culture, norms, and demographic groups) for evaluating performance across varying contexts. For instance, medical advice might change depending on the culture (AlKhamissi et al. 2024; Shetty et al. 2025). Future work should develop improved pluralistic sensitivity metrics. Similarly, the reasoning integrity of pluralistic models remains under-explored. Evaluations should consider logical consistency, epistemic calibration, and causal understanding (Lanham et al. 2023; Wei et al. 2022). Finally, evaluation should go beyond utility and assess potential alignment taxes (Ouyang et al. 2022; Yuan et al. 2023) introduced by pluralistic alignment, including over-refusal (Cui et al. 2025), increased jailbreak susceptibility (Greenblatt et al. 2024), reduced expressivity (Kirk et al. 2024c; Meister et al. 2025), and related trade-offs (Perez et al. 2023; Wolf et al. 2024). We encourage the use of automated red-teaming (Wang et al. 2023), as well as jury (Bai et al. 2022b; Sorensen et al. 2024b) and debate-based (Michael et al. 2023; Xu et al. 2024) evaluation frameworks for studying value conflicts and their resolution (Kirk et al. 2024b). Improved evaluation also requires reliable auditing; interpretability might help, which we discuss next.

• Mechanistic Interpretability.

As these “black-box’’ LLMs are used for high-stakes decisions, it is crucial to understand their internal workings (or interpretability) (Lipton 2018; Shen et al. 2023; Vilone and Longo 2020). Although interpretability might not seem directly applicable to alignment, techniques in this area could still be beneficial. One particular direction, mechanistic interpretability (Bereska and Gavves 2024; Nanda et al. 2023), is an interesting avenue whose objective is to reverse engineer the reasoning processes of LLMs. It elucidates the internal mechanisms by which LLMs transform inputs into outputs, providing deeper insight into their internal workings. VISPA (Zheng et al. 2026) showed promising results by applying ideas from mechanistic interpretability at inference time to achieve pluralism. They steer models’ internal activations at run time to satisfy different pluralistic objectives depending on the context. We advocate for further research on the interplay between mechanistic interpretability and pluralistic alignment. This work could extend beyond post-hoc applications and also be used to monitor and guide the pre-training process, ensuring that desired objectives (i.e., interpretability evaluation metrics) are met (Naseem 2026; Sharkey et al. 2025).

• Other Framework Extensions.

Although the jury-pluralistic benchmark was briefly touched on in Sorensen et al. (2024b), it is another way of aggregating diverse preferences. In jury learning, the model aims to resolve these disagreements explicitly via a jury, i.e., groups in some proportions defined as per task determine the final prediction label. Other extensions include POPE (Huang et al. 2025), the first framework for offline pluralistic policy alignment in LLMs, which targets diverse rather than averaged preferences; deliberative approaches inspired by social science and law, sketching consensus-seeking in LLMs (Blair et al. 2025); and temporal pluralism to handle evolving values over time, adapting temporal fairness evaluation methods so AI reflects different stakeholders’ values at different periods (Klassen et al. 2024a; Klassen et al. 2024b). In agentic AI, work remains limited despite agents’ growing deployment: Alamdari et al. (2024) proposes that agents learning policies considerate of others’ future well-being can foster pluralistic alignment, though empirical evidence is lacking and further study is needed for collective welfare; Plurals (Ashkinaze et al. 2025) provides a multi-agent deliberation framework with persona-based social ensemble simulations, showing promise in case studies; and geo-alignment (Janowicz et al. 2025) advocates spatio-temporal adaptation to geographic, cultural, and legal contexts for agents in real-world spaces. Related ideas draw from legal theories on democratic legitimacy and deliberation (Caputo 2024), and political pluralism seeking (approximations of) neutrality amid subjectivity (Fisher et al. 2025). These conceptual extensions merit more empirical validation and integration into pluralistic alignment research.

• Misuse of Pluralistically Aligned AI.

There is a risk that AI systems could be used for manipulative purposes, such as influencing people’s political opinions and social behaviours, if they are customised to closely match personal values. Additionally, users’ autonomy may be diminished by hyper-individualised human-AI interaction, endangering independent thought. Safeguards should be in place to guarantee that AI systems empower users rather than manipulate them based on their personal values, preserving diversity and fairness in the process. The current research in this space have only evaluated pure performance, attacks are not evaluated (Ji et al. 2023a). Although there is a need for more datasets for pluralistic alignment, one must be aware of potential privacy infringement. These alignment techniques use data that contains private details, such as individuals’ values and preferences (Jiang et al. 2025d). This is further exacerbated by the anthropomorphisation of LLMs through user tailoring to their values, which increases the risk of further private information leakage. Furthermore, when using real-world data, researchers must obtain consent and be transparent about the data usage to the users.

7. Conclusion

We highlighted the critical role of pluralistically aligned LLMs for improving their overall safety. We analysed pluralistic alignment through a framework in three modes: Overton, Steerable, and Distributional, providing a comprehensive overview of current strategies and their applications. By reviewing the relevant literature, datasets, and evaluation measures, we identified important research gaps and outlined the core limitations of current methodologies. Future research should aim to broaden pluralism, encompassing a wider range of applications and dynamic contexts. By further advancing pluralistic alignment approaches to be more adaptive and context-aware, we can support the development of AI systems that better reflect diverse human perspectives while remaining robust, reliable, and appropriately balanced in terms of user helpfulness.