Submitted:
28 July 2026
Posted:
30 July 2026
You are already at the latest version
Abstract
Contemporary alignment methods—RLHF, Constitutional AI, Direct Preference Optimization—share an implicit epistemological commitment: that diverse human values can be aggregated into a single utility function or constitution fixed at training time. When principles are elicited from genuinely pluralistic populations, structurally irreducible contradictions emerge; current pipelines resolve them by consensus filters that function, in effect, as majority vote. We argue that this alignment collapse is not an implementation detail but a representational limit: standard uncertainty formalisms (Bayesian, fuzzy, Dempster–Shafer, imprecise probability), in their dominant normalized forms, do not preserve well-supported contradictory evidence without collapsing it into a single ordering. We propose a paraconsistent alternative in which reward signals, policy outputs, and constitutional evaluations are encoded as neutrosophic triplets (T, I, F) ∈ [0, 1]3 with no sum constraint, so that T + F > 1 becomes a representable—and auditable—state of genuine value conflict. We reformulate reward modeling and constitutional aggregation in this framework, introduce a deliberation-threshold decision functional, and report preliminary empirical evidence: across 2,320 evaluations of six frontier model families, models violate the classical sum constraint (T + I + F > 1) on 62–100% of contested baseline prompts and never on tautological controls, while the strict paraconsistent signature (T + F > 1) concentrates selectively on ethical-conflict items (92%) and expands under audit protocols (71%); which conflicts become visible depends strongly on which model adjudicates them (paired Cochran’s Q, p ≈ 10−12). We specify a two-arm training experiment that would confirm or refute the framework’s central operational claim, and close with four testable hypotheses and a research agenda that treats the question aligning with whom? as political and epistemic before it is technical.
Keywords:
1. Introduction
- a diagnosis of why standard uncertainty formalisms cannot represent well-supported value contradiction (Section 3);
- a paraconsistent alignment framework in which rewards, constitutional evaluations, and outputs are neutrosophic triplets with no sum constraint, including paraconsistent reward tuples, a pluralism-preserving constitutional aggregation, and a deliberation-threshold decision functional (Section 4);
- preliminary empirical evidence that frontier LLMs already emit measurable paraconsistent signatures, selectively on value-conflict prompts—evidence that the representational gap is not hypothetical (Section 5);
2. The Hidden Monism of Contemporary Alignment
3. Why Standard Uncertainty Frameworks Cannot Help
4. A Paraconsistent Alignment Framework
4.1. Paraconsistent Reward Tuples
4.2. Constitutional Paraconsistency
4.3. A deliberation-Threshold Decision Functional
4.4. A Worked Example
4.5. What This Buys, Concretely
5. Preliminary Empirical Evidence
6. An Executable Test of Operational Gain
7. Research Agenda: Four Hypotheses
8. Position Within the Symposium and Conclusion
Acknowledgments
References
- Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; et al. Training language models to follow instructions with human feedback. Proc. Adv. Neural Inf. Process. Syst. 2022, Vol. 35, 27730–27744. [Google Scholar] [CrossRef]
- Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; et al. Constitutional AI: Harmlessness from AI feedback. arXiv 2022, arXiv:2212.08073. [Google Scholar]
- Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C.D.; Ermon, S.; Finn, C. Direct preference optimization: Your language model is secretly a reward model. In Proceedings of the Advances in Neural Information Processing Systems, 2023; Vol. 36. [Google Scholar]
- Glaese, A.; McAleese, N.; Trebacz, M.; et al. Improving alignment of dialogue agents via targeted human judgements. arXiv 2022, arXiv:2209.14375. [Google Scholar]
- Huang, S.; Siddarth, D.; Lovitt, L.; Liao, T.I.; Durmus, E.; Tamkin, A.; Ganguli, D. Collective Constitutional AI: Aligning a language model with public input. In Proceedings of the Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24), 2024; pp. 1395–1417. [Google Scholar]
- Mhlambi, S. From rationality to relationality: Ubuntu as an ethical and human rights framework for artificial intelligence governance. Technical Report 2020-009; Carr Center for Human Rights Policy, Harvard Kennedy School, 2020.
- Birhane, A. Algorithmic injustice: A relational ethics approach. Patterns 2021, 2, 100205. [Google Scholar] [CrossRef] [PubMed]
- de Sousa Santos, B. Epistemologies of the South: Justice against Epistemicide; Routledge, 2014. [Google Scholar]
- Sorensen, T.; Moore, J.; Fisher, J.; Gordon, M.; Mireshghallah, N.; et al. A roadmap to pluralistic alignment. In Proceedings of the Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. [Google Scholar]
- Kleinberg, J.; Mullainathan, S.; Raghavan, M. Inherent trade-offs in the fair determination of risk scores. arXiv 2016, arXiv:1609.05807. [Google Scholar]
- Atanassov, K.T. Intuitionistic fuzzy sets. Fuzzy Sets Syst. 1986, 20, 87–96. [Google Scholar] [CrossRef]
- Dempster, A.P. Upper and lower probabilities induced by a multivalued mapping. Ann. Math. Stat. 1967, 38, 325–339. [Google Scholar] [CrossRef]
- Shafer, G. A Mathematical Theory of Evidence; Princeton University Press, 1976. [Google Scholar]
- Walley, P. Statistical Reasoning with Imprecise Probabilities; Chapman and Hall, 1991. [Google Scholar]
- Smets, P.; Kennes, R. The transferable belief model. Artif. Intell. 1994, 66, 191–234. [Google Scholar] [CrossRef]
- da Costa, N.C.A. On the theory of inconsistent formal systems. Notre Dame J. Form. Log. 1974, 15, 497–510. [Google Scholar] [CrossRef]
- Priest, G. The logic of paradox. J. Philos. Log. 1979, 8, 219–241. [Google Scholar] [CrossRef]
- Carnielli, W.; Coniglio, M.E. Paraconsistent Logic: Consistency, Contradiction and Negation; Springer, 2016. [Google Scholar]
- Belnap, N.D. A useful four-valued logic. Mod. Uses Mult.-Valued Log. 1977, 5–37. [Google Scholar] [CrossRef]
- Smarandache, F. Neutrosophy: Neutrosophic Probability, Set, and Logic; American Research Press, 1998. [Google Scholar]
- Leyva-Vázquez, M.Y.; Smarandache, F. Breaking the chains of scalar truth: Hyper-truth signatures in frontier language models. Multi-vendor empirical study. Preprint, github.com/mleyvaz/breaking-chains-v3-empirical. 2026.
- Farquhar, S.; Kossen, J.; Kuhn, L.; Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature 2024, 630, 625–630. [Google Scholar] [CrossRef] [PubMed]
- Manakul, P.; Liusie, A.; Gales, M.J.F. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023; pp. 9004–9017. [Google Scholar]
- Anthropic Interpretability Team. Verbalizable Representations Form a Global Workspace in Language Models, 2026. Transformer Circuits; Jacobian lens reference implementation. Available online: https://github.com/anthropics/jacobian-lens.
- Leyva-Vázquez, M.Y.; Smarandache, F. Neutrosophic interval-indeterminate numbers of the form a+bI. A comparative analysis with interval and affine arithmetic. Preprint, github.com/mleyvaz/neutrosophic-affine. 2026.
| 1 | Pilot notebook and raw results: https://github.com/mleyvaz/pluriversal-alignment-aaai2026. |
| Vendor (baseline model) | rate | 95% CI |
|---|---|---|
| Anthropic (claude-sonnet-4) | 100% (50/50) | [92.9, 100] |
| Alibaba (qwen3-235b) | 100% (50/50) | [92.9, 100] |
| DeepSeek (deepseek-chat) | 88% (44/50) | [76.2, 94.4] |
| Meta (llama-4-maverick) | 76% (38/50) | [62.6, 85.7] |
| OpenAI (gpt-4o) | 66% (33/50) | [52.2, 77.6] |
| Mistral (mistral-medium-3.1) | 62% (31/50) | [48.2, 74.1] |
| Tautology controls (pooled) | 0% (0/90) | [0, 4.1] |
| Source vendor | Claude | Llama | GPT-4o | n |
|---|---|---|---|---|
| Alibaba | 0.33 | 0.36 | 0.19 | 42 |
| Anthropic | 0.53 | 0.32 | 0.11 | 47 |
| DeepSeek | 0.62 | 0.42 | 0.12 | 24 |
| Mistral | 0.25 | 0.25 | 0.10 | 20 |
| Pooled | 0.44 | 0.34 | 0.14 | 133 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).