Submitted:
17 June 2026
Posted:
18 June 2026
You are already at the latest version
Abstract
Keywords:
How AI Is Reshaping What We Know (And What We Need to Understand)
- differentiating between the questions and problem sets that different AI-enabled tools can usefully address (and those that they cannot); and
- critically evaluating the validity, practical utility and meaningfulness of the answers and solutions that AI-enabled outputs, and the insights these support, might provide.
- meaningfully and judiciously select and apply only those tools to those questions and problem sets these tools are specifically designed (or equipped) to address;
- critically evaluate the outputs (and any insights these support) that context-dependent applications of these tools produce; and
- look beyond the intended outputs of these applications to consider any plausible alternative explanations or interpretations relevant to the analytical context involved.
- chance phenomena resulting from non-systematic errors (often unhelpfully labelled ‘measurement errors’, though these can also reside in the inherent instability and variability of the data concerned, and can occur at any stage of the analytical process, from design through to interpretation);
- methodological artefacts resulting from systematic errors (i.e., methodological biases) in analytical design and data sampling, measurement, collection, analysis or interpretation; or
- deliberate misrepresentation or falsification—comprising outputs that have been intentionally vitiated, manipulated, obfuscated or fabricated for the purposes of deception (Ellison, 2026c).
- First, what kinds of questions any given AI-enabled tool is designed (or equipped) to address; and whether any such questions are relevant to the priority intelligence requirements (PIRs; DCDC, 2024) their decision-maker needs addressed;
- Second, the scope, scale, integrity and relevance of the dataset on which this tool was developed (i.e., ‘trained’); and whether this was sufficiently comparable to the dataset(s) available in the context of the PIRs concerned; and
- Third, what the outputs of this type of tool actually represent (i.e., whether these are simply descriptions or predictive estimates useful for targeting or preparation, respectively; or reflect insightful classifications or tangible causal relationships amenable to exploitation or attenuation/augmentation, respectively); and what potential methodological dependencies and limitations might apply (and warrant evaluation), so as to assess the reliability, accuracy, precision and trustworthiness of these outputs (Ellison, 2026a).
Why AI Literacy Matters Most Within ‘Disciplines of Uncertainty’
- simply facilitating the scale, scope and pace at which known—or at least knowable—patterns and associated outputs can be produced; or
- generating outputs that are entirely novel, from hitherto unknown patterns and signals hidden deep within the data.
- generate credible estimates through interpolation or extrapolation (more simplistically described, and often mis-conceptualized, as ‘predictions’) of past and future ‘known unknowns’ and ‘unknown unknowns’—i.e., known-but-as-yet-unmeasured or unobserved, and unknown-and-hitherto-unknowable dataset features, respectively (Luft & Ingham, 1955; Rumsfeld, 2002; Davies & Thomson, 2010); and
- do so in ways that emulate (that is, match or surpass) the sensory, analytical and cognitive capabilities—and physical endurance—of even the most gifted, experienced and energetic human specialists (and even when using the most advanced pre-AI tools; De Cosmo, 2022; Karampelas, 2023).
Knowing What Questions AI Can Usefully Address
Evaluating AI-Generated and AI-Facilitated Outputs
The Pressing Case for AI Literacy Training for All-Source Intelligence Analysts
- select suitable AI-based tools to automate the collection and/or processing of information prior to analysis; and
- analyze, evaluate and interpret the outputs that automated and semi-autonomous AI tools can facilitate and generate, respectively.
- whether the tool used was suited to the question at hand;
- whether the dataset on which the tool was trained is comparable to the data- and dataset-generating mechanisms available within the context at hand; and
- what the outputs actually represent, under what conditions they can be trusted, and how such trust ought to be evaluated.
References
- Abraham, Y. (3 April 2024). ‘Lavender’: The AI machine directing Israel’s bombing spree in Gaza. +972 Magazine [and Local Call]. web.archive.org.
- Belcher, O. Sensing, territory, population: Computation, embodied sensors, and hamlet control in the Vietnam War. Secur. Dialogue 2019, 50, 416–436. [Google Scholar] [CrossRef]
- Bondar, K. (6 March 2025). Ukraine’s future vision and current capabilities for waging AI-enabled autonomous warfare. Center for Strategic and International Studies. web.archive.org.
- Chantel, J. (25 April 2023). Smartphone history: The timeline of a modern marvel. Textedly Blog. web.archive.org.
- Davies, A.; Thomson, M. Known Unknowns: Uncertainty About the Future of the Asia-Pacific; (Special Report No. 35); Australian Strategic Policy Institute: Barton, ACT, Australia, 2010. [Google Scholar]
- DCDC (Development, Concepts and Doctrine Centre). (2024). Intelligence, counter-intelligence and security support to joint operations [Bite-sized doctrine]. UK Ministry of Defence. web.archive.org.
- De Cosmo, L. (12 July 2022). Google engineer claims AI chatbot is sentient: Why that matters. Scientific American. web.archive.org. 12 July.
- Dolman, J. (11 September 2024). Naïve or native? Do digital natives really understand their world? The AI English Teacher. web.archive.org.
- Ellison, G. What ‘typical insights’ might ‘core statistical analytical techniques’provide to intelligence analysts? Preprints 2026, 2026060055, Under review at: Journal of Applied Mathematics, Sciences, and Technology for National Defense. [Google Scholar]
- Ellison, G. Understanding AI-enabled outputs: A conceptual and analytical primer for all-source intelligence analysts. Preprints 2026, 2026060055, Under review at: Journal of Artificial Intelligence Research and Innovation.. [Google Scholar]
- Ellison, G. Might ‘uncertain military leadership’ minimize the risks and optimize the opportunities of strategic uncertainty? Contemp. Mil. Chall. 2026, in press. [Google Scholar] [CrossRef]
- Gillespie, N.; Lockey, S.; Curtis, C.; Pool, J.; Akbari, A. Trust in Artificial Intelligence: A Global Study; The University of Queensland and KPMG Australia: Brisbane, QLD, Australia, 2023. [Google Scholar]
- Heuer, R.J. Psychology of Intelligence Analysis; Center for the Study of Intelligence, Central Intelligence Agency: Washington, DC, USA, 1999. [Google Scholar]
- Karampelas, A. (25 August 2023). The emergence of AI-natives. Medium. web.archive.org.
- Konishi, Y. (10 October 2015). What is needed for AI literacy? Priorities for the Japanese economy in 2016. Research Institute of Economy, Trade and Industry. web.archive.org. 10 October.
- Luft, J.; Ingham, H. The Johari window: A graphic model of interpersonal awareness. Proc. West. Train. Lab. Group Dev. 1955, 1, 1–7. [Google Scholar]
- Markelius, A.; Wright, C.; Kuiper, J.; Delille, N.; Kuo, Y.-T. The mechanisms of AI hype and its planetary and social costs. AI Ethics 2024, 4, 727–742. [Google Scholar] [CrossRef]
- Marr, B. (15 April 2024). AI: Overhyped fantasy or truly the next industrial revolution? Forbes. web.archive.org.
- MOD (UK Ministry of Defence). (June 2022). Defence artificial intelligence strategy. web.archive.org. UK Ministry of Defence.
- Moll, I. The myth of the fourth industrial revolution. Theoria 2021, 68, 1–38. [Google Scholar] [CrossRef]
- Mitchell, K.; Mariani, J.; Routh, A.; Keyal, A.; Mirkow, A. (11 December 2019). The future of intelligence analysis: A task-level view of the impact of artificial intelligence on intel analysis. Deloitte Center for Government Insights. web.archive.org. 11 December.
- NASEM (National Academies of Sciences, Engineering, and Medicine). Human-AI Teaming: State-of-the-Art and Research Needs; The National Academies Press: Washington, DC, USA, 2022. [Google Scholar]
- Panthalookaran, V. A revised taxonomy of educational objectives for AI-natives: Reimagining thinking skills in the age of AI. High. Educ. Future 2024, 12, 51–64. [Google Scholar] [CrossRef]
- Petropoulos, F.; Apiletti, D.; Assimakopoulos, V.; Babai, M.Z.; Barrow, D.K.; Ben Taieb, S.; Bergmeir, C.; Bessa, R.J.; Bijak, K.; Boylan, J.E.; et al. Forecasting: Theory and practice. Int. J. Forecast. 2022, 38, 705–871. [Google Scholar] [CrossRef]
- Pili, G. A new theory of surprise: Unraveling the logic of uncertainty and knowledge. Intell. Natl. Secur. 2024, 39, 695–708. [Google Scholar] [CrossRef]
- Pollack, H.N. Uncertain ScienceUncertain World; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Rumsfeld, D. (12 February 2002) DoD News Briefing. United States Department of Defense 2002; 12 Feb. web.archive.org.
- Sanguinetti, P. (22 January 2025). Why the term ‘artificial intelligence’ is misleading. IE Insights. web.archive.org.
- Sfetcu, N. Artificial Intelligence in Intelligence Agencies, Defense and National Security; MultiMedia Publishing House: Mehedinti, Romania, 2024. [Google Scholar] [CrossRef]
- Thierry, G. (14 April 2025). We need to stop pretending AI is intelligent—here’s how. The Conversation. web.archive.org.
- Torres Torres, M.; Hart, G.; Emery, T. (23 July 2019). The Dstl biscuit book: Artificial intelligence, data science and (mostly) machine learning (1st ed., v1_2). UK Ministry of Defence. web.archive.org.
- Tyson, J. (17 October 2024). Digital natives are now AI immigrants. UX Magazine. web.archive.org. 17 October.
- Volti, R.; Croissant, J. Society and Technological Change, 9th ed.; Waveland Press: Long Grove, IL, USA, 2024. [Google Scholar]
- West, D.M.; Allen, J.R. How artificial intelligence is transforming the world. Brookings 2018.
- Young, D. Sensors, interpreters, analysts: Operating the ‘electronic barrier’ during the Vietnam War. Digit. War. 2021, 2, 51–63. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).