Might Wargaming be Another Instance Where “Anything You Can Do, AI Can Do Better”?

George Ellison; Andrew Shepherd

doi:10.20944/preprints202401.1311.v1

Submitted:

17 January 2024

Posted:

17 January 2024

You are already at the latest version

Abstract

This article offers a pragmatic ‘epistemology of wargaming’ which views wargames as immersive ‘thought experiments’ in which the human players involved use their experiential, empirical, and theoretical knowledge – together with whatever cognitive models they are able to deploy, or develop anew – to generate a conceptual, operational understanding of the adversarial scenario in which they are immersed; and exploit this understanding to craft tactical decisions designed to optimise the likelihood they will achieve their strategic objectives. From this perspective, contemporary interest in the use of ‘AI’-enabled tools to augment the validity of wargaming outputs – where these outputs constitute the decisions players make and the insights such decisions reveal – might most purposefully focus on: the design and implementation of wargames (to strengthen the architecture these provide to support immersive decision-making); and the analysis of players’ decisions (to better understand the cognitive models these involve and reflect). While the focus we suggest might disappoint those keen to replace human players with (semi-) autonomous decision-making machines, as long as the principal objectives of wargaming are to assess and enhance the decision-making capabilities of human players and human personnel, ‘AI’-enabled applications can only ever play a supporting role (albeit a potentially invaluable one) in the design, presentation, implementation, and analysis of wargames. As Irving Berlin might have it: ‘AI’ might soon be able to do most things better than us, but it can never replace humans when only a human will do.

Keywords:

Wargaming

;

Artificial Intelligence

;

AI

;

Decision-making

Subject:

Social Sciences - Decision Sciences

Introduction

The recent renaissance in military wargaming2^,3 – both analogue4 and digital5 – has coincided with a step change in applications using advanced, machine learning techniques (and in the hype surrounding ‘artificial intelligence’; ‘AI’).6 This has involved a dramatic shift in the performance, capability and application of computer-generated algorithmic protocols that can: elucidate ambiguous or hidden features within complex real world datasets; and estimate/simulate ‘ersatz’ data whose values and distributions are consistent with, and faithfully represent, these (real world) features.7 Such protocols have more recently been used to emulate increasingly complicated natural phenomena, and fabricate material artefacts including images and sounds, speech and behaviour. More recently still, the machines used to generate such phenomena/artefacts have been ‘trained’ (using ‘reinforcement learning’) to target the sensory and cognitive experiences these can provoke – which appear to include the suspicion, belief or delusion that the machines involved may actually be sentient or ‘alive.8

Similar advances in the affordability and accessibility of personal computing during the early 1980s led to the development of novel, computer-based, and computerised/automated wargaming systems. Unsurprisingly, wargaming professionals are keen to consider how recent advances in machine learning applications and ‘AI’ might enhance and extend the quality, reach, and impact of wargames.^2,9 Our article aims to inform these considerations by addressing the last of the four key recommendations emanating from the most recent (Dstl-funded) review of ‘AI’ in wargaming – undertaken by the Centre for Emerging Technology and Security [CETaS] at the Alan Turing Institute.10 This recommendation called for further research into “wargaming epistemology and decision-making in wargaming… [to support] the design of AI-enabled tools that augment rather than add uncertainty to the validity of wargaming outputs”.

To this end, we first set out to elucidate an “epistemology of wargaming” before examining each of the subsidiary processes involved in designing and planning, executing and playing, a wargame.^2,9,10 These processes themselves provide a substantive source of insight into the information, structures, rules and procedures that are available to wargame designers, facilitators, adjudicators and players; and those that are considered appropriate, necessary, or sufficient to incorporate within the wargame concerned. However, such insights are somewhat incidental (if nonetheless integral) to the ‘architecture’11 that wargames provide to: provoke the decisions that players must make; reveal the insight their decisions reflect; and generate the foresight these decisions might infer or evoke. For this reason we have interpreted the “wargaming outputs” to which the CeTAS review refers¹⁰ as the decisions made by the players involved, rather than those made by the wargame’s designers and planners, facilitators and adjudicators. We nonetheless recognise that these (subsidiary, planning and operational) decisions also constitute tangible “wargaming outputs”. Although secondary, these ‘secondary outputs’ offer substantive insights that should prove invaluable for improving wargaming design and practice; and evaluating the role that “AI-enabled tools” might play therein.^10,12 For this reason, once we have elucidated a functional epistemological basis for the decisions that players make during the course of a wargame, we will examine how the subsidiary processes involved in designing, planning and executing a wargame might enhance or constrain: the decisions that players make (and are able to make); the insight that might be extracted from their decisions; and any foresight that might be generated as a result of these decisions. We will then conclude by evaluating what role “AI-enabled tools” might play in: capturing and interrogating the decisions that players make; strengthening the subsidiary wargaming processes on which players’ decisions rely; and enhancing the value of the insights provided.

Indeed, the value that can be had from wargaming has led to its use in a diverse range of applications, including:

Three overarching “aim[s] and purpose[s]”, namely: “creating knowledge”; “conveying knowledge” and “team building”;13 and
Seven “applications”: three of which focus on “the details of decisions” (“analysis/research”, “capability development” and “support to operations”); three on “decision makers” (“training/education”; “decision making” and “command teams”); and one (“decision support”) on both.14

As such, there is value to be added not only to the insight that players’ decisions might reveal (and the foresight such insight might support), but also to a range of other ancillary ‘outputs’, including: the competencies and operational preparedness/effectiveness of the players involved; the experience, expertise, and professionalism of those contributing to the wargame’s design, planning, execution, facilitation, and adjudication; the variety, quality and consistency of wargames available to support these applications; and the confidence required to critically evaluate the application of wargames, and the insight, foresight, and associated capabilities they (cl)aim to produce – the latter being particularly important wherever the evidence to support such (cl)aims is missing, incomplete, equivocal, contested, or vulnerable to bias.

Towards a Pragmatic Epistemology of Wargaming, and of Decision-Making Within Wargaming

To guide our epistemological deliberations, we have adapted and extended the definition of wargaming contained in the online (pre-)reading pack provided to delegates at the Connections UK 2017 conference (itself based on Peter Perla’s 2008 definition).15 On this basis, our working definition of ‘serious’ or ‘professional’² wargames (as opposed to ‘recreational’ wargames) views these as:

“immersive, imaginary, adversarial scenarios in which the flow of events shapes, and is shaped by, decisions made by […] human players in accordance with explicit and predetermined rules operating within constraints imposed by: the wargame’s design and contextual/operational domain(s); and the roles assigned to each player” (Perla’s original text¹⁵ in italics).

Put more simply, we view wargames as ‘thought experiments’16 in which the players involved use their experiential, empirical, and theoretical knowledge – together with whatever cognitive models (or ‘heuristics’) they are able to deploy or develop anew – to generate a conceptual, operational understanding of the adversarial scenario in which they are immersed; and exploit this understanding to craft tactical decisions designed to optimise the likelihood they will achieve their strategic objectives. As such, we propose that a pragmatic ‘epistemology of wargaming’ – that is, a functional and operational understanding of the process(es) and mechanism(s) involved in the production of knowledge by players taking part in a wargame – might simply constitute the conceptual models the players create so as to fulfil their decision-making responsibilities to the best of their knowledge and ability.

From this perspective, the “validity” of insights generated as a direct consequence (and as intentional “outputs”)^2,10 of the decisions that wargaming entails, relies less on whether the decisions that players make successfully optimise the likelihood they will achieve their strategic objectives, and more on whether the decisions faithfully reflect the conceptual models the players concerned have deployed or developed. This is because judging the ‘validity’ of a decision on the basis of its subsequent success mistakes decision-making as simply instrumental, definitive, and binary (i.e. either ‘right’ or ‘wrong’), when in practice decisions are more often judgemental and tentative, or conjectural and speculative. At the same time, in those wargames where an element of chance (such as a roll of two or three dice) is used to introduce an element of uncertainty, variability, ‘fog’ or ‘friction’,17 it is unlikely to be possible (or even practicable) to assess whether any decision successfully optimises the likelihood of an outcome where the outcome of that decision partly depends on chance. Moreover, since the conceptual models available to (and created by) the same player at different times – and by different players, with different experiential, empirical and theoretical knowledge, and different heuristics, at their disposal – are likely to vary substantively, it is also unlikely that the ‘validity’ of the insights generated by the decisions players make will be evident in the observed consistency/reliability of their decisions, even when these are made in exactly the same phase of an identical wargaming scenario.

How then might the ‘validity’ of a wargame’s outputs (i.e. its ‘decision-dependent insights’) be judged if: this cannot be derived from the outcomes these decisions elicit; these outputs depend upon the intangible (and potentially unmeasurable)18 cognitive models of the players involved; and the nature of these models (and the insight they represent) might only be inferred from the decisions the players make? Worse still, how might validity be reliably assessed when players can come up with identical decisions/solutions on the basis of substantively different cognitive models? Under such circumstances a player’s decisions (and their subsequent impacts and outcomes) offer: only a very loose indication of the actual cognitive model(s) involved; and little if any insight into how these model(s) might operate. Wargaming therefore faces a substantial challenge whenever its outputs comprise the insights generated on the basis of players’ decisions; and these insights offer the only clues as to the cognitive models deployed/developed by the players involved. Although such insights are not necessarily the only (or even the principal) intended outputs of serious/professional wargames (as evident in the seven different “applications” summarised above),¹⁴ they are nonetheless central to the rationale for wargaming, and to the emphasis wargames place on player decisions and decision-making.^2,9 There is therefore a pressing need to: develop consensus on the information and analysis required to best determine the nature of the cognitive models that underlie the decisions that players make;¹⁸ and use these techniques to strengthen the validity and value of the insight these decisions provide (and the foresight this insight might evoke). This is a need to which we will return in the concluding section of this article.

Improving the Validity of Decision-Related Insights through Wargame Design, Planning and Execution

To explore how the subsidiary processes involved in designing, planning and executing a wargame might enhance or constrain the decisions that players make – and (the validity of) the insights these decisions provide – it is worth examining four of the critical ingredients we considered essential to, and definitive of (serious/professional) wargames in the working definition proposed earlier. These are:

The necessity of crafting wargames that enable players to become immersed in the game’s imaginary scenario so that their decisions not only depend upon their prior decision-making experience, skills and expertise, but are also substantively informed by: the scenario-related and domain-specific information provided for/available to them (including the wargame’s predetermined rules); and the roles, attributes and objectives to which they have been assigned;
The inherently adversarial nature of wargames, through which insight is generated as a result of the crucial decisions that opposing players make when interacting as adversaries with incompatible or antagonist objectives (such as securing exclusive access to essential or critical resources), rather than simply as contestants or competitors who have a shared need or desire for limited resources/rewards;
The need to: decide in advance (i.e. predetermine) the rules that will be applied when determining the success or failure of each player’s decisions; and ensure that these rules are made known (i.e. are explicit) to the players concerned, so that they can craft decisions designed/intended to maximise their chances of success (while minimising the likelihood or impact of any residual possibility of failure); and
Their central analytical focus on the decisions that human players make – and the impact these decisions have on both the flow of events and (subsequent) decisions – decisions that are commonly considered a wargame’s principal ‘outputs’ and sources of insight.^{2,4,9,10,13,14}

The first of these ingredients may prove less critical to the successful implementation of a wargame where the players involved have the willingness and commitment, experience, aptitude and ability to role-play within incomplete, inelegantly prepared and inexpertly presented wargames (and not least because all serious/professional wargames need to be technically ‘imperfect’).19 However, in such circumstances the ‘immersivity’ of the wargame’s imaginary scenario is likely to be excessively ‘co-produced’ by the players themselves (as they ‘muddle along’ and ‘fill in the gaps’). As a result, the content and ‘feel’ (i.e. the mood, atmosphere and temperament) of such wargames can deviate substantively from that intended by the game’s designers – thereby undermining the reliability and validity of any insight(s) generated therefrom. Such games will also be vulnerable to any disparities in commitment or experience amongst opposing players, which can influence: the effort, thoughtfulness, and purpose they bring to bear in the decisions they make; and their subsequent chances of success (or failure). Such biases would constitute a pernicious flaw within wargames intended to generate insight into what decisions are developed by, available to, and considered optimal by, the players involved.

As for the adversarial nature of (serious/professional) wargames – where the strategic objectives of opposing combatants are incompatible (or better still, irreconcilable) – we consider this an essential/definitive ingredient because it elevates the stakes to a sufficiently critical level that the players must focus intently, diligently and even ruthlessly on the decisions they craft to optimise the likelihood that they will prevail. From this perspective, adversariality constitutes an integral part of the immersivity required to ensure that players commit, engage and strive – to the best of their ability – to make the very best decisions they possibly can. As such, adversariality adds an edge to the contest that sets ‘serious’² and ‘professional’^13,14 wargames apart from ‘recreational’ wargames (and other competitive ‘games’), in which any conflict can be somewhat contrived, and ostensibly irrelevant.

Meanwhile, the reason why a (serious/professional) wargame must have rules that are both: predetermined (that is, established in advance of any decisions that players make); and explicit (so the rules are known to the players before they make their decisions), is again to ensure they are encouraged to make (and rewarded for making) decisions that optimise the likelihood of success (albeit within the limits and constraints these rules impose/prescribe). Indeed, this ingredient is likely to be critical to the success of any wargame that aims to ensure that players make the very best decisions they can. And this will remain the case even when the outcomes of their decisions are also determined (if only in part) by chance – as is often the case with: wargames designed to include a (more realistic) level of ‘contextual instability’, uncertainty, ‘fog’ and ‘friction’;¹⁷ and those keen to avoid players crafting decisions simply to ‘beat the rules’,¹⁹ as opposed to achieving their longer term strategic objectives (albeit, again, within the constraints imposed by their role, assets and adversaries and by the wargame’s scenario and context).

Without predetermined rules any game becomes either a game of chance or a game whose outcomes depend solely on the whim of the facilitators and adjudicators involved. While this does not mean that a wargame’s rules cannot be changed before or after each decision-making ‘turn’ – which might be desirable when facilitators/adjudicators want or need to (re)direct the “flow of events” – but any such changes must be known to the players in advance of their (next) decision, so that: the revised rules can inform the decisions they make; and they do not get discouraged when (unbeknown to them) undeclared rule changes undermine the acuity of the decisions they make, and the likelihood of their success, rendering these somewhat arbitrary and thereby disheartening. In this regard, explicit and predetermined rules are also instrumental in ensuring players can become immersed in the wargame’s contextual scenario, and can commit to making the best possible decisions available to them (albeit, once more… within the constraints imposed by these predetermined rules).

Finally, the last of these four essential/definitive ingredients arguably owes more to this article’s focus on traditional/commonplace wargaming practices,⁴ and to the role that humans retain in tactical, operational and strategic decision-making, than to likely developments in ‘artificial’ decision-making applications (i.e. those applications designed to augment or replace one or more of the human players involved). Indeed, the most recent review of ‘AI’ applications in wargaming¹⁰ cited multiple open-source reports of unclassified projects pitting ‘AI-enhanced’ adversaries against humans (or against one another) to better understand/exploit their relative strengths and weaknesses; and it seems likely that many similar projects are underway behind closed doors or at higher classifications. While these developments lie beyond the scope of this article, they warrant a similar level of consideration given: the likely pace of advances in machine-generated decision support;⁵ and enduring interest in machines considered capable of making ‘independent’ decisions (much of which may yet reflect wildly unrealistic expectations regarding the likelihood that ‘AI’ applications will ever, might soon or already operate as ‘thinking machines’ with ‘general intelligence’).⁹

These ingredients are central to the ‘architecture’¹¹ that wargame designers and planners must erect to support the immersive (and imaginary) scenarios in which adversarial decision-making can take place. They are also integral to the facilitation and adjudication required to: support players navigating and operating within these (imaginary) scenarios; ensure players are appropriately motivated to make suitably informed decisions; and generate outcomes as a result of these decisions that are credible and relevant to the players’ immediate (tactical and operational) and ultimate (strategic) objectives.^2,14 The subsidiary processes involved in developing and implementing a wargame’s architecture can therefore be broadly classified as those occurring during a preceding ‘preparatory’ and subsequent ‘operational’ phase:

The first, ‘preparatory’ phase comprises: the design of the wargame, its associated aims and objectives, rules and procedures, outputs and outcomes; the collation of the resources/personnel available/required for implementation (in the subsequent, ‘operational’ phase); and the collection of data generated during the subsidiary processes necessary for wargame preparation.
The subsequent, ‘operational’ phase comprises: the facilitation and adjudication required to generate decision-making outputs and determine their consequential outcomes; the decisions players make to generate these outputs and outcomes; and the collection of data generated during the subsidiary processes necessary for wargame implementation.

These two phases, and examples of their subsidiary processes, have been summarised in Figure 1 in the form of the resources, actions and related events that can offer insights relevant to enhancing the design, planning, execution, facilitation and adjudication of the wargame. In the process, such insights should help strengthen the wargame’s capacity to support immersive, adversarial decision-making that can itself offer substantial insight into both: the range and variety of decisions that players can make; and how players go about making these decisions. However, the activities required to exploit the insights available during each of these two phases necessitate a third (‘analytical’) phase with its own subsidiary processes – as summarised in the dark red box to the right hand side of Figure 1, and as outlined below,

A final, ‘analytical’ phase comprises: the collation, description and interrogation of data collected during the two preceding phases; the analysis of these data to identify associations/relationships, patterns and features that might inform both improvements in wargame design and implementation and the interpretation of players’ decisions (and their subsequent outcomes) – and thereby generate insights regarding the deliberative processes and cognitive models most likely to be involved (and any foresight such insights might reveal or inform).

As such, this final phase offers two critical opportunities to strengthen and “augment… the validity of wargaming outputs” by:

identifying potential improvements in the design/implementation of wargames that make these more robust, effective, coherent and consistent tools/platforms for extracting the most thoughtful decisions possible from the players involved (albeit within the contextual and operational constraints imposed by the wargame’s imaginary scenario and predetermined rules); and
undertaking an in-depth interrogation/examination of the decisions that each player makes (in line with, and in response to, any preceding decisions they and their adversaries have made, and the subsequent consequences of these decisions); and comparing these to all of the decisions that were available to them (i.e. those permissible within the wargame’s contextual/operational constraints and predetermined rules).

From Figure 1 it seems clear that any (serious/professional) wargame could generate an extraordinary variety of insights that might be exploited to “augment… the validity of [their] outputs”, provided the planners, facilitators and adjudicators involved devote the time and resources required to: capture the wealth of information/data emanating from the wargame’s ‘preparatory’ and ‘operational’ phases (i.e. beyond recording any decision-related ‘outputs’ and their subsequent ‘outcomes’); subject these data to purposeful interrogation and analysis; and thereby identify what improvements might be made to the wargame’s ability to generate ‘the most thoughtful decisions possible’ from the players involved (and thereby more valid insights from, and more comprehensive understanding of, the deliberative processes and cognitive models players use when making these decisions).

However, capturing the diverse types of data (and insight) that are likely to be available during the preparation and execution of a wargame – data that might include physical and digital copies of documents and images, as well as audio/visual recordings of discussions, meetings and the ‘game itself – is unlikely to constitute a trivial task. Neither is collating, managing, formatting and analysing such data – since this also requires substantial time as well as specialist expertise and resources.20 It is therefore unsurprising that few wargames are able to exploit much of the accessible/available data, and many of the potential insights these data/analyses might generate. Instead, most wargames (predominantly) focus/rely on records of decision-related outputs and their subsequent consequences – together with feedback and the narratives provided by facilitators, adjudicators and players – as their principal sources of insight on decision-making (and on the effectiveness of wargaming for one or more of the seven discrete ‘applications’ summarised earlier).¹⁴ Indeed, the very different skills required to plan/execute a wargame, and to analyse/evaluate a wargame, make it unlikely that most wargame designers, facilitators and adjudicators can/should seek to extract further insight from their wargames (beyond that evident from the decisions players make, and the foresight these decisions might infer or evoke) without specialist advice and support.

Figure 1.

Conclusion: “Some of the Things You Need to Do, AI Might Do Better”

All contemporary ‘AI’ applications ultimately rely on comparatively simple machine learning techniques to: identify and elucidate self-evident, ambiguous, and hidden features within complex real world datasets; and estimate/simulate ‘ersatz’ data based thereon.⁷ However, the conditional automation of multiple, responsive, machine learning tasks, operating in near real time, can not only outperform humans across a range of complex sensory, cognitive, and analytical tasks; but can also complete many such tasks simultaneously, more accurately, and far more quickly and efficiently than human beings.⁶ As a result, a growing number of advanced ‘AI’ applications are becoming available which can imitate, emulate, and fabricate natural phenomena and material artefacts so proficiently that the ‘novel counterfeits’ these applications produce are increasingly (and challengingly) indistinguishable from the genuine article.21

There are therefore already a number of “AI-enabled tools” that might “augment… the validity of wargaming outputs” by improving the quality of the materials used to enhance a wargame’s ‘immersivity’. In the process, such improvements would strengthen the sensory ‘feel’ of the wargame concerned, and thereby the coherence and consistency of a player’s experience and performance. For example, ‘AI’ generated documents, images, audio files and video footage would not only lend an aura of authenticity to wargaming scenarios, but might also help to conceal, mask or mitigate their imaginary and fictitious nature.^5,9,22 There are, likewise, a number of advanced simulation and optimisation procedures⁹ that might be used to ensure that a wargame’s format, structure, content, and rules – and any element of chance included therein – permit/elicit the full range of decisions and outcomes desired. This approach should also make it possible to explore the likelihood of each of these (possible) decisions/outcomes occurring as a benchmark against which players’ decisions might then be compared and evaluated; and not least to assess how players’ prior experience, knowledge, expertise and training might influence the decisions they make; and how these decisions vary, change, improve, or decline during the course of (and as a result of) playing the wargame.^11,18

Elsewhere, there are a growing number of ‘AI’-enabled data capture and data preparation applications that can ingest complex and multi-source, structured and unstructured, ‘messy’ and ‘fuzzy’ data23 – including somewhat incidental and ephemeral observations and recordings. Together with recent advances in the diversity of sensors available for capturing raw contextual, social, psycho-biological and behavioural information, these applications could dramatically extend the variety and volume of information that might be extracted from both the preparatory and operational phases of a wargame – far beyond the traditional notes made during the former; and records of players’ decisions, and their subsequent consequences, during the latter.^2,4,20 Much methodological work will still be required to harness and extract meaningful insight from such data; and to make the techniques involved, and the expertise required, accessible to wargame designers, planners and facilitators. Yet recent advances in ‘AI’-enabled applications place these insights firmly within our reach; and while this work would help inform improvements in wargame design/implementation, it is also likely to be the only available, credible, and practicable approach to better understand the deliberative processes and cognitive models players use to make their decisions – an advance in understanding that would far surpass what players’ narratives, and the decisions they make, can currently reveal.²⁰

These exciting possibilities and opportunities aside, we remain several steps short of augmenting the “validity of wargaming outputs” using “AI-enabled tools”,¹⁰ and there is reason to believe the substantial investment required may be diverted elsewhere given the hype, enthusiasm and excitement surrounding the possibility of ‘AI’-enabled, (semi-)autonomous decision-making. This possibility is an aspiration (and, for some, a fear) that might render wargaming’s interest in human decisions somewhat obsolete were any/all of the decision-making therein (and elsewhere) to be taken over by ‘AI’. Whether ‘AI’ can ever, will ever, and should ever replace humans in strategic decision-making falls well beyond the scope of this article;24 but as long as the decisions that humans make have any influence on the outcomes achieved, there will be a place for (human-centric) wargames;⁴ and there will be merit in exploiting the opportunities that ‘AI’-enabled applications offer to augment their immersivity, insightful productivity and, yes, their validity.

Acknowledgements

This article was prepared for the ‘AI in Wargaming’ Deep Dive Panel at the Connections UK 2023 conference, Royal Military Academy Sandhurst on 5th - 7th September 2023, and has been accepted for publication in Cognitio. The slides and audio-recording of the presentation are available from the Connections UK 2023 website. We are indebted to, the Co-organisers of the conference (particularly Graham Longley-Brown); the Chair of the Panel (Major Tom Mouat - Head of the Defence Modelling and Simulation School); and the two other Co-Panellists (Anna Knack - Senior Research Associate in the Centre for Emerging Technology and Security [CETaS] at the Alan Turing Institute; and Lewis Griffin - Co-Director of the Centre for Mathematics and Physics in the Life Sciences and Experimental Biology [CoMPLEX] at University College London); and Thea de Wet (Professor of Anthropology and ADS Senior Director at the University of Johannesburg). The arguments presented in this article owe much to the ideas they aired, and the resources they have generated/shared,^8,10,13,14 while discussing the initial question posed by the conference Co-organisers, namely: “How might AI assist the design, development and execution of wargames (including data capture and analysis)?”

References

Kimball, R.; Emmet, L. The Complete Lyrics of Irving Berlin; Alfred A Knopf: New York, NY, USA, 2001; 560p, ISBN 0679419438. [Google Scholar]
By “military wargaming” we include all ‘serious’ and ‘professional’ wargames encompassed by paragraphs 1.5 and 1.6 within “What is Wargaming?” – Section 2, pp. 5–8 of: UK MoD. Wargaming Handbook. Development, Concepts and Doctrine Centre, Ministry of Defence; MoD Shrivenham, UK: 2017. 97pp. Accessed in July 2023 here, and archived as accessed here; and within: Smith RD. Military Simulation and Serious Games: Where We Came From and Where We Are Going. Modelbenders LLC; Oviedo, FL: 2009. 412pp. ISBN 0982304064.
Hirst, A. States of play: Evaluating the renaissance in US military wargaming. Critical Military Studies 2022, 8, 1–21. [Google Scholar] [CrossRef]
Brynen, R. Virtual paradox: How digital war has reinvigorated analogue wargaming. Digital War 2020, 1, 138–143. [Google Scholar] [CrossRef]
Reddie AW, Goldblum BL, Lakkaraju K, Reinhardt J, Nacht M, Epifanovskaya L. Next-generation wargames. Science 2018, 21, 1362–1364.
Vinsel, L. Don't get distracted by the hype around generative AI. MIT Sloan Management Review 2023, 64, 1–3. [Google Scholar]
By “ersatz data” we mean “substitute data”; though not necessarily “inferior” copies of ‘real-world’ or ‘naturally occurring’ data. See the definition provided by Oxford Languages here; accessed July 2023, and archived as accessed here.
Griffin LD, Kleinberg B, Mozes M, Mai KT, Vau M, Caldwell M, Marvor-Parker A. Susceptibility to influence of large language models. arXiv 2023, 2303.06074, 1-24. https://doi.org/10.48550/arXiv.2303.06074, and Burtell M, Woodside T. Artificial influence, An analysis of AI-driven persuasion. arXiv 2023, 2303.08721, 1-8. https://doi.org/10.48550/arXiv.2303.08721.
Goodman J, Risi S, Lucas S. AI and wargaming. arXiv 2020; 2009.08922: 1-53. DOI:10.48550/arXiv.2009.08922; and Davis PK, Bracken P. Artificial intelligence for wargaming and modeling. Journal of Defense Modeling and Simulation: Applications, Methodology, Technology 2022; Feb 8: 1-16. DOI:10.1177/1548512921107312615485129211073126.
Knack A, Powell R. Artificial Intelligence in Wargaming: An evidence-based assessment of AI applications. Centre for Emerging Technology and Security, Alan Turing Institute; London, UK. 2023, 1-58. Accessed in July 2023 here, and archived as accessed here. 20 July.
Butt AJ, Butt NA, Mazhar A, Khattak Z, Sheikh JA. The SOAR of cognitive architectures. Proceedings of the International Conference on Current Trends in Information Technology; Dubai, UAE: 20132013. pp. 135-142.
Landers RN, Marin S. Theory and technology in organizational psychology: A review of technology integration paradigms and their effects on the validity of theory. Annual Review of Organizational Psychology and Organizational Behavior 2021, 21, 235–258.
Mouat T. Wargaming 101. Connections UK 2017 Conference; Kings College London, UK. 2017, 5-7 September, 1-44. Accessed in July 2023 here, and archived as accessed here. 20 July.
Mouat T. Introduction to Wargaming Pre-Reading. Connections UK 2017 Conference; Kings College London, UK. 2017, 5-7 September, 1-26. Accessed in July 2023 here, and archived as accessed here. 20 July.
Perla P. Wargaming for the Future. 2008 DoD Modeling and Simulation Conference; Orlando, Florida. 2008, 10-14 March, 1-12. Accessed in July 2023 here, and archived as accessed here. 20 July.
Brown JR, Fehige Y. Thought experiments. In: Zalta EN and Nodelman U (Eds.), The Stanford Encyclopedia of Philosophy. 2020, Winter, un-numbered. ISSN 1095-5054. Accessed in July 2023 here, and archived as accessed here; and Rubel RC. The epistemology of war gaming. Naval War College Review 2006, 59, 108–128. 2006, 59, 108–128.
Wallace, R. Carl von Clausewitz, the Fog-of-War, and the AI Revolution: The Real World is not a Game of Go. Springer Briefs in Applied Sciences and Technology, Computational Intelligence, Springer Verlag; Berlin, DE: 2018. 101pp. ISBN: 978-3-319-74632-6.
Naveed Uddin, M. Cognitive science and artificial intelligence: simulating the human mind and its complexity. Cognitive Computation and Systems 2019, 1, 113–116. [Google Scholar] [CrossRef]
Yang S, Barlow M, Townsend T, Liu X, Samarasinghe D, Lakshika E, Moy G, Lynar T, Turnbull B. Reinforcement learning agents playing ticket to ride: A complex imperfect information board game with delayed rewards. IEEE Access 2023, 11, 60737–60757. [CrossRef]
Whang SE, Roh Y, Song H, Lee JG. Data collection and quality challenges in deep learning: A data-centric AI perspective. International Journal on Very Large Data Bases (VLDB) 2023, 32, 791–813. [CrossRef]
Czichon R. Umělá Inteligence a Autorské Právo [Artificial Intelligence and Copyright Law]. Thesis, Faculty of Law, Charles University; Prague, CZ: 2021. 73pp. Accessed in July 2023 here, and archived as accessed here.
Liu J, Snodgrass S, Khalifa A, Risi S, Yannakakis GN, Togelius J. Deep learning for procedural content generation. Neural Computing and Applications. 2021, 33, 19–37, and Beukman M, Cleghorn CW, James S. 2022. Procedural content generation using neuroevolution and novelty search for diverse video game levels. Proceedings of the Genetic and Evolutionary Computation Conference, July 9–13, Boston, MA, 2022. 17pp. Accessed as archived in July 2023 here.
Viertl R. Statistical Methods for Fuzzy Data. John Wiley & Sons, Inc; Toronto, CA: 2011. 268pp. ISBN: 978-0-470-69945-4, and von Benzon N, O’Sullivan K. Analyzing messy data. Chapter 27 in, von Benzon N, Holton M, Wilkinson C and Wilkinson S (Eds.), Creative Methods for Human Geographers. SAGE Publications Ltd; Newbury Park, CA, USA: 2021. 432pp. ISBN: 9781526496973.
Landgrebe J, Smith B. There is no general AI: Why Turing machines cannot pass the Turing test. arXiv 2019; 1906.05833: 1-44. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.