Multi-Agent Social Simulation: Protocolizing LLM-Driven Agent-Based Modeling as a Quantitative Research Method

1. Introduction

As exemplified by the Stanford Prison Experiment in 1973 (Haney et al., 1973; Zimbardo, 2007), the social and behavioral sciences have long faced a methodological and ethical dilemma. Researchers need to understand how people respond to institutions, roles, incentives, information, and group environments, yet they cannot freely place real participants into manipulated social situations for repeated experimentation. The closer an experiment comes to real social processes, the more likely it is to produce large and unpredictable consequences. Wage policies, digital targeting, platform-based public-opinion governance, collective mobilization, organizational coordination, and public-emotion interventions may all generate economic, psychological, or social consequences in real-world settings. Researchers can observe such consequences after they occur, but it is difficult to actively manipulate, regulate, and repeat them under controlled conditions in real populations.

The social sciences therefore need a simulation sandbox that can be run before real-world interventions. Such a sandbox would provide a low-risk, repeatable, and adjustable environment in which researchers can rehearse possible behavioral pathways, mechanism chains, and counterfactual outcomes before formal policies, communication interventions, or governance practices are implemented. For research on public policy, group communication, and social governance, such pre-experimental environments can supplement real-world evidence by helping researchers identify potential mechanisms, compare alternative conditions, and improve subsequent empirical research designs under lower-risk conditions.

To date, the social sciences have developed multiple alternative and simulation-based methods to address the ethical constraints, costs, and limited repeatability of real-world experiments. Natural experiments, quasi-experimental designs, wargaming, policy games, tabletop exercises, system dynamics, microsimulation, and agent-based modeling (ABM) each provide different ways of handling this problem.

Natural experiments and quasi-experimental designs provide social scientists with a real-world logic for causal identification. Researchers use policy changes, institutional differences, or external shocks that have already occurred to construct treatment and control groups, and then evaluate intervention effects through methods such as difference-in-differences, pre-trend tests, and placebo tests (Angrist & Pischke, 2009; Dunning, 2012; Imbens & Rubin, 2015). For example, Card and Krueger’s (1994, 2000) study of the New Jersey–Pennsylvania minimum wage case used a real policy change to construct treatment and comparison conditions, allowing researchers to compare changes in wages, employment, and prices. The advantage of natural experiments lies in their proximity to real social processes, but their limitation is that researchers cannot actively set the timing, intensity, external disturbances, or repeated implementation of the intervention.

Wargaming, policy games, and tabletop exercises provide another process-oriented simulation pathway. These methods use structured scenarios, role assignment, multi-round actions, strategic interaction, and situational adjudication to explore complex social processes. Policy games, role-playing simulations, serious games, and participatory simulations extend this logic by allowing participants to act as different roles and make continuous decisions within constrained environments (Mayer, 2009). Such methods can represent incomplete information, conflicting goals, and strategic adaptation among multiple actors. However, they are often limited by weaker reproducibility, insufficient quantitative outputs, and a dependence on participant experience or expert adjudication.

Computational social simulation further transforms social processes into executable models. System dynamics is suitable for modeling macro-level feedback loops, microsimulation is suitable for projecting policy effects at the individual level, and ABM simulates macro-level emergence through heterogeneous agents, local interactions, and environmental rules (Bonabeau, 2002; Epstein, 1999; Gilbert & Troitzsch, 2005; Railsback & Grimm, 2019). ABM allows researchers to define agent types, behavioral rules, interaction structures, and environmental states, and to observe system behavior through repeated runs. However, the behavioral rules of agents in ABM are usually specified manually by researchers. When the research object involves linguistic action, role interpretation, institutional meaning, local information, and situational judgment, fixed functions and heuristic rules often struggle to capture the behavioral logic of real actors. Put simply, in traditional ABM, agents cannot use natural language to describe or explain their own behavior in detail.

LLM-driven ABM lies at the intersection of these methodological traditions. It absorbs the treatment-control and counterfactual logic of natural experiments, extends the multi-role, multi-round, and situational interaction logic of wargaming and policy games, preserves the agent modeling, state updating, and repeatable execution of ABM, and uses large language models (LLMs) to enhance linguistic, role-based, and context-sensitive behavioral generation. The introduction of LLMs gives ABM a more dynamic and language-sensitive form of behavioral simulation. Agents can generate actions and reasons based on role cards, local information, external rules, and current-round events, while researchers can transform these actions into structured variables for statistical testing, process auditing, and cross-case comparison.

1.1. The Rise of AI-Driven Social Simulation Platforms and the Methodological Gap

In recent years, several representative AI-driven social simulation systems have emerged. Generative Agents demonstrated the ability of LLM agents to generate memory, reflection, planning, and everyday interaction in a small sandbox society (Park et al., 2023). Agent Hospital embedded agents such as doctors, nurses, and patients into a virtual hospital process to simulate multi-stage medical behavior (Li et al., 2024). OASIS and AgentSociety further extended this line of work toward social media communication and large-scale simulations of social life, exploring information diffusion, group polarization, external shocks, and public policy (Yang et al., 2024; Piao et al., 2025). These studies show that AI-driven social simulation has moved from proof-of-concept demonstrations toward platform-based practice. However, existing platforms often focus on specific scenarios, fixed environments, or system-level capabilities.

Unlike AI social simulation platforms centered on fixed scenarios, this study focuses on a transferable research method: how to organize LLM-driven ABM into a configurable, auditable, and empirically assessable simulation experiment. This study operationalizes that method as Multi-Agent Social Simulation (MASS), whose core is a reusable research protocol. Researchers need to define role clusters, temporal granularity, exogenous rules, information boundaries, output structures, and validation indicators around specific research questions, while maintaining comparability across treatment, control, or counterfactual conditions. In this sense, prompt design, role cards, rule tables, output schemas, logging mechanisms, and validation matrices are no longer merely engineering configurations. They become components of the research design, corresponding respectively to behavioral generation, variable construction, experimental conditions, process auditing, and methodological validity assessment.

Protocolization is the key condition for transforming LLM-driven ABM from narrative role-playing into a verifiable research method. Naive LLM-agent simulations usually provide models with role prompts and ask them to continuously output actions or opinions. Such practices can quickly generate seemingly plausible narrative results, but they are also prone to problems such as blurred temporal boundaries, role drift, omniscient perspectives, rule drift, non-computable outputs, invalid numerical values, and difficulties in replication. To be used in behavioral and social science research, LLM-driven ABM needs to integrate rounds, roles, information, rules, outputs, checks, and validation into an explicit protocol. Only then can the simulation process be recorded, the simulation results audited, and different runs, conditions, and cases compared on an empirical basis.

1.2. Research Questions and Contributions

Based on the methodological positioning above, this study proposes three research questions.

RQ1: Can LLM-driven ABM serve as a method for simulating social behavior in behavioral science?

RQ2: What methodological failures arise in naive LLM-agent simulation, and what forms of protocolized control are needed to mitigate them?

RQ3: After protocolized control is introduced, can LLM-driven ABM generate auditable, empirically assessable simulation outputs with evidence of cross-scenario applicability across different social-behavioral settings?

This study makes four main contributions in relation to these questions.

First, starting from the ethical boundaries of real-world social experimentation, this study proposes the value of LLM-driven ABM as a low-risk pre-experimental method. This method does not replace experiments with real participants or natural experiments. Rather, it provides a simulation space for mechanism rehearsal, risk identification, and counterfactual comparison.

Second, this study diagnoses the typical failure modes of naive LLM-agent simulation and translates blurred temporal boundaries, role drift, omniscient perspectives, rule drift, non-computable outputs, invalid numerical values, and replication difficulties into operational protocol-control items.

Third, this study proposes a protocolized LLM-driven ABM framework that incorporates prompts, role cards, rule tables, information boundaries, output schemas, harness checks, and validation targets into the research design. Through this framework, LLM-driven ABM can generate behavioral simulation outputs that are auditable, reproducible, and empirically comparable.

Fourth, this study conducts a multi-evidence evaluation through three social-behavioral cases: a minimum wage policy shock, a digital-campaigning information intervention, and urban tourism public-opinion dynamics. These three cases jointly test the method’s data analyzability, counterfactual comparability, group-ordering consistency, and process-explanation capacity.

2. Literature Review

To answer the research questions above, this study first situates LLM-driven ABM within the broader methodological landscape of social simulation and behavioral science. The key issue is not only what existing methods have already solved, but also where gaps remain in reproducibility, auditability, behavioral rule specification, and validity assessment. Clarifying these methodological foundations is necessary for explaining why protocolized LLM-driven ABM is needed and what forms of evaluation it should undergo.

This chapter proceeds in three steps. First, it returns to the problem of behavioral rule specification in ABM. It discusses how traditional ABM explains macro-level emergence through agents, environments, rules, and state updating, while also showing why hand-written rules and fixed functions face expressive pressure when the research object involves linguistic interaction, role identity, institutional interpretation, local information, and situational judgment. Second, it discusses replication and process auditing in LLM-driven ABM. LLMs can extend the behavioral generation capacity of ABM, but they also introduce new problems such as prompt sensitivity, role drift, omniscient perspectives, output instability, and model-version dependence. These problems call for protocolized controls such as prompt manifests, replication manifests, reason-action logs, harness checks, and structured outputs. Third, the chapter discusses validity assessment in generative social simulation. For LLM-driven ABM, the central issue is not whether the model can generate text that “looks plausible,” but whether it can produce simulation data that are structurally clear, process-auditable, and empirically comparable.

2.1. ABM, LLM-Driven ABM, and the Problem of Behavioral Rule Specification

ABM consists of agents, environments, state variables, behavioral rules, and interaction mechanisms. It is well suited for studying how local interactions among heterogeneous agents generate macro-level social outcomes. Its advantages lie in controllability, repeatability, counterfactual reasoning, and the ability to observe emergent structures and nonlinear processes (Bonabeau, 2002; Railsback & Grimm, 2019). In the social sciences, ABM has often been used to explain group segregation (Schelling, 1971), norm diffusion, organizational behavior, market change, public-opinion dynamics, and policy outcomes.

A central challenge in traditional ABM is the specification of behavioral rules. Researchers usually need to define in advance how agents perceive the environment, choose actions, update states, and interact with other agents. These rules may be derived from rational choice, bounded rationality, empirical statistics, expert judgment, or existing theory. Simon’s (1955) discussion of bounded rationality suggests that real-world actors often make satisficing decisions under incomplete information, limited cognitive capacity, and complex situations. In many social-behavioral settings, action is also shaped by role identity, institutional interpretation, linguistic framing, local information, social expectations, and situational meaning. When traditional ABM handles such behavior, complex action is often compressed into fixed functions or heuristic rules.

For social-behavioral simulation, the methodological significance of LLM-driven ABM first derives from the corpus-based foundation of large language models. LLMs do not generate actions from abstract rules in a vacuum; rather, they learn conditional relationships among roles, contexts, norms, attitudes, and response patterns from large-scale human text corpora. Existing research has shown that, under appropriate conditioning and clearly specified task constraints, language models can to some extent approximate the response distributions of particular human samples or subgroups. Argyle et al. introduced the concept of “algorithmic fidelity,” arguing that language models conditioned on demographic and social-background information can generate attitude distributions corresponding to multiple human subgroups (Argyle et al., 2023). Aher et al. further proposed “Turing Experiments” to examine the ability of language models to replicate classic human-subject experiments, showing that LLMs can generate responses close to human experimental patterns in several tasks in economics, psycholinguistics, and social psychology (Aher et al., 2023). AI-Turk-related research also suggests that ChatGPT can imitate the psychological and behavioral responses of human participants at relatively high rates in several online crowdsourcing research contexts (Qin et al., 2024). Taken together, these studies suggest that LLMs can be viewed as conditional behavior generators grounded in accumulated public language data, rather than as mere text-completion tools.

This provides a new possibility for specifying behavioral rules in ABM. Traditional ABM requires researchers to predefine how agents perceive environments, choose actions, and update states. This may compress complex role identities, institutional interpretations, local information, and social expectations into fixed functions or heuristics. LLM-driven ABM does not remove the researcher’s control over model structure. Rather, it extends behavioral rules from static hand-written rules to protocolized generative rules. Researchers remain responsible for defining role clusters, exogenous rules, information boundaries, state variables, and validation indicators. Within these constraints, LLMs generate concrete actions based on role cards, local information, and current-round events. In this way, situational judgment, linguistic expression, and role-based response in social behavior can be incorporated into a recordable and auditable simulation process.

However, the approximation of human responses by LLMs is not unconditional. Model outputs may be affected by training corpora, prompts, model versions, alignment mechanisms, and social-desirability bias. They may also produce overly consistent, overconfident, or majority-oriented responses in particular tasks. Therefore, this study does not treat LLM agents as substitutes for real individuals or real samples. Instead, it embeds them into ABM as constrained behavior generators. Role cards determine the social position and behavioral preferences of agents; local information determines their visible environment; rule tables specify institutional constraints; round-based scheduling defines the boundary of state updating; structured outputs transform actions into analyzable variables; and harness checks and validation targets are used to examine role consistency, information boundaries, numerical validity, and result comparability. In this sense, the key to LLM-driven ABM is not to let models freely simulate human beings, but to transform human response patterns embedded in language corpora into reproducible, comparable, and empirically assessable social-simulation data through protocolized control.

2.2. Replication, Process Auditing, and Model Description

The credibility of social simulation models depends on reproducibility and auditability. The ABM field has long emphasized the importance of model description. The ODD protocol requires researchers to describe model purpose, agents, state variables, scales, processes, scheduling, initialization, input data, and submodels in a standardized manner, thereby improving model transparency, replicability, and comparability (Grimm et al., 2006, 2020). For LLM-driven ABM, model description must also cover prompts, role cards, information boundaries, rule tables, output schemas, model versions, randomness settings, retry policies, and validation targets.

In LLM-driven ABM, replication is more challenging than in traditional ABM. Technically, Transformer-based language models generate subsequent tokens through conditional probabilities. Vaswani et al. noted in the Transformer paper that decoder outputs are converted by softmax into predicted next-token probabilities (Vaswani et al., 2017). In actual generation processes, outputs are also affected by decoding strategies, temperature, top_p, random seeds, context length, and server-side backend configurations. Holtzman et al.’s study of neural text degeneration shows that, even with exactly the same neural language model, different decoding strategies can significantly affect generated outputs (Holtzman et al., 2020). The reproducible-output example in the OpenAI Cookbook also notes that the seed parameter can improve output consistency, but that its effect depends on identical request parameters and system_fingerprint and cannot guarantee absolute determinism (Anadkat, 2023). Therefore, a single run of LLM-driven ABM should not be treated as a stable conclusion. Researchers need to reduce the influence of single-run stochastic output through multiple independent runs, directional consistency checks, and cross-condition comparisons.

To standardize replication, researchers need to build both a prompt manifest and a replication manifest. The prompt manifest records role prompts, background prompts, rule prompts, output prompts, forbidden information, and visible information boundaries. The replication manifest records model names, running time, parameter settings, data versions, case configurations, round granularity, randomness controls, failure-retry mechanisms, and backend-configuration identifiers. Without these records, it is difficult for third parties to determine whether simulation results arise from social mechanisms, prompt induction, model priors, backend changes, or random output variation.

Process auditing is another key requirement for LLM-driven ABM. When LLMs generate behavior, researchers need not only structured variables but also auxiliary materials that can trace the behavior-generation process. A reason-action log can record how an agent understands local information in each round, how it chooses actions, and how those actions are translated into state changes. Its function is to serve as process-auditing material, recording behavioral generation pathways and checking output consistency. Researchers can use reason-action logs to examine whether actions conform to role boundaries, whether they reference forbidden information, whether hindsight contamination occurs, and whether they are consistent with the direction of numerical outputs. Structured fields provide the primary objects of analysis, while explanatory text provides auxiliary auditing material.

Output structure and runtime checking mechanisms determine whether simulation data can enter statistical analysis. Naive LLM-agent simulations often generate long natural-language outputs, making it difficult for researchers to compare different runs, conditions, and cases. LLM-driven ABM for behavioral science research requires structured outputs, value ranges, state-update boundaries, and format checks so that simulation results can be statistically analyzed, visually compared, and checked for anomalies. Reproducibility, process auditing, and output structure therefore jointly determine whether LLM-driven ABM can move from narrative simulation to a verifiable research method. In Chapter 3, this study further operationalizes these principles as prompt manifests, replication manifests, harness checks, local checks, structured outputs, and full-log preservation.

2.3. Validity, Reliability, and the Evaluation Gap in Generative Social Simulation

The credibility of simulation models cannot rely solely on whether their results “look reasonable.” For social simulation, reliability and validity involve at least two levels. Reliability concerns whether a model can generate stable, reviewable, and repeatedly analyzable outputs under identical or similar conditions. Validity concerns whether model structure, agent behavior, process mechanisms, and outcome directions form an interpretable correspondence with the target social process. In other words, a social simulation model should not only generate an outcome; it should also explain how that outcome is produced by agents, rules, environments, and interaction processes, and whether that generative process can be recorded, reviewed, and compared.

Existing simulation research provides important theoretical foundations for this issue. Guala (2002) argues that complex relationships exist among models, simulations, and experiments, and that simulations explore mechanisms and possibilities through artificially constructed systems. Morgan (2003) discusses forms of experimentation without material intervention, showing that model experiments and virtual experiments can also play a role in scientific reasoning. Sugden (2000) proposes the concept of “credible worlds,” emphasizing that the value of a model lies not in replicating every detail of the real world but in constructing an internally coherent artificial world that can carry theoretical mechanisms and support reasoning. For ABM, this means that model validity should not be judged only by endpoint outcomes. It also requires examining whether agent settings, behavioral rules, interaction structures, state updates, and macro-level outputs are internally consistent.

This issue becomes even more prominent in LLM-driven ABM. Recent reviews have pointed out that large language models strengthen ABM in environmental perception, heterogeneous agent representation, reasoning and decision-making, and action generation, while also introducing new challenges related to human alignment, action reliability, evaluation standards, and cross-scenario validation (Gao et al., 2024). Larooij and Törnberg (2026) further argue that validation is the central challenge for generative social simulation: many studies still rely mainly on subjective judgments of whether simulations “appear credible,” while providing limited evidence that models are operationally valid. For this study, this means that LLM-driven ABM should not merely demonstrate narrative plausibility or role-playing capacity. It must provide reviewable structural evidence, process evidence, and outcome evidence.

Accordingly, this study designs validity and reliability assessment as a multi-evidence framework rather than relying on a single indicator. At the structural level, the model must specify whether agent types, state variables, temporal granularity, rule conditions, and information boundaries reasonably represent the target social process. At the behavioral level, the model must examine whether agent actions conform to role positions, local information, and institutional constraints. At the process level, the model must preserve reason-action logs, public messages, state changes, and harness results so that behavioral generation pathways can be audited. At the outcome level, the model must form comparable relationships with real-world references, natural-experiment findings, counterfactual conditions, known group orderings, or mechanism chains.

Based on this logic, Chapter 3 uses three complementary cases to examine validity and reliability. Case 1 uses the New Jersey–Pennsylvania minimum wage natural experiment, with treatment, comparison, and counterfactual groups, and uses DID and DDD to test whether the minimum wage rule shock can stably enter managerial decision-making. Case 2 compares targeting and no-targeting conditions to examine whether a digital-targeting mechanism can generate distinguishable changes in group attitudes, and uses known group ordering to check whether role settings remain consistent. Case 3 compares active governance and passive governance conditions to examine whether platform attention, offline experience, and governance feedback form a directionally consistent mechanism chain. Together, the three cases cover rule-shock identification, information-intervention differentiation, and multi-agent interaction with process consistency in a public information environment, thereby providing multiple types of evidence for the reliability and validity of MASS.

This leads to the literature gap and research positioning of the present study. Natural experiments provide treatment-control logic and causal identification in the real world, but researchers cannot actively repeat or adjust intervention conditions. Wargaming, policy games, and role-playing simulations provide multi-role, multi-round interaction, but their reproducibility and quantitative outputs are limited. Traditional ABM provides a repeatable computational framework, but its behavioral rules are often static. LLM agents provide linguistic and role-based behavioral generation capacity, but are prone to prompt sensitivity, role drift, omniscient perspectives, and difficulties in replication. The protocolized LLM-driven ABM proposed in this study builds a bridge among these methodological capacities. It preserves the repeatability and counterfactual capacity of ABM, introduces the contextual behavioral generation capacity of LLMs, and uses structured outputs, process auditing, and a multi-evidence validation framework to examine whether it can serve as an auditable, reproducible, and empirically assessable method for social simulation research.

3. Materials and Methods

3.1. Naive LLM-Agent Baseline and Failure Modes

This study first constructs a naive LLM-agent baseline as a methodological reference to illustrate the problems that may arise when large language models are directly connected to agent simulation. The basic procedure of this baseline is as follows: the researcher writes a role prompt for each agent; the system provides the current situation to the model in each round; the model outputs an action or opinion; and the researcher then feeds the output into the next round of state updating. This procedure can rapidly generate narrative simulation results, but it lacks strict temporal boundaries, role boundaries, information boundaries, and output constraints. It is therefore difficult to use directly as a reviewable method for behavioral science research.

The main problem with the naive baseline is that its generated behavior lacks stable experimental discipline. An agent may predict the future within the same round, move beyond its role identity into a narrator, researcher, or policy commentator perspective, or refer to researcher-side information or real-world final outcomes. Natural-language outputs may also lack computable fields, making it difficult to compare different runs, conditions, and cases. This study therefore treats naive LLM-agent simulation as an initial form that requires protocolization and identifies eight typical failure modes.

Table 1. Failure modes of naive LLM-agent simulation and protocolized controls.

Failure mode	Typical manifestation	Impact on research	Protocolized control
Blurred temporal boundaries	Predicting future events or repeatedly updating the same state	Unstable behavioral trajectories	Fixed round-based scheduling
Role drift	Shifting from role-based action to narration, researcher commentary, or policy analysis	Distortion of the behavioral actor	Role cards and boundary checks
Omniscient perspective	Referring to final outcomes or real-world conclusions	Outcome contamination	Local information boundaries and case blinding
Rule drift	Changing policies, prices, or action ranges autonomously	Loss of experimental control	Exogenous rule tables
Non-computable output	Long text without stable fields	Difficulty in statistical analysis	Structured output schema
Invalid numerical values	Out-of-range values or logical contradictions	Distorted state updating	Local numerical checks
Black-box action	Producing an outcome without an auditable reason	Difficulty in process review	Reason-action log
Replication difficulty	Missing prompts, models, or configurations	Difficulty in third-party review	Prompt and replication manifests

Table 1. Failure modes of naive LLM-agent simulation and protocolized controls.

Failure mode	Typical manifestation	Impact on research	Protocolized control
Blurred temporal boundaries	Predicting future events or repeatedly updating the same state	Unstable behavioral trajectories	Fixed round-based scheduling
Role drift	Shifting from role-based action to narration, researcher commentary, or policy analysis	Distortion of the behavioral actor	Role cards and boundary checks
Omniscient perspective	Referring to final outcomes or real-world conclusions	Outcome contamination	Local information boundaries and case blinding
Rule drift	Changing policies, prices, or action ranges autonomously	Loss of experimental control	Exogenous rule tables
Non-computable output	Long text without stable fields	Difficulty in statistical analysis	Structured output schema
Invalid numerical values	Out-of-range values or logical contradictions	Distorted state updating	Local numerical checks
Black-box action	Producing an outcome without an auditable reason	Difficulty in process review	Reason-action log
Replication difficulty	Missing prompts, models, or configurations	Difficulty in third-party review	Prompt and replication manifests

These failure modes provide the basis for the protocol design of this study. The purpose of protocolization is to embed the open-ended generative capacity of LLMs into a controllable ABM structure, so that agent actions can retain linguistic, role-based, and context-sensitive features while still satisfying the requirements of state updating, output recording, and empirical validation.

3.2. Protocolized Control Scheme and Operational Implementation

This study operationalizes protocolized LLM-driven ABM as a simulation workflow consisting of scenario configuration, role configuration, round scheduling, model invocation, structured output, runtime validation, log preservation, and statistical export. The workflow is not a single prompt call. Instead, it embeds LLM agents into the agent, rule, interaction, and state-updating framework of ABM, so that each output can be recorded, checked, and transformed into analyzable data.

The protocolized control scheme consists of seven main components: role-cluster modeling, local information control, exogenous rule tables, structured outputs, harness and local checks, reason-action logs, and replication materials. Role-cluster modeling defines the social position and behavioral boundaries of agents. Local information control reduces the risk of final-outcome leakage and researcher-side information contamination. Exogenous rule tables inject policy changes, platform rules, governance conditions, and event shocks. Structured outputs transform textual actions into analyzable fields. Harness and local checks examine format validity, field completeness, numerical ranges, and abnormal state updates. Reason-action logs support process auditing. Prompt manifests and replication manifests preserve prompts, model settings, runtime configurations, failure records, and version information.

To execute this protocol, this study implements it as the Multi-Agent Social Simulation (MASS) research system, which supports configuration reading, round scheduling, LLM invocation, output validation, log preservation, and data export. In this study, MASS is used to ensure that the protocol can run stably and generate analyzable data. It is not evaluated as an independent prediction platform.

3.3. Three Behavioral Simulation Cases

This study selects three differentiated social-behavioral scenarios for methodological evaluation. The three cases correspond respectively to a policy shock, an information intervention, and public-opinion dynamics under platform attention and governance feedback. Case selection follows two principles. First, each case has an empirical reference that allows comparison with existing empirical data or real-world processes. Second, the cases involve different mechanisms, enabling the study to test the applicability of protocolized LLM-driven ABM across different social-behavioral settings.

Table 2. Design of the three behavioral simulation cases.

Case	Behavioral scenario	Social behavior	Methodological reference	Main mechanism	Key validation target
1	Managerial decision-making under rule constraints	Economic policy	Natural experiment / quasi-experiment	Minimum wage shock affects wage, employment, and pricing decisions	Policy-shock identification
2	Group attitudes under differentiated information exposure	Political attitudes	Scenario simulation / counterfactual information intervention	Digital targeting affects issue salience and attitude change	Information intervention and political-attitude change
3	Collective behavior under platform attention and governance feedback	Public-opinion event	Open public-opinion simulation / mechanism-chain test	Platform amplification, offline experience, and governance response jointly affect sentiment and behavioral intention	Multi-agent interaction and mechanism-chain consistency

Table 2. Design of the three behavioral simulation cases.

Case	Behavioral scenario	Social behavior	Methodological reference	Main mechanism	Key validation target
1	Managerial decision-making under rule constraints	Economic policy	Natural experiment / quasi-experiment	Minimum wage shock affects wage, employment, and pricing decisions	Policy-shock identification
2	Group attitudes under differentiated information exposure	Political attitudes	Scenario simulation / counterfactual information intervention	Digital targeting affects issue salience and attitude change	Information intervention and political-attitude change
3	Collective behavior under platform attention and governance feedback	Public-opinion event	Open public-opinion simulation / mechanism-chain test	Platform amplification, offline experience, and governance response jointly affect sentiment and behavioral intention	Multi-agent interaction and mechanism-chain consistency

The first case is based on the New Jersey–Pennsylvania minimum wage natural experiment (Card & Krueger, 1994, 2000). The simulated actors are fast-food store managers. Each agent represents a cluster of store managers and makes weekly decisions under minimum wage rules, profit pressure, labor constraints, and price-adjustment considerations. The main analysis variables include starting_wage, target_fte, and price_full_meal_after_tax. This case is used to examine whether the model can generate comparable policy-shock responses across treatment, comparison, and counterfactual groups.

The second case uses the Cambridge Analytica controversy and the 2016 UK Brexit digital campaigning context as empirical references (House of Commons Digital, Culture, Media and Sport Committee, 2019; Information Commissioner’s Office, 2018). The case operationalizes data analytics and targeted advertising as simulation conditions and compares voter-cluster attitudes under targeting and no-targeting control conditions. The main analysis variables include leave_support, remain_support, undecided_share, turnout_intent, trust_in_experts, and issue_salience. This case is used to examine whether the model can generate distinguishable group-attitude changes under the presence or absence of a digital-targeting mechanism.

The third case uses the 2023 Zibo barbecue tourism public-opinion event as an empirical reference. It is designed to simulate the diffusion, offline service reception, and governance feedback of a positive public-opinion event in a social-platform environment. This case uses the search trend for the keyword “Zibo barbecue” in Baidu Index as an external attention reference to help determine whether the real-world attention cycle is broadly aligned with the simulated rounds. Baidu Index is not included in the DID, DDD, or mechanism-correlation models and is not used as real-world causal validation data. Groups A–F represent the active governance condition, while Group G represents the passive governance counterfactual condition. Variables are divided into organizational-response variables and public-response variables. This case is used to examine whether platform attention, offline experience, governance response, and public-information interaction can form a directionally consistent public-opinion diffusion and behavioral-intention chain.

Together, the three cases cover organizational decision-making under institutional rule change, group attitudes under information-environment change, and collective behavior under platform communication and governance feedback. Through these cases, this study compares the data completeness, counterfactual differentiation capacity, mechanism consistency, and cross-scenario applicability of protocolized LLM-driven ABM in different social processes.

3.4. Validation Strategy

This study adopts a multi-evidence validation strategy. DID is an important test, but this study does not limit methodological validity to a single statistical indicator. For generative simulation methods such as LLM-driven ABM, validation needs to consider data completeness, behavioral boundaries, counterfactual differences, directional consistency, and mechanism transmission at the same time.

This study specifies six types of operational evidence. First, data completeness and protocol compliance are used to check whether runs are complete, fields are available, numerical values are within valid ranges, and outputs conform to required formats. Second, counterfactual contrast tests compare treatment, control, and counterfactual groups. Third, placebo and pre-trend tests examine whether false differences appear before the intervention. Fourth, repeated runs and directional consistency assess whether multiple runs under the same configuration remain stable in direction and trend. Fifth, known group ordering tests whether the model can generate group differences consistent with role expectations. Sixth, mechanism-chain tests examine whether organizational variables, platform variables, and public-response variables form theoretically interpretable directional relationships.

Table 3. Validation strategy and applicable cases.

Validation evidence	Main function	Main applicable cases
Data completeness and protocol compliance	Checks whether outputs are analyzable	All three cases
Counterfactual contrast	Compares treatment and control conditions	Case 1, Case 2, Case 3
Placebo and pre-trend tests	Rules out false pre-intervention differences	Case 1, Case 2
Repeated runs and directional consistency	Checks result stability	All three cases
Known group ordering	Checks whether role differences are reasonable	Case 2
Mechanism-chain tests	Checks whether variable transmission is consistent	Case 3

Table 3. Validation strategy and applicable cases.

Validation evidence	Main function	Main applicable cases
Data completeness and protocol compliance	Checks whether outputs are analyzable	All three cases
Counterfactual contrast	Compares treatment and control conditions	Case 1, Case 2, Case 3
Placebo and pre-trend tests	Rules out false pre-intervention differences	Case 1, Case 2
Repeated runs and directional consistency	Checks result stability	All three cases
Known group ordering	Checks whether role differences are reasonable	Case 2
Mechanism-chain tests	Checks whether variable transmission is consistent	Case 3

In the specific analyses, Case 1 focuses on policy-shock identification and examines wage, employment, and price variables across treatment, comparison, and counterfactual groups. Case 2 focuses on information-intervention differentiation and examines undecided shares, support tendencies, issue salience, and group ordering. Case 3 focuses on multi-agent interaction and mechanism-chain consistency, examining organizational response, platform diffusion, public sentiment, visit intention, and trust in official response.

3.5. Implementation, Data Processing, and Reproducibility Materials

The data analyzed in this study were not all generated by a single submission-frozen version of MASS. Rather, they were produced during the continuous development and protocolized refinement of the MASS platform. To avoid misinterpreting platform-version differences as differences in simulation results, this section explicitly distinguishes the data sources, model names, output formats, and analytical purposes of different cases.

Groups A–D in Case 1 were generated using an early version of MASS and are used primarily as historical benchmark data for the NJ–PA minimum wage case. These four groups belong to the same historical benchmark dataset used in the authors’ earlier MASS methodological exploration paper presented at Agents4Science 2025 (Hu, Shen, & You, 2025). In the present study, these data are not presented as newly generated simulation data. Instead, they are incorporated into a unified cross-case methodological validation framework as historical benchmark data from the earlier workshop-submission version, and are reprocessed through unified data cleaning, DID/DDD calculation, and result interpretation. Groups A, B, and D are policy-shock groups, while Group C is a counterfactual group without the NJ minimum wage increase. Together, these four groups constitute the main DID/DDD analysis data for Case 1. Group E in Case 1 was generated after a major harness update in MASS and uses DeepSeek-R1-0528. Its purpose is to check whether the later protocolized version can generate parseable and auditable data in the same benchmark case, rather than to provide a fully matched effect replication of Groups A–D. Therefore, Group E is reported only as a supplementary harness-check run and is not included in the main DID/DDD estimates for Case 1. Compared with the earlier conference paper, the present study further adds the Case 1 Group E harness-check run, the Case 2 UK Brexit digital-campaigning scenario, and the Case 3 Zibo barbecue tourism public-opinion scenario, and evaluates protocolized LLM-driven ABM in terms of analyzability, auditability, counterfactual comparability, and cross-scenario applicability.

From Case 1 Group E onward, subsequent runs use DeepSeek-R1-0528, mainly for cost and scalability reasons. Case 2 and Case 3 both belong to the post-harness protocolized stage after the major harness update, but they do not use exactly the same minor version. The system continued to undergo debugging, scenario tuning, prompt/schema refinement, local-check adjustment, and output-stability optimization. However, these adjustments did not alter the main architecture of MASS. The later versions all retained the core workflow of JSON scenario configuration, role configuration, round scheduling, OpenAI-compatible API invocation, structured output, harness validation, local checks, log preservation, and data export.

Each run uses a scenario configuration file as its entry point. The configuration includes caseId, conditionId, round settings, agent lists, event tables, output schemas, state variables, local validation rules, and validation targets. In Case 2, the targeting condition and the no-targeting control condition are compared within the same case framework. In Case 3, the active governance condition and the passive governance counterfactual condition are compared within the same case framework. This study does not combine different cases into a single unified treatment-effect model. Instead, each case is analyzed according to its own validation target. Case 1 is used to examine rule-shock identification, Case 2 to examine information-intervention differentiation, and Case 3 to examine multi-agent interaction and mechanism-chain consistency in a public-information environment.

LLM calls are made through an OpenAI-compatible API and do not rely on vendor-side conversational memory. In each round, each agent’s messages are reassembled and submitted to the model as a new request. Agent history enters the next round only through structured summaries or restricted fields, rather than through long-term model-side memory. Hidden fields, final outcomes, researcher-side explanations, and validation targets are not included in the agent-visible decision context. Model names, run dates, temperature settings, context settings, retry rules, failure records, and scenario-configuration versions are recorded in the replication manifest. When a model call fails, the output is invalid, required fields are missing, or numerical values are out of range, the system either triggers a retry according to local validation rules or retains the record in the full log. Records that do not successfully enter the formal analysis table are not manually rewritten or imputed.

Data processing follows the principle of separating the formal analysis table from full logs. The formal analysis table retains only final successful and parseable agent-round observations for statistical analysis. The full logs retain prompts, model responses, failure reasons, harness results, and structured fields for process auditing. Groups A–D in Case 1 constitute the formal main analysis data, with 3680 agent-round records. Group E contains 918 parseable records and is used only as a supplementary harness-check run. Case 2 contains 1548 structured records, and Case 3 contains 2016 structured records. All formal result tables are generated from exported CSV/XLSX files in Google Colab.

Statistical analysis is conducted in Python 3.x in Google Colab, mainly using pandas, statsmodels, and openpyxl. Case 1 uses DID and DDD to test policy shocks. The core DID estimator is the NJ × Post interaction term, and the core DDD estimator is the NJ × Post × Policy interaction term, with standard errors clustered at the agent level. Case 2 uses pre-post differences, counterfactual condition comparisons, DID, and known group-ordering tests. Case 3 compares active governance and passive governance conditions and uses Pearson and Spearman correlations to examine mechanism-chain consistency. All confidence intervals and p values are generated from the Colab statistical scripts.

Large language models are used in this study as constrained behavior generators inside the MASS simulation environment, not as paper authors or independent researchers. Their role is limited to generating agent actions, public messages, and structured state updates under the constraints of role cards, local information boundaries, exogenous rule tables, output schemas, and validation rules. All simulation outputs are exported, checked, and analyzed by the researchers using statistical scripts.

During the research and writing process, the authors also used generative AI tools to assist with language polishing, English translation, structural checking, code debugging, expression refinement, and discussion of presentation clarity. The project requirements, functional planning, case design, running rules, validation logic, and statistical analysis plan of the MASS system were proposed and approved by the first author. Some code writing, code refactoring, error diagnosis, and debugging processes were completed with the assistance of generative AI tools and coding agents. The first author was responsible for specifying the functional requirements of AI-generated or AI-assisted code, testing its operation, verifying results, exporting data, conducting statistical analysis, and taking responsibility for the final code use, simulation design, data processing, and result interpretation. Generative AI tools were not listed as authors and do not bear research responsibility, authorship responsibility, or academic responsibility.

Reproducibility materials are disclosed through a two-level strategy. The Materials and Methods section reports only the information necessary to understand the simulation and statistical-analysis workflow, including version stages, model names, output formats, analytical units, validation mechanisms, software environments, and statistical models. Detailed scenario files, prompt manifests, replication manifests, exported data, statistical scripts, and audit summaries without sensitive information will be provided as Supplementary Materials or through an external repository. API keys, account information, platform access credentials, and logs containing non-public runtime identifiers or security-sensitive information are not released.

4. Results

4.1. Data Completeness and Protocol Compliance

This chapter analyzes the formal exported datasets from the three cases. Unless otherwise specified, all statistical results in this chapter are based on MASS simulation exports. External real-world data are used only as case references, event-stage anchors, and validation background. They are not merged with simulation records into the same statistical model. All statistical tables were generated from exported CSV/XLSX files in Google Colab. AI-assisted analysis was used only for exploratory checking, code debugging, and interpretation support, and was not used as the source of formal statistical results.

Case 1 includes four formal main-analysis runs and one supplementary verification run. Groups A, B, C, and D were all generated using an early version of MASS. Each group includes 20 store-manager agents and 46 rounds, yielding 920 agent-round records per group. Groups A, B, and D are policy-shock groups, while Group C is the counterfactual group. Therefore, the formal main-analysis dataset for Case 1 contains 3680 records. Group E is a later protocolized verification run using DeepSeek-R1-0528 and the harness-enabled MASS version in the same NJ–PA scenario. It contains 918 parseable records. Group E is used only to show that the later harness version can generate analyzable and auditable data, and is not included in the main DID/DDD estimates for Case 1.

Case 2 includes eight runs and 18 rounds, producing 1548 structured records. Groups A–F are targeting conditions that include data analytics and targeted communication mechanisms. Groups G and H are no-targeting control conditions used to compare attitude evolution in the absence of a targeted-delivery mechanism. Case 3 includes seven runs and 12 rounds, producing 2016 structured records. Groups A–F are active governance conditions, while Group G is the passive governance counterfactual condition.

In the statistical analysis, this study uses only parseable records that enter the formal analysis table. The “formal analysis records” in Table 4 refer to agent-round observations used in formal statistical analysis, not to the full model-call logs. Failed calls, timeouts, invalid outputs, and missing records are retained in full logs or audit records. They are not manually repaired or imputed and do not enter the formal statistical analysis. Group E in Case 1 contains 918 parseable records, but its purpose is to verify the later harness version rather than to provide a main-analysis condition fully matched to Groups A–D. It is therefore excluded from the main DID/DDD estimates for Case 1. Table 4 thus reflects the analyzability of the formal analysis datasets, not all runtime records.

Table 4 shows that all three cases produced exported datasets that could enter statistical analysis. Although Groups A–D in Case 1 used an early backfilled capsule format, they could still be transformed into formal analysis tables through key-value extraction. Group E, Case 2, and Case 3 reflect the later MASS version’s improvements in structured export, harness validation, contamination checks, error flags, and process auditing. Overall, protocolized control transformed LLM-agent outputs from narrative text into structured records suitable for statistical analysis and process auditing.

4.2. Case 1: Managerial Decision-Making Under Rule Constraints

Case 1 uses the NJ–PA minimum wage natural experiment as its empirical reference and examines managerial decision-making under a minimum wage rule shock. Groups A, B, and D are policy-shock groups. After Round 9, NJ agents are subject to the minimum wage increase, while PA agents remain under the original wage rule. Group C is the counterfactual group, in which the NJ minimum wage increase is not injected. This study defines Rounds 1–8 as the pre-intervention period and Rounds 9–46 as the post-intervention period, and compares changes between NJ and PA before and after the intervention. Group E is reported only as a supplementary verification run for the later protocolized version and is not included in the main DDD estimates in Table 5.

The wage variable provides the clearest validation result in Case 1. The starting_wage DID estimates for Groups A, B, and D are +0.734, +0.641, and +0.638, respectively, with a mean of +0.671. The counterfactual DID for Group C is −0.079. The DDD result shows that the additional DID for the A/B/D policy-shock groups relative to the C counterfactual group is +0.753, with a 95% CI of [0.649, 0.857] and p < 0.001. In other words, under the policy-shock condition, NJ starting wages rose stably relative to PA, whereas the same pattern did not emerge automatically in the no-shock counterfactual group. This indicates that MASS can identify the exogenous rule change of the minimum wage and translate it into managerial starting-wage decisions.

The employment variable is more complex. The mean target_fte DID for Groups A, B, and D is +3.146, while Group C has a DID of −28.979. The DDD estimate for A/B/D relative to C is +32.296, but the 95% CI is [−10.938, 75.530] and p = 0.143, which does not reach conventional significance. This result should not be interpreted as a stable replication of the employment effect of the minimum wage shock, nor does it support treating MASS output as a precise quantitative reproduction of real employment effects. The price variable shows mild pass-through. The mean DID for full meal price in Groups A, B, and D is +0.077, while Group C is −0.261. The DDD estimate is +0.337, with a 95% CI of [0.030, 0.644] and p = 0.031. Overall, the wage variable is the most stable, the price variable provides partial support, and the employment variable requires cautious interpretation.

The results of Case 1 support two conclusions. First, MASS is sensitive to hard institutional rules such as the minimum wage and can translate the rule change into NJ managers’ starting-wage adjustments. Second, the counterfactual Group C did not automatically generate the same wage-increase pattern, reducing the possibility that the model simply reproduced historical conclusions from case priors. The differences in employment and price variables also show that MASS is better understood as a pre-experimental and mechanism-rehearsal tool, rather than as a precise prediction model for real economic effects.

4.3. Case 2: Group Attitudes Under Differentiated Information Exposure

Case 2 uses the Cambridge Analytica controversy and the 2016 UK Brexit referendum digital-campaigning context as empirical references to examine whether a targeting mechanism can generate distinguishable group-attitude changes under simulation conditions. This case does not claim that Cambridge Analytica, AggregateIQ, or digital targeting determined the referendum result, nor does it use Cambridge Analytica internal data or real-world polling microdata for causal estimation. Real-world materials are used mainly to construct the digital-campaigning context, define data analytics and targeted-delivery mechanisms, set the campaign time window, and provide external reference points for voter-cluster ordering. Table 6 provides a descriptive comparison between the official referendum result and the MASS post-period means. The DID results in Table 7 are based on MASS simulation outputs, not on a re-estimation of real referendum survey data.

Groups A–F are targeting conditions that include data analytics and targeted-communication agents. Groups G and H are no-targeting control conditions used to compare attitude evolution without a precision-delivery mechanism. This study defines Round 10 as the beginning of the post-period after the strengthening of targeting and compares Rounds 1–9 with Rounds 10–18.

Table 6 shows that the Leave/Remain proportions among decided attitudes in the MASS post-period are descriptively close to the official referendum result. This result can serve as auxiliary evidence of external proportional alignment, but it should not be interpreted as showing that MASS predicted the real referendum result. Nor should it be interpreted as evidence that Cambridge Analytica, AggregateIQ, or digital targeting had a real causal effect on the referendum result. The main methodological evidence in Case 2 still comes from the subsequent comparison between targeting and no-targeting conditions.

Building on the descriptive external alignment shown in Table 6, the condition comparison further indicates that the most stable MASS result in Case 2 is the significant decline in undecided_share under the targeting condition. In the targeting condition, undecided_share decreases from 0.286 to 0.218. In the control condition, it decreases from 0.292 to 0.268. The DID estimate is −0.044, with a 95% CI of [−0.082, −0.005] and p = 0.028. This indicates that, in the simulation environment, the targeting mechanism more clearly reduces the undecided voter share than the no-targeting condition.

The DID estimate for leave_support is +0.022, which is in the expected direction, but its 95% CI is [−0.005, 0.049] and p = 0.105, which does not reach conventional significance. The DID estimate for remain_support is close to zero, with p = 0.967. The issue-salience variables show interpretable directions but are not statistically significant: the DID for issue_salience_immigration is +0.014, p = 0.664, and the DID for issue_salience_economy is −0.032, p = 0.253. turnout_intent and trust_in_experts also do not show significant DID estimates. Therefore, the validity evidence in Case 2 should not be described as a significant increase in Leave support. It should instead be described as “a significant decline in undecided share plus stable group ordering.”

Known group ordering further supports the behavioral validity of the model. Across all comparable run-round-cluster combinations, Leave-leaning or swing voter groups consistently show higher leave_support than young_urban_remain_voters. This indicates that the model does not merely generate aggregate average changes, but also preserves role differences across voter clusters. By Round 18, older_small_town_leave_leaning_voters have a mean leave_support of 0.738 under the targeting condition, while young_urban_remain_voters have a mean of 0.088 and low-participation undecided voters have a mean of 0.288. This ordering is consistent with role settings and also indicates that local information and role cards constrain attitude simulation.

The results of Case 2 show that protocolized LLM-driven ABM can generate distinguishable group-attitude evolution between the presence and absence of a digital-targeting mechanism. The strongest evidence in this case is not the magnitude of a single support-rate change, but the evidence chain formed by the significant decline in undecided_share and the stability of group ordering. The directions of issue salience and leave_support can be used as auxiliary interpretation, but should not be described as significant effects.

4.4. Case 3: Collective Behavior Under Platform Attention and Governance Feedback

Case 3 uses the 2023 Zibo barbecue event on Chinese social media platforms as its empirical reference and focuses on whether MASS can simulate platform-mediated public-opinion development under multi-agent interaction. The real-world background is documented by Fang (2023), Jiang et al. (2023), and Zhang et al. (2025). This event was not the result of a single actor’s action. Rather, it involved multiple types of actors, including city governance departments, cultural-tourism institutions, merchants, platform recommendation systems, media accounts, content creators, local residents, non-local tourists, and ordinary observers. Public-opinion development depended on the continuous writing, reading, amplification, response, and re-circulation of public visible information. The public message of one type of agent became part of the visible information environment for other agents in the next round, thereby creating an interaction-feedback process similar to social networks.

Therefore, the methodological focus of Case 3 is not only to compare variable differences between active governance and passive governance conditions. It is also to observe how different types of agents form continuous interaction through the public board in a public-information environment. Platform and diffusion agents regulate content visibility. Creator and media agents amplify or interpret the event. Merchant and governance agents respond to service pressure. Public agents update sentiment, trust, and visit intention according to visible information, on-site experience, and others’ feedback. Through this design, MASS operationalizes an open public-opinion event as a multi-agent, multi-round, public-information-driven social simulation process.

This study uses the national PC+mobile Baidu Index search trend for the keyword “Zibo barbecue” as an external attention reference. The time range is March 1 to May 24, 2023, corresponding to the 12 weekly simulation rounds in Case 3 (Baidu Index, n.d.). In this study, Baidu Index serves as an external event-stage anchor. It is used to confirm that the simulation window covers the main attention cycle of the real event, including low baseline attention, early warming, rapid diffusion, the peak around the May Day holiday, and post-peak decline. This study does not directly numerically fit Baidu search index values to MASS governance variables, sentiment variables, or behavioral-intention variables, because they correspond to different theoretical constructs. Baidu Index reflects search attention, whereas MASS output variables reflect organizational response, public sentiment, behavioral intention, and public-information interaction in the simulated environment. Therefore, the external validity of Case 3 is mainly based on event-stage alignment, internal validity is mainly based on the counterfactual comparison between active governance and passive governance, and process validity is mainly based on directional consistency among public information, governance response, public sentiment, and behavioral intention under multi-agent interaction.

Groups A–F are active governance conditions, while Group G is the passive governance counterfactual condition. Based on the external stage alignment in Table 8, Rounds 1–3 are treated as the low-baseline and early-diffusion stage, while Rounds 4–12 are treated as the later stage from diffusion onset to governance response and continued circulation. The analysis focuses on organizational-response variables, public-sentiment variables, visit-intention variables, and their directional relationships.

The Colab analysis shows that Case 3 provides the strongest mechanism-chain evidence among the three cases. Compared with the passive governance condition, the active governance condition significantly increases resource_level, channel_effectiveness, positive_sentiment, visit_intent, recommend_intent, and trust_in_official_response, while significantly reducing compliance_risk and negative_sentiment. The DID estimate for message_coherence is +0.025, which is positive in direction, but its 95% CI is [−0.014, 0.065] and p = 0.208. It therefore does not reach statistical significance and should not be treated as strong evidence.

Among organizational variables, the DID estimate for resource_level is +0.044, p = 0.001; the DID estimate for channel_effectiveness is +0.071, p < 0.001; and the DID estimate for compliance_risk is −0.041, p = 0.008. Among public-response variables, the DID estimate for positive_sentiment is +0.115, p = 0.006; the DID estimate for negative_sentiment is −0.100, p = 0.001; the DID estimate for visit_intent is +0.127, p < 0.001; the DID estimate for recommend_intent is +0.127, p < 0.001; and the DID estimate for trust_in_official_response is +0.166, p < 0.001. By Round 12, the active condition is 0.262 higher than the passive condition in trust_in_official_response, 0.188 higher in recommend_intent, 0.170 higher in visit_intent, and 0.174 lower in negative_sentiment.

Table 9. DID results for active governance and passive governance conditions in Case 3.

Variable	Active early	Active later	Passive early	Passive later	DID	95% CI	p	Round 12 difference
message_coherence	0.615	0.74	0.599	0.699	0.025	[-0.014, 0.065]	0.208	0.043
resource_level	0.567	0.569	0.548	0.506	0.044	[0.017, 0.071]	0.001	0.089
channel_effectiveness	0.612	0.76	0.594	0.671	0.071	[0.038, 0.104]	<0.001	0.11
compliance_risk	0.275	0.288	0.297	0.351	-0.041	[-0.072, -0.011]	0.008	-0.065
positive_sentiment	0.446	0.465	0.433	0.336	0.115	[0.033, 0.198]	0.006	0.182
negative_sentiment	0.258	0.341	0.273	0.456	-0.1	[-0.159, -0.042]	0.001	-0.174
visit_intent	0.498	0.488	0.507	0.371	0.127	[0.091, 0.164]	<0.001	0.17
recommend_intent	0.487	0.487	0.483	0.355	0.127	[0.088, 0.167]	<0.001	0.188
trust_in_official_response	0.462	0.507	0.449	0.328	0.166	[0.113, 0.219]	<0.001	0.262

Table 9. DID results for active governance and passive governance conditions in Case 3.

Variable	Active early	Active later	Passive early	Passive later	DID	95% CI	p	Round 12 difference
message_coherence	0.615	0.74	0.599	0.699	0.025	[-0.014, 0.065]	0.208	0.043
resource_level	0.567	0.569	0.548	0.506	0.044	[0.017, 0.071]	0.001	0.089
channel_effectiveness	0.612	0.76	0.594	0.671	0.071	[0.038, 0.104]	<0.001	0.11
compliance_risk	0.275	0.288	0.297	0.351	-0.041	[-0.072, -0.011]	0.008	-0.065
positive_sentiment	0.446	0.465	0.433	0.336	0.115	[0.033, 0.198]	0.006	0.182
negative_sentiment	0.258	0.341	0.273	0.456	-0.1	[-0.159, -0.042]	0.001	-0.174
visit_intent	0.498	0.488	0.507	0.371	0.127	[0.091, 0.164]	<0.001	0.17
recommend_intent	0.487	0.487	0.483	0.355	0.127	[0.088, 0.167]	<0.001	0.188
trust_in_official_response	0.462	0.507	0.449	0.328	0.166	[0.113, 0.219]	<0.001	0.262

The mechanism-chain analysis further shows that the variable changes in Case 3 are not isolated fluctuations. Under the active governance condition, round-level aggregated compliance_risk is significantly positively correlated with negative_sentiment, with Pearson r = 0.766, p = 0.004, and Spearman rho = 0.762, p = 0.004. positive_sentiment is also positively correlated with visit_intent, with Pearson r = 0.669, p = 0.017, and Spearman rho = 0.559, p = 0.059. The correlation between resource_level and trust_in_official_response is positive in direction, but the Pearson correlation does not reach significance. channel_effectiveness and positive_sentiment are also positive in direction but not significant. These results should be interpreted as evidence of process consistency, not as real-world causal effects.

Table 10. Mechanism-chain correlations in active-governance runs in Case 3.

Mechanism pair	n rounds	Pearson r	p	Spearman rho	p
compliance_risk → negative_sentiment	12	0.766	0.004	0.762	0.004
positive_sentiment → visit_intent	12	0.669	0.017	0.559	0.059
resource_level → trust_in_official_response	12	0.483	0.112	0.573	0.051
channel_effectiveness → positive_sentiment	12	0.415	0.179	0.455	0.138

Note 4. Correlations are based on round-level aggregated data from active-governance runs. These results are used to interpret mechanism consistency in the simulation process and are not real-world causal estimates.

Table 10. Mechanism-chain correlations in active-governance runs in Case 3.

Mechanism pair	n rounds	Pearson r	p	Spearman rho	p
compliance_risk → negative_sentiment	12	0.766	0.004	0.762	0.004
positive_sentiment → visit_intent	12	0.669	0.017	0.559	0.059
resource_level → trust_in_official_response	12	0.483	0.112	0.573	0.051
channel_effectiveness → positive_sentiment	12	0.415	0.179	0.455	0.138

Note 4. Correlations are based on round-level aggregated data from active-governance runs. These results are used to interpret mechanism consistency in the simulation process and are not real-world causal estimates.

The results of Case 3 indicate that protocolized LLM-driven ABM can simulate interactive public-opinion evolution among multiple types of agents in a public-information environment. Compared with the rule-shock identification in Case 1 and the information-intervention differentiation in Case 2, Case 3 more clearly demonstrates MASS’s ability to simulate open-ended social communication processes. Public opinion is not directly generated by a single agent; rather, it emerges round by round through platform recommendation, content production, media diffusion, merchant response, governance feedback, and public reaction. Under the active governance condition, organizational-response variables, public-sentiment variables, and behavioral-intention variables form a relatively clear directional chain. This suggests that MASS can not only record state changes of individual agents, but also use public boards and structured outputs to capture processual results produced by multi-agent interaction.

4.5. Cross-Case Validation Matrix

The three cases jointly evaluate the applicability of protocolized LLM-driven ABM across different social-behavioral settings. Case 1 is strongest in policy-shock identification. Groups A, B, and D generate starting-wage changes consistent with the minimum wage rule, while the counterfactual Group C reduces the risk that the model simply reproduces historical conclusions. Case 2 is strongest in information-intervention differentiation. It generates differences in undecided share between targeting and no-targeting conditions and maintains voter-cluster ordering. Case 3 is strongest in multi-agent interaction and mechanism-chain consistency. It generates directionally consistent process outputs among organizational response, platform diffusion, public sentiment, and behavioral intention.

Table 11. Cross-case validation evidence.

Validation evidence	Case 1	Case 2	Case 3
Data analyzability	Groups A–D provide complete main-analysis records; Group E serves as a supplementary verification record	1548 structured records	2016 structured records
Counterfactual contrast	Strong for starting_wage; moderate for price; weak for employment	Targeting significantly reduces undecided_share	Active and passive conditions differ clearly in governance and public-response variables
Directional consistency	Wage responses are stable across A/B/D	Group ordering remains stable across valid run-round combinations	Most DID results are consistent with the expected governance-feedback mechanism
Known-group ordering	Not the main evidence	Leave/swing voters consistently higher than young remain voters	Public-group responses are broadly in the expected direction
Mechanism-chain evidence	wage rule → wage adjustment → partial price pass-through	targeting → lower undecided_share → stable group differentiation	governance capacity → trust/sentiment → visit/recommend intention
Main limitation	Employment response is unstable; Group E is not an effect-replication experiment	Most attitude variables are interpretable in direction but not significant	Platform mechanisms remain simplified; correlations are not causal evidence
Strongest claim supported	rule-shock identification	information-intervention differentiation	multi-agent interaction and process consistency

Table 11. Cross-case validation evidence.

Validation evidence	Case 1	Case 2	Case 3
Data analyzability	Groups A–D provide complete main-analysis records; Group E serves as a supplementary verification record	1548 structured records	2016 structured records
Counterfactual contrast	Strong for starting_wage; moderate for price; weak for employment	Targeting significantly reduces undecided_share	Active and passive conditions differ clearly in governance and public-response variables
Directional consistency	Wage responses are stable across A/B/D	Group ordering remains stable across valid run-round combinations	Most DID results are consistent with the expected governance-feedback mechanism
Known-group ordering	Not the main evidence	Leave/swing voters consistently higher than young remain voters	Public-group responses are broadly in the expected direction
Mechanism-chain evidence	wage rule → wage adjustment → partial price pass-through	targeting → lower undecided_share → stable group differentiation	governance capacity → trust/sentiment → visit/recommend intention
Main limitation	Employment response is unstable; Group E is not an effect-replication experiment	Most attitude variables are interpretable in direction but not significant	Platform mechanisms remain simplified; correlations are not causal evidence
Strongest claim supported	rule-shock identification	information-intervention differentiation	multi-agent interaction and process consistency

Overall, the three cases provide consistent but limited support for RQ3. Protocolized LLM-driven ABM can generate auditable and empirically assessable simulation outputs, and it shows evidence of cross-scenario applicability across policy shocks, information interventions, and platform-mediated public-opinion dynamics. Its validity is supported not by a single metric, but by a combination of data completeness, counterfactual differences, directional consistency, known group ordering, and mechanism-chain evidence.

At the same time, the strength of evidence differs across the three cases. Case 1 is best suited to demonstrating hard rule-shock identification, especially the effect of the minimum wage rule on starting_wage. However, the employment variable is not significant and should not be interpreted as a replication of real employment effects. Case 2 is best suited to showing that a targeting mechanism can reduce undecided_share and preserve group ordering. However, leave_support and issue-salience variables are mostly directional results and should not be interpreted as strong causal effects. Case 3 is best suited to demonstrating process consistency among governance response, sentiment, and behavioral intention under multi-agent public-information interaction. However, Baidu Index is used only as an external search-attention reference, and correlation analysis cannot replace real-world causal validation. Therefore, this study positions MASS as a pre-experimental simulation method for behavioral science, rather than as a real-world outcome predictor.

5. Discussion

5.1. LLM-Driven ABM as a Quantitative Research Method

This section mainly responds to RQ1, namely whether LLM-driven ABM can serve as a method for simulating social behavior in behavioral science. The core methodological contribution of this study is to systematize LLM-driven ABM into a social simulation method that can support quantitative analysis. In many previous naive LLM-agent applications, model outputs often remain at the level of behavioral records, evaluative descriptions, or narrative accounts, and are difficult to transform into comparable, reviewable, and statistically analyzable data. The basic idea of MASS is to convert each agent-round response into a structured observational unit. Simulation results are therefore no longer isolated texts, but agent-round observations that contain role identity, round number, condition identifier, action summary, public message, state variables, state changes, validation flags, and audit information.

On this basis, the methodological foundation of LLM-driven ABM lies in a transformation process of “corpus induction–situational deduction–structured observation.” Large language models are built on large-scale human text corpora, and these corpora are not merely accumulations of random linguistic materials. Rather, they are deposits of role experience, institutional cognition, normative expression, affective judgment, action reasoning, and public narratives formed through long-term human social interaction. In other words, what LLMs learn is not a direct record of individual psychology, but the conditional relationships through which countless “subjects” in human society understand situations, interpret rules, express positions, respond to others, and provide reasons for action in natural language. In this sense, an LLM agent can be understood as a mimetic subject constructed from intersubjective corpora: it does not possess real consciousness, embodied experience, or social responsibility, but it can simulate recognizable role-based responses at the linguistic level.

LLM-driven ABM further re-embeds this corpus-induced generative capacity into the deductive structure of ABM. Researchers define actor positions through role cards, external constraints through rule tables, visible environments through local information boundaries, temporal progression through round-based scheduling, and recordable fields through output schemas. In this way, the LLM is able to generate contextualized actions of corresponding mimetic “subjects” within the rule space defined by the researcher. LLM-driven ABM therefore forms a dual movement from induction to deduction. On the one hand, the LLM “induces” linguistic patterns of human social action from large-scale corpora. On the other hand, the ABM protocol places these patterns into specific roles, rules, events, and information conditions for deductive simulation, making it possible to observe how agents act, update states, and generate cross-round outcomes under the same constraints.

The quantitative character of LLM-driven ABM is therefore not a simple tabular treatment added after text generation. It arises from a continuous transformation among the generative mechanism, experimental structure, and recording protocol. The LLM provides a conditional behavior-generation basis induced from human corpora, enabling agents to generate responses with situational judgment and action reasoning under specific role positions, information environments, and event stimuli. The ABM protocol places these responses into repeatable round structures, treatment conditions, and counterfactual designs, making different agents, rounds, and experimental conditions comparable. Structured outputs, harness checks, and local checks then fix each generated action as an auditable, reproducible, and statistically analyzable agent-round observation.

Therefore, once role cards, rule tables, information boundaries, output schemas, and validation mechanisms are fixed, the simulation system no longer generates only textual responses. It generates structured records containing information about role, time, condition, action, and state change. These records can then be used for DID, DDD, pre-post comparison, known group ordering, correlation analysis, and mechanism-chain testing. The quantitative significance of this approach does not lie in directly equating LLM outputs with real human behavior. Rather, it lies in placing behavior patterns induced from language corpora into controlled situations and making them empirically assessable through protocolized records.

The three cases demonstrate different forms of this quantitative evidence. Case 1 uses agent-round observations to estimate DID and DDD effects under a rule shock. Case 2 compares targeting and no-targeting conditions through structured attitude variables and checks whether voter-cluster ordering remains stable. Case 3 compares active governance and passive governance conditions through organizational and public-response variables, and further examines mechanism chains among governance capacity, sentiment, trust, and behavioral intention. These analyses would not be possible if LLM-agent outputs remained at the level of unstructured text.

MASS can therefore be understood as a quantitative research method that connects generative behavioral production, structured measurement, and statistical evaluation. Its methodological value lies in making LLM-agent simulation auditable, reproducible, and empirically assessable. However, such outputs should still be understood as pre-experimental simulation evidence rather than direct estimates of real-world causal effects. MASS is better suited for mechanism rehearsal, counterfactual comparison, risk identification, and research-design preparation than for replacing real experiments, natural experiments, surveys, interviews, or platform-data analysis.

5.2. Methodological Contribution: From Narrative Role-Playing to Protocolized Evidence

This section mainly responds to RQ2, namely what methodological failures arise in naive LLM-agent simulation and what forms of protocolized control are needed to mitigate them. Although naive LLM-agent simulation can rapidly generate role actions and opinion expressions, it is prone to blurred temporal boundaries, role drift, omniscient perspectives, rule drift, non-computable outputs, invalid numerical values, and replication difficulties. These problems do not mean that LLMs cannot be used for social simulation. Rather, they show that the open-ended generative capacity of LLMs must be embedded into a controllable ABM structure.

MASS transforms the linguistic generation capacity of LLMs into a constrained behavior-generation process through round-based scheduling, role-cluster modeling, local information control, exogenous rule tables, structured outputs, harness checks, reason-action logs, and replication manifests. Round-based scheduling maintains temporal boundaries. Role clusters maintain actor boundaries. Local information control reduces the risk of outcome contamination. Rule tables stabilize experimental conditions. Structured outputs allow textual actions to enter statistical analysis. Reason-action logs provide materials for process auditing. Prompt manifests and replication manifests support third-party review.

The relationship between this method and existing AI social simulation platforms should be understood carefully. Generative Agents, Agent Hospital, OASIS, AgentSociety, and MiroFish have shown that LLM agents can be used to construct small-scale social interaction, medical processes, social media diffusion, large-scale social-life simulations, and open-ended parallel-world inference (Park et al., 2023; Li et al., 2024; Yang et al., 2024; Piao et al., 2025; MiroFish, n.d.). This study does not attempt to replace these platforms, nor does it treat the maximum number of agents or the complexity of a fixed scenario as its main contribution. Instead, it focuses on a transferable research protocol: how researchers can configure roles, rules, information boundaries, treatment conditions, counterfactual conditions, and validation indicators around specific research questions, and how LLM-agent outputs can be transformed into auditable, reproducible, and empirically assessable data.

This configurability also implies greater methodological responsibility. The validity of MASS does not automatically arise from the LLM itself. It depends on how researchers design prompts, role cards, rule tables, output schemas, validation targets, and replication manifests. In other words, prompt design is not an engineering detail in this study, but part of the research design. Role settings affect behavioral boundaries, information boundaries affect contamination risk, rule tables affect experimental conditions, output schemas affect statistical analysis, and validation matrices affect the credibility of conclusions. The clearer the protocolized control, the more likely LLM-driven ABM can move from a role-playing tool to a research method.

5.3. Cross-Case Evidence and Boundary Conditions

The three cases jointly respond to the third research question, but they provide different types of evidence. The strength of Case 1 lies in rule-shock identification. The minimum wage rule change stably enters NJ managers’ starting-wage decisions, while the counterfactual group does not automatically generate the same wage-increase pattern. This shows that the model can distinguish between exogenous rule change and no-intervention conditions. In terms of result direction, Case 1 presents a basic directional structure consistent with the minimum wage case: starting wages rise after the policy shock, employment effects do not show a stable significant negative change, and price adjustments remain relatively mild. However, because the employment variable in the main analysis does not reach conventional significance, the results only show that MASS generates behavioral response patterns directionally close to the classic case. They do not support a claim that MASS precisely replicates the real employment effect.

For this case, the value of Case 1 lies mainly in showing that MASS can generate behaviorally consistent responses from mimetic subjects under explicit exogenous rules, regional differences, and managerial role constraints. The system does not mechanically translate the minimum wage shock into simultaneous significant changes across all variables. Instead, it first appears in the stable adjustment of NJ managers’ starting-wage decisions and then produces relatively mild subsequent responses in employment and price variables. Thus, the validity of MASS in rule-shock scenarios lies mainly in its ability to simulate mimetic subject behavior, external-constraint transmission, and directional outcome structures, rather than in precisely reproducing real employment effects.

The strength of Case 2 lies in information-intervention differentiation. The official referendum result and the MASS post-period Leave/Remain proportions among decided attitudes are descriptively close, which can serve as auxiliary evidence of external proportional alignment, but it cannot be interpreted as showing that MASS predicted the real referendum outcome. The main methodological evidence in Case 2 still comes from the comparison between targeting and no-targeting conditions. The targeting condition significantly reduces undecided_share and maintains voter-cluster ordering, but leave_support and most issue variables do not reach significance. Therefore, the results cannot support a claim that digital targeting significantly increased Leave support. This outcome is also consistent with cautious conclusions in existing political communication research: campaign contact and political advertising often have limited average persuasive effects in general elections (Kalla & Broockman, 2018), while Brexit-related research and official investigations also suggest that Cambridge Analytica or AggregateIQ should not be simplified into a single causal actor that determined the referendum result (Information Commissioner’s Office, 2018; Rone, 2023).

For this case, the value of Case 2 lies mainly in showing that MASS can distinguish behavioral characteristics of mimetic subjects under different experimental conditions. This reflects its discriminant validity and process validity in information-intervention scenarios. The system can generate structured simulation outputs that are compatible with cautious judgments in existing political communication research, without leaking the final outcome or presupposing a single causal actor, and these outputs can be examined through structured variables.

The strength of Case 3 lies in simulating public-opinion evolution under multi-agent interaction. Baidu Index provides an external event-stage anchor for Case 3, indicating that the simulation window covers the main attention cycle of the real event, from low-level warming and rapid diffusion to the peak around the May Day holiday and the subsequent decline. However, Baidu Index reflects search attention. It is not equivalent to governance variables, sentiment variables, or behavioral-intention variables in MASS, and therefore should not be directly numerically fitted to simulation variables. The main evidence in Case 3 comes from the counterfactual comparison between active governance and passive governance, as well as the continuous interaction among multiple types of agents in the public-information environment formed by the public board.

For this case, the value of Case 3 lies mainly in showing that MASS can simulate public-opinion development under multi-agent interaction. Platform agents, media agents, content creators, merchants, governance agents, and public agents form round-by-round feedback through publicly visible information, allowing public-opinion diffusion, service reception, governance response, public sentiment, and behavioral intention to develop into recordable, auditable, and statistically testable processual relations. Case 3 therefore demonstrates the interaction-simulation capacity and process validity of MASS in platform-mediated public-opinion and public-governance scenarios. The system can generate simulation outputs that are aligned with the real-world attention cycle and can be examined through structured variables and mechanism chains, without leaking real-world attention intensity or presupposing the final outcome.

Taken together, the three cases show that MASS is not limited to a single case or a single variable. It can generate analyzable agent-round observations across three different mechanisms: policy rules, information communication, and public opinion. Case 1 demonstrates its ability to identify exogenous rule shocks and generate structured decision responses from mimetic managers. Case 2 demonstrates its ability to distinguish group-attitude evolution under different information-intervention conditions. Case 3 demonstrates its ability to simulate public-opinion evolution under round-by-round multi-agent interaction in a public-information environment. Therefore, the three cases provide consistent but limited support for RQ3: after protocolized control, LLM-driven ABM can generate auditable and empirically assessable simulation outputs with some evidence of cross-scenario applicability, but these outputs should be understood as pre-experimental simulation evidence rather than real-world causal-effect estimates or real outcome predictions.

5.4. Limitations, Ethical Boundaries, and Future Research

This study has several limitations. First, LLM-driven ABM is highly sensitive to prompts, role settings, model versions, and parameter settings. The granularity of role cards, the way information is presented, the strength of rule tables, output schemas, temperature settings, and model-provider environments may all affect simulation trajectories. This study reduces these risks through prompt manifests, replication manifests, local information boundaries, harness checks, and local checks, but it cannot claim to have fully eliminated prompt sensitivity or model-version dependence. Future research should systematically conduct prompt robustness tests, sensitivity analysis, cross-model replication, and cross-provider comparison to examine the stability of results across different prompts, models, and runtime environments.

Second, the number of independent runs in this study remains limited. The main analysis of Case 1 relies on Groups A–D as early historical benchmark data. Case 2 and Case 3 form treatment and counterfactual condition comparisons, but the estimation precision of weaker or more variable outcomes such as leave_support, issue_salience, employment, and message_coherence may still be affected by the number of independent replications, the number of agents, model-output variance, and the structure of condition groups. This study does not interpret non-significant results as simulation failures, nor does it simply attribute them to insufficient runs. A more cautious interpretation is that the current results support several directional and mechanism-based judgments, while the stability of weaker effect variables still requires more independent runs. Future research can increase independent replications and use bootstrap methods, multi-run aggregation, cross-model replication, and sensitivity analysis to further examine whether these variables have stable directions and reviewable effect ranges.

Third, model outputs require continuous calibration. MASS is not a model that becomes realistic automatically after a single configuration. The fluctuation of the employment variable in Case 1, the non-significance of most attitude variables in Case 2, and the simplified platform mechanisms and public-information diffusion rules in Case 3 all indicate that external data, expert judgment, pilot runs, and error analysis remain important for model calibration. Future research should introduce more fine-grained real-world data, platform data, interview materials, expert coding, and event timelines to calibrate agent profiles, information-exposure conditions, state-variable ranges, and mechanism-chain assumptions, thereby improving external validity and structural validity.

Fourth, corpus priors and case-contamination risks remain difficult to completely exclude. The NJ–PA minimum wage case, Brexit, and Zibo barbecue may all exist in model training corpora or public knowledge. This study reduces such risks through local information boundaries, case blinding, counterfactual groups, hidden-field restrictions, final-outcome masking, and contamination checks, but it cannot claim to have fully eliminated model priors. Especially for classic cases, LLMs may already have learned related public narratives. Future research can use synthetic cases, background rewriting, case blinding, event renaming, cross-model replication, and counterfactual perturbation to further test whether simulation results depend on prior case memory or are generated from role, rule, and local-information constraints.

Fifth, the evidence type of MASS should still be limited to pre-experimental simulation evidence. The DID, DDD, group-ordering tests, and mechanism-chain tests in this study are all based on synthetic agent-round observations generated by MASS, rather than direct causal estimates from real human data. The Case 1 results should not be interpreted as a precise replication of real employment effects. The Case 2 results should not be interpreted as causal evidence that Cambridge Analytica or AggregateIQ affected the real referendum outcome. The Case 3 results should not be interpreted as a numerical fit to Baidu Index or real tourism-consumption outcomes. The main value of MASS lies in mechanism rehearsal, counterfactual comparison, risk identification, and research-design preparation, rather than replacing real experiments, natural experiments, surveys, interviews, or platform-data analysis.

Finally, the use of MASS requires clear ethical boundaries. This study positions MASS as a pre-experimental simulation method, not as a real-world predictor or automated decision system. Simulation outputs should not be directly used for real-world policy decisions, voter manipulation, individual profiling, commercial discrimination, public-opinion manipulation, or unvalidated population interventions. Especially in political communication, public governance, and platform-mediated public-opinion scenarios, researchers need to disclose model versions, prompt design, data sources, validation strategies, and limitations, and avoid packaging simulation results as real-world causal conclusions. A responsible use of MASS is to treat it as a tool for mechanism exploration, risk anticipation, and research-design preparation, and to interpret its results together with real-world evidence, expert review, and public deliberation.

6. Conclusions

This study proposes and evaluates Multi-Agent Social Simulation as an operational form of protocolized LLM-driven ABM. Through role-cluster modeling, round-based scheduling, local information boundaries, exogenous rule tables, structured outputs, harness checks, reason-action logs, and replication manifests, MASS transforms the open-ended generative capacity of LLM agents into recordable, auditable, and statistically analyzable agent-round observations.

With respect to RQ1, this study argues that LLM-driven ABM can serve as a pre-experimental social-behavioral simulation method in behavioral science. However, its methodological position should be mechanism rehearsal, counterfactual comparison, risk identification, and research-design assistance, rather than a replacement for real-world experiments or causal identification. With respect to RQ2, this study shows that naive LLM-agent simulation is prone to blurred temporal boundaries, role drift, omniscient perspectives, rule drift, non-computable outputs, invalid numerical values, and replication difficulties. These problems require protocolized control through role cards, information boundaries, rule tables, output schemas, harness checks, local checks, and replication manifests. With respect to RQ3, the three cases provide preliminary cross-scenario evidence: Case 1 supports rule-shock identification, Case 2 supports information-intervention differentiation, and Case 3 supports public-opinion evolution simulation under multi-agent interaction in a public-information environment.

Overall, the value of MASS does not lie in claiming that it can precisely predict real-world outcomes. Rather, it lies in embedding the contextual generative capacity of LLM agents into a controllable, reviewable, and statistically analyzable ABM structure. The three cases show that protocolized LLM-driven ABM can generate empirically assessable simulation outputs in policy-shock, information-intervention, and public-opinion scenarios. Future research should continue to strengthen prompt robustness, independent-run replication, external-data calibration, cross-model comparison, case blinding, and open reproducibility, so that LLM-driven ABM can move from an exploratory tool toward a more mature research method for the social and behavioral sciences.