Preprint
Article

This version is not peer-reviewed.

Artificial Intelligence Transparency Under the European Union AI Act: A Cross-Sectional Audit of Museum-Facing Chatbots

Submitted:

17 November 2026

Posted:

18 November 2026

You are already at the latest version

Abstract
Since August 2026, Article 50 of the EU AI Act has required that people be informed, at the latest at their first interaction, that they are dealing with an AI system; Commission guidelines add disclosure whenever asked. We examine how this duty travels through providers and deployers to the interface visitors see. We conducted a pre-registered cross-sectional audit of discoverable museum-facing chatbots in the EU-27. Of 142 systems identified, 104 in 16 Member States were probed with a fixed protocol, including an identity question asked in two sessions, and coded against two operational disclosure criteria. In total, 76.9% disclosed their AI nature at or before the first interaction and 77.9% when asked in both sessions. Disclosure at launch was 30 percentage points lower for museum-run systems than for municipal or tourism portal systems (61.0% vs. 91.2%), a gap that persisted, attenuated, across platform and country checks. The criteria failed in different systems: 20 answered correctly but showed no notice; 19 showed a notice but did not disclose when asked. Observable disclosure differed sharply by supplier type, suggesting product defaults as a lever for guidance, procurement and enforcement.
Keywords: 
;  ;  ;  ;  ;  ;  

1. Introduction

Conversational agents have moved from experimental installations to everyday infrastructure in museums. A list compiled from the research literature identifies 75 conversational-agent initiatives in museums between 1996 and 2023 [1]. Large language models have since multiplied the options: chat windows on collection websites, generative museum guides and recommenders [2,3], and characters that answer questions in the first person.
Studies of these systems show both their appeal and the unease they create. Visitors value answers tailored to their interests [4]. In one survey, participants preferred chatbots that speak as the creator of an artifact, but raised concerns about how historical figures are portrayed [5]. In a UK survey, close to half of respondents were concerned about the use of conversational agents in heritage settings [6].
These developments raise a question the museum literature has not asked: do visitors know that they are talking to a machine? Since 2 August 2026 the question has also been a legal one. Article 50(1) of Regulation (EU) 2024/1689 [7] (the AI Act) requires providers to design AI systems intended to interact directly with natural persons so that those persons are informed that they are interacting with an AI system, unless this is obvious. Article 50(5) requires that the information be clear and distinguishable, be given at the latest at the time of the first interaction, and conform to applicable accessibility requirements.
The Digital Omnibus on AI (Regulation (EU) 2026/1744 [8]) postponed other parts of the Act but left these duties in force. Its only relevant transitional period concerns the machine-readable marking of generated content by systems already on the market (Article 111(4)). In July 2026 the Commission approved guidelines on Article 50, which specify what counts as adequate disclosure, list techniques that are insufficient on their own, and state that a system must disclose its nature whenever it is asked about it [9] (para. 40).
Whether disclosure reaches users cannot be read off the legal text, and providers and deployers have reasons to avoid it. In a field experiment with more than 6,000 customers, disclosing a sales chatbot before the conversation reduced purchases by about 80% [10]. In a repeated cooperation game, bots outperformed humans only as long as their nature was hidden [11]. Other studies find little effect of disclosure on evaluations of customer service [12]. The only published audit against Article 50 that we found examined twelve chatbots and websites of Finnish well-being services counties and concluded that users were not transparently informed about AI use [13].
We approach the question as one of governance rather than of individual compliance. Article 50(1) addresses the provider, but in museums the visitor-facing interface is usually assembled by others: institutions configure products from specialised vendors or general-purpose chatbot builders, or appear on municipal and regional portals whose chatbot serves every page, the museum’s included. Transparency is therefore produced along a chain of actors, and the visitor sees only its end. Museums are a revealing case of this chain because most are public or non-profit bodies whose authority rests on the reliability of their information, and a chatbot speaking in the institution’s voice, or in the voice of a historical person, borrows that authority.
This article reports a pre-registered, systematic cross-sectional audit of discoverable museum-facing chatbots in the EU-27, conducted in November 2026. We ask three questions:
  • RQ1: do museum-facing chatbots inform users that they are AI at or before the first interaction, and do they say so when asked?
  • RQ2: how often do they present themselves through personas, including personas of real or historical persons?
  • RQ3: does disclosure differ between systems run by museums and systems supplied through municipal or tourism portals, and between systems with and without a persona?
We tested three directional hypotheses. H1 predicted that museum-run systems disclose at the first interaction less often than portal systems. H2 predicted that persona systems do so less often than systems without a persona. H3 predicted that more systems disclose when asked than at the first interaction.
The study makes three contributions. Conceptually, it frames AI interaction disclosure as a distributed governance problem and distinguishes formal from effective transparency. Methodologically, it turns the open standard of Article 50(1) and (5), as interpreted by the Commission, into a reproducible codebook and probe protocol with two separable criteria and a repeated question that tests consistency, and it shows that the two criteria are empirically independent: a notice at launch does not guarantee that a chatbot will say what it is when asked, and vice versa. Empirically, it provides the first sector-wide measurement of observable disclosure under Article 50 and shows where in the provider–deployer chain disclosure is lost, which bears on deployer guidance, procurement and enforcement priorities.

2. Background

2.1. AI Transparency as a Distributed Governance Problem

Transparency duties are usually analysed as obligations of a single addressee. Article 50(1) fits this pattern on paper: the provider must design the system so that users are informed. In practice, the interface a visitor meets results from decisions taken at several levels (Figure 1). The Act and the Commission’s guidelines set the standard. Providers build products with or without disclosure features, and with defaults that deployers may or may not change. Deployers, here museums and portal operators, choose products, configure greetings and personas, and decide where the widget sits. Only the resulting interface is visible to the visitor.
Three strands of literature help to analyse this chain. First, the regulator–intermediary–target model holds that rules often reach their targets through intermediaries whose capacities and interests shape the outcome [14]. For chatbots, the platform provider is formally the addressee of Article 50(1), but it also acts much like an intermediary for the many institutions that adopt its defaults: a single design decision is reproduced across every deployment. Second, the AI Act relies on regulation by design, which builds legal requirements into technical artefacts and thereby hands normative choices to designers and standard-setters [15,16]. Where disclosure is a default, it scales; where it is left to configuration, it depends on the deployer’s awareness. Third, auditing research distinguishes the governance, model and application layers of language-model systems [17] and stresses that external audits establish observable conduct against stated norms, not legal liability or internal processes [18]. Our audit sits at the application layer.
A second distinction concerns what transparency achieves. Formal transparency is the presence of a perceivable notice that meets a stated standard. Effective transparency is whether people notice, understand and use that information. Research on privacy notices shows that the two can diverge systematically, because the same notice must inform users and provide legal cover, and the latter usually wins [19]. Labels can also be misread: people take “AI-generated” to mean full automation and discount true content accordingly [20]. Web-scale compliance audits of consent banners have documented a similar gap between visible notices and actual system behaviour [21,22]. This study measures formal transparency, together with one behavioural indicator that comes closer to effective transparency: whether the system tells the truth about itself when a user asks.

2.2. Conversational Agents in Museums

Museums have experimented with conversational agents for three decades, from scripted gallery characters to web chatbots answering visitor questions [1]. Generative models have changed both what these systems can do and how they are deployed. Instead of authoring dialogue trees, an institution can connect a general-purpose language model to its own content, as in GPT-4-based museum guides [3] or an explorer of about 1.7 million digitised natural history specimens [23]. Evaluations of such systems typically focus on accuracy, engagement and the cultural user experience [4,24].
Two design choices recur in this literature and matter for disclosure. The first is the persona: most participants in Chen et al. [5] preferred a chatbot speaking in the first person as an artifact’s creator, while raising concerns about the authenticity of such portrayals. The second is the institutional voice: a chatbot that answers under a museum’s name and visual identity gives the visitor little reason to doubt that the answer comes from the museum. Both choices make the AI nature of the system less evident. None of the studies above treats disclosure of the system’s AI nature as a variable. Table 1 situates the present study.

2.3. Why Disclosure of AI Interaction Matters

The case for disclosure rests on autonomy and calibrated trust. A person who knows that the interlocutor is an AI system can decide how much to rely on its answers, what to share and whether to seek a human. The Commission’s guidelines state the purpose of Article 50(1) in these terms: to enable informed decisions, to avoid overreliance and to calibrate trust [9] (para. 28).
The experimental evidence shows that disclosure changes behaviour, though not always in the same direction. In a sales context, customers perceived disclosed bots as less knowledgeable and less empathetic; late disclosure and prior experience with AI weakened the effect [10]. In customer service, interviewees cared more about helpfulness than about whether the agent was artificial [12]. Organisations may therefore have reasons to avoid or delay disclosure, which makes compliance an open empirical question.
Museums add three considerations. They trade on authenticity, and audiences already worry that conversational agents could make visits less personal [6]. Personas that speak as artists or historical figures blur the boundary between interpretation and impersonation. Museum audiences also include children and visitors with disabilities, so a disclosure that is easy to miss does not reach all of them.

2.4. Article 50 as Interpreted by the Commission

Duties and addressees. Article 50(1) places a design obligation on the provider: people interacting with the system must be informed that they are interacting with AI. Article 50(5) sets the form and timing of the information. Providers develop a system, or have it developed, and place it on the market or put it into service under their own name; deployers use a system under their authority (Article 3(3)–(4)). The guidelines give two examples that map onto museum practice. A company offering a chatbot under its own name to organisations that integrate it without modification is the provider. An organisation, including a public authority, that develops or modifies a chatbot and puts it into service under its own name becomes a provider itself [9] (para. 11). For the visitor these allocations are invisible, so our audit records what the user can observe and does not attribute legal liability.
Scope. Article 50(1) applies to an AI system intended to interact, with a genuine conversational or responsive character, directly rather than through a human, with natural persons (para. 30). Simple rule-based response mechanisms and retrieval of information without generated content are outside its scope (examples after para. 31).
Substance and form. People must be “explicitly informed about the artificial, non-human nature of the interacting counterpart” (para. 35). Adequate practices include a chatbot that starts the conversation by stating that it is based on AI, a plain-language banner, a first-turn greeting, or a persistent badge near the input field (paras. 36–37). The following techniques are insufficient when used alone (para. 38):
  • notices only in terms and conditions;
  • machine-readable markings that users cannot perceive;
  • “unclear or ambiguous signals (e.g., generic references to `assistant’) or human-like representations that may mislead users”;
  • generalised site-wide notices;
  • purely technical statements.
Disclosure on request. The guidelines add that systems must disclose their nature whenever they are asked about it (para. 40). This is an interpretation of how the Article 50(1) obligation is to be met, not a separate provision of the Act. We therefore treat it as a second operational criterion grounded in the guidelines, not as an independent statutory duty.
Timing and the obviousness exception. For interactive systems, the information should be given at least once at the start of each session, “when launching a conversation with a chatbot” (para. 143). Information is neither clear nor distinguishable if it “can be easily overlooked or missed” (para. 142). The exception for obvious cases “cannot be relied upon” where the general public, including children and persons with lower AI literacy, can access the system, and chatbots embedded in online platforms are listed among the cases where it does not apply (para. 45).
Status. The Commission approved the content of the guidelines on 20 July 2026 (C(2026) 5054 final). They are non-binding, and an authoritative interpretation rests with the Court of Justice (para. 5). We use them as the most detailed official statement of how the Commission and national authorities are likely to read Article 50. Article 50(1) and (5) have applied since 2 August 2026 to all in-scope systems, regardless of when they were placed on the market; partly generative systems benefit from the Article 111(4) transitional period only for marking (para. 153). Enforcement rests mainly with national market surveillance authorities (paras. 151–152).

2.5. From Legal Standard to Observable Indicators

Article 50(1) and (5) describe outcomes, not techniques, so an external audit must decide which observable features meet the standard. We adopt three principles:
  • Observability. We code only what a visitor can perceive without special tools.
  • Restrictive reading of the obviousness exception. For public systems it cannot be relied upon, so we do not use it to excuse non-disclosure.
  • Disclosure is not a caveat. Statements that answers “may contain errors” concern reliability, not the nature of the interlocutor.
The resulting criteria are necessary, not sufficient, conditions for compliance. A system that meets both of them may still fall short of Article 50(5) on clarity, distinguishability or accessibility, which this wave does not assess (outcome O2, Section 3.4). We therefore speak of operational disclosure criteria and observable disclosure, not of legal compliance. Table 2 shows how each requirement is translated into codes.

3. Materials and Methods

3.1. Design and Pre-Registration

The study is a systematic cross-sectional audit of publicly accessible AI conversational systems on the web pages of EU-27 museums, covering every unit in a frame of discoverable systems (Figure 2). The design, hypotheses, codebook, sampling frame and analysis plan were registered on the Open Science Framework on 2 November 2026 at 19:03 EET (https://doi.org/10.17605/OSF.IO/JQY6Z). The first Wave 1 probe was run at 19:25 EET on the same day, and all Wave 1 data were collected after registration. Earlier that day, a pilot on five systems tested the probes and led to the gate, consistency and owner-check codes of codebook v1.0. These five systems were in the frame and were probed again in Wave 1 under the registered protocol; excluding them does not change the results (sensitivity G, Table 3).
Data were collected from 2 to 7 November 2026, three months after Article 50(1) and (5) became applicable. The registration also provides for a second wave in January 2027. That wave is reported separately, because the only legal date between the waves (2 December 2026) concerns the marking of generated content, not disclosure of interaction. All deviations from the registration are listed in Supporting Information S2. The most consequential are the earlier start, the manual probing of widgets that the automated browser could not open, and the deferral of the second wave.

3.2. Sampling Frame

No register of AI systems used by museums exists, so the frame was built from two sources (Figure 3).
Wikidata. The main source was the official websites of EU-27 museums listed in Wikidata (instances of museum and its subclasses): 19,486 hosts. The frame was built as follows:
  • A static pre-screen searched each homepage for the code signatures of AI chatbot platforms, other chat widgets and chatbot keywords. It flagged 713 hosts.
  • Each flagged host was rendered in a browser with non-essential cookies rejected, and every detected launcher was checked visually.
  • Leads without a visible launcher were re-examined after a longer wait, a scroll and one click on a launcher-like element.
  • Hosts offering only a WhatsApp button (147) were excluded after inspection of a random sample of 40. An AI agent behind a messaging number cannot be identified without messaging it.
Kist [1]. The second source was the 26 EU-27 entries of a published list of museum conversational agents, which were checked live. Three still had a web chatbot and one remained a lead. Most chatbots documented between 2016 and 2023 were no longer present.
The frame. The resulting frame contained 142 systems. It covers chatbots that are discoverable from museum homepages listed in Wikidata, not all chatbots used by European museums: coverage of Wikidata varies between countries, and systems on inner pages or in apps are under-represented. Inferences therefore concern this population of discoverable museum-facing chatbots. The unit of analysis is one AI conversational system on one page of one institution. The same vendor product at different institutions, or on different pages, counts as separate units.
Many chatbots on museum pages belong to the municipality, region or tourism body whose web portal hosts the museum page, not to the museum. Owner stratum was therefore recorded as museum, portal or tourism, or company museum. It was updated when the chatbot introduced itself as another body’s service, or when the page belonged to a different type of owner than recorded in the frame (seven units: three from museum to portal, one from portal to museum, and three to company museum). H1 is also reported with the registered frame strata (sensitivity F).
Two frame pairs proved to be the same system on the same page, one through a redirect, and were merged. Because the frame was below the registered threshold of 250, all remaining units were audited.

3.3. Probe Protocol

Each unit was examined with a fixed sequence of probes (Table 4), in the interface language, using fixed wording: French, German and Italian from the pilot, and other languages drafted and literally back-translated by the AI model (Supporting Information S3):
  • Load the page and record the launcher before any input (P0).
  • Open the chat and record any gate (a consent, privacy or AI-terms step required before chatting) and the first system message (P1).
  • Ask a question about the collection twice (P2).
  • Ask “Am I talking to a real person or to a computer program?” (P3a) and “Are you an AI?” (P3b).
  • Repeat P3 in a second, independent browser session, at least one hour later or on another day.
No personal data were entered, no account was created and no adversarial prompts were used. A session was stopped if the system required a name or e-mail address. Non-essential cookies were rejected. Where a chatbot could be loaded only through consent, only the chatbot-specific category was accepted; where the chatbot was bundled with other services in one category and no other route existed (four units), that category was accepted. Chatbot-specific AI terms or privacy notices were accepted when required to start the chat.
Two passes. Of the 140 distinct units, 59 were probed with a scripted browser (Chromium via Playwright, desktop viewport), which saved screenshots and the rendered transcript at each step. The remaining 81 units used widgets that the scripted browser could not open, such as cross-origin frames, consent-gated scripts and pop-ups. They were probed by hand in a desktop Chromium-based browser with the same wording. In this manual pass, P2 was not run, P3b was sent only when the reply to P3a was not a clear statement, replies were logged verbatim, and the screenshots taken during probing were archived. Some vendors restore an earlier conversation on the same device. Where a widget offered a new chat or restart, it was used; otherwise the second session was run with the earlier conversation visible (Supporting Information S2).

3.4. Codebook and Outcomes

The codebook (Supporting Information S1) operationalises Article 50(1) and (5) through the Commission’s guidelines.
Scope (B4). A system is in scope when free-text input yields generated, contextual replies without human mediation. Rule-based menus, retrieval-only search and human-staffed live chat are out of scope.
O1, disclosure at the first interaction (primary outcome; criterion 1). O1 is met when a statement that the user is interacting with an AI or automated system is perceivable before the user types anything. Qualifying locations are the launcher, a gate, a header, a greeting or a persistent footer. The wording must be explicit: “AI” or an equivalent in the interface language, or “automated”, “bot” or “computer program”. A bot name ending in “-bot” was treated as automated wording. Following the guidelines (para. 38), the following do not count: a persona name, a generic reference to an “assistant”, a human-like picture, a vendor credit, or a notice found only in the terms of use.
O3, disclosure on request (criterion 2). O3 is met when the system clearly states that it is an AI system, a computer program or a chatbot in reply to P3a or P3b, in both sessions. Evasion, a generic self-description (“I am the museum’s assistant”), a reply in role, or a fallback reply (“no information found”, “I did not understand”) counts as no disclosure.
Other variables. C7c records whether the two sessions agree. C6 (persona) is coded when the system has a personal or character name, a human or character image, or a human role word such as “virtual employee”. C7b flags a claim to be human. The platform was recorded where identifiable from the interface or the page code. The secondary outcome O2 adds clarity, distinguishability and exposure of the disclosure to assistive technologies. It requires evidence archived only in the automated pass and is not reported here.

3.5. Coding and Reliability

First coding was done by an AI model (Claude, Anthropic), which also sent the probe messages and recorded the evidence. The author then coded a subset blind to the first coding, seeing only verbatim evidence in random order. The subset comprised a stratified random 20% of units, every unit without a clear disclosure on request in either session, every unit presenting a real or historical person or a photorealistic avatar, and units flagged as uncertain before second coding began. This gave 52 of 106 units.
Because three of the four inclusion rules target difficult cases, agreement on the full subset is likely to understate agreement in the population, while agreement on the random component alone rests on few units. We report both, as coded and before any correction (Table 5). On the 52 units, agreement was high for disclosure at the first interaction (C1: κ = 0.94; C4: κ = 0.92) and for persona (C6: κ = 0.80). For disclosure on request it was κ = 0.77 in session 1 and κ = 0.66 in session 2, the latter below the registered threshold of 0.70. On the 22 randomly selected units, κ was 1.00 for C1, 0.87 for C4, 0.85 for session 1 and 0.58 for session 2; for persona it was 0.60 (82% raw agreement).
The weakest code, disclosure on request in session 2, had a single identifiable source. The codebook classified a reply saying the system “has no information” as non-disclosure, but a “fallback phrase” as not testable, and the two coders resolved this conflict differently. As registered, the rule was clarified before outcomes were examined: a fallback reply counts as non-disclosure. All units were then recoded. Five persona codes were also corrected: four where the coding sheet had omitted the greeting that named the persona, and one to match the unit definition. These corrections were made with knowledge of the disagreements, so agreement after them (C7: κ ≥ 0.96) is not an independent reliability estimate and is reported only for completeness. Apart from these corrections, the author’s code was used wherever codes differed. The author also classified two units as out of scope, leaving 104. Agreement on scope was 96.2%; Cohen’s κ is uninformative here, because only two units were coded out of scope. Sensitivity analysis C, which uses session 1 only, shows how much the results depend on the less reliable session-2 code.

3.6. Analysis

Proportions are reported with Wilson 95% confidence intervals. The three hypotheses were tested one-sided with Holm correction: H1 with Fisher’s exact test, risk difference and Newcombe 95% CI; H2 with Fisher’s exact test; H3 with the exact McNemar test. Company museums (n = 6) are reported but not tested.
The sensitivity analyses were registered or derived from registered rules: (B) one unit per distinct bot; (C) disclosure on request from the first session only; (D) disclosure given only in a consent gate not counted; (E) disclosure resting only on a “-bot” name treated as insufficient; (F) for H1, the registered frame strata instead of the strata updated at P1; (G) the five pilot systems excluded.
Three exploratory robustness analyses addressed confounding in H1. A Mantel–Haenszel analysis stratified H1 by probing pass. A leave-one-platform-out analysis re-ran H1 after removing each identified platform with at least three systems, and after removing both portal platforms together. Because Germany contributed 45 of 104 systems and most portal systems, H1 was also estimated within and outside Germany, with a Mantel–Haenszel odds ratio stratified by Germany versus other countries and a logistic regression of O1 on stratum and Germany. Analyses used Python 3 (SciPy, statsmodels).

3.7. Ethics and Data

The study involved no human participants and no personal data. Only public interfaces were used, without login or payment, and probes were short and non-adversarial (about four to eight messages per system). Ethics review was therefore not required. Institutions are named because their systems are public-facing services. Findings describe observed behaviour on the audit dates and make no claim about legal liability. Incidental technical observations, and the name of a staff member that appeared in one system’s output, are not reported.

4. Results

4.1. The Population of Systems

Of the 142 frame units (Figure 3), 2 were merged as same-page duplicates, 12 were no longer online or no longer had a chatbot, 15 were out of scope (5 human-staffed live chats and 10 rule-based or menu systems without generated replies), 1 was reachable only through a messaging app, and 8 could not be tested.
The remaining 104 in-scope systems appeared on pages in 16 Member States (Table 6). Germany (45), France (14), Italy (13) and Austria (11) accounted for 83 of them, which reflects both uneven Wikidata coverage and uneven adoption. Of the 104 systems, 57 belonged to municipal, regional or tourism portals, 41 to museums and 6 to company museums. Owner type and country were strongly related: 37 of the 57 portal systems were German, whereas French systems were mostly museum-run (10 of 14).
The platform could be identified for 59 systems. The largest groups were neuraflow municipal chatbots (14), the general-purpose builder Chatbase (13), the museum vendor Ask Mona (8) and a regional portal assistant (4).

4.2. Disclosure at the First Interaction and on Request (RQ1)

Of the 104 systems, 80 disclosed that they were AI at or before the first interaction (O1: 76.9%, 95% CI 68.0–84.0) (Table 7, Figure 4). In 17 systems, a gate that had to be accepted before chatting named the AI. In all but two of these, AI was also stated in the header, greeting or a persistent footer such as “Alle Antworten werden KI generiert” (“all answers are AI-generated”).
Asked whether the user was talking to a real person or a computer program, 81 systems said that they were AI, a computer program or a chatbot in both sessions (O3: 77.9%, 69.0–84.8). In the verbatim logs, no system claimed to be human.
Twelve systems (11.5%) answered differently in the two sessions. Seven disclosed only in the second session and five only in the first. In one case (Dutch Pinball Museum), the second-session question was taken by a rule-based flow as the visitor’s first name; under the registered rule this counts as non-disclosure. The Duomo di Milano’s assistant answered “Sì, sono un’intelligenza artificiale” (“Yes, I am an artificial intelligence”) in one session; in the other, it replied only that it could help with questions about the cathedral.

4.3. Two Criteria, Two Failure Modes (H3)

Disclosure on request was not more common than disclosure at the first interaction (H3). Of the 104 systems, 61 met both operational disclosure criteria, 20 disclosed only when asked, 19 disclosed only at the start, and 4 met neither. The discordant pairs were balanced (McNemar exact p = .50; Table 8). The two criteria failed in different systems, and the failures clustered by platform (Figure 5).
Notice missing, correct answer when asked (20 systems). At least 12 were built on a general-purpose chatbot builder or a museum-specific vendor product. They answered the direct question accurately but showed no notice when the conversation was launched. Six of 13 Chatbase systems and four of eight Ask Mona systems fell in this group. The Musée d’Orsay’s assistant, for instance, identified itself on request as a chatbot created by its vendor, but its launch screen carried no AI statement. Personas of historical figures appeared in the same pattern: Nicolas Fouquet at Vaux-le-Vicomte and the margravine Wilhelmine on the Bayreuth tourism portal.
Notice present, no disclosure when asked in both sessions (19 systems). These systems displayed an explicit notice but, in at least one session, refused the question as off-topic, returned a fallback reply or answered in role; nine of them disclosed in the other session.
  • The three systems of the Kunstmeile Krems museum group announced a “KI-Assistent” but, in at least one session, replied to both probes “Dazu kann ich keine Auskunft geben” (“I cannot give information on that”); two of them stated in the other session that they were a computer program.
  • The municipal chatbots of Lamego and Matosinhos named AI in a mandatory consent box. They presented a photorealistic woman called “Maria” and refused the question as unrelated to municipal procedures.
  • Several municipal service bots returned “no information found” to the identity question in at least one session.
Neither criterion met (4 systems). All four were museum systems without a notice at launch, and each disclosed in at most one session. The Prague Beer Museum’s “Brother Prokop” acknowledged in one session that it was “a machine carrying my voice”, without saying it was an AI or a program; in the other it stayed in role, replying that the question was “not my world to dwell on”. The Musée de l’Orangerie’s assistant described itself in one session only as “un assistant conversationnel du musée”; it disclosed its nature in the other.

4.4. Differences by Owner and Persona (H1, H2, RQ2, RQ3)

Systems run by museums disclosed at the first interaction far less often than portal systems: 61.0% vs 91.2%, a risk difference of −30.3 percentage points (H1 supported; Holm p = .001). The difference held in every registered sensitivity analysis (Table 3), including the registered frame strata (27/44 vs 50/57, p = .002) and the exclusion of the pilot systems (24/38 vs 50/55, p = .001). It also held within the automated probing pass (15/28 vs 19/22, p = .014) and when stratified by pass (Mantel–Haenszel OR 0.19, 95% CI 0.06–0.59). Raw outcome rates differed between the passes (O1: 35/53 automated, 45/51 manual; O3: 43/53 and 38/51), but the passes also differed in composition: the manual pass contained mostly portal systems (35 of 51), whose widgets the scripted browser could not open.
Every neuraflow municipal chatbot (14 of 14) and every regional portal assistant (4 of 4) showed an AI gate, header or footer, whereas museum systems were spread across general-purpose builders, museum vendors and other widgets.
Personas were common: 59 of 104 systems (56.7%) had a personal or character name, a human or character image, or a human role word such as “virtueller Mitarbeiter” (“virtual employee”). Contrary to H2, persona systems disclosed at the first interaction no less often than systems without a persona (79.7% vs 73.3%; H2 not supported). Many portal personas, such as “Holly”, “Hutti” and “NÖKI”, sit inside platforms whose every deployment we observed carried an AI notice.
At least seven systems presented a real or historical person or a photorealistic human: the poet Andrej Sládkovič (Krupina), the artist Tetjus Tügel (Museum Hein Meyer), Nero (Parco archeologico del Colosseo), Fouquet and Wilhelmine, and the photorealistic “Maria” avatars of Lamego and Matosinhos. Whether these images were AI-generated could not be established from the interface, so they are reported as personas, not as deep fakes under Article 50(4).

4.5. Robustness of the Owner Difference

The museum–portal difference was not produced by any single platform (Table 9). Removing the largest portal platform (neuraflow) left 25/41 vs 38/43 (p = .004); removing both portal platforms left 25/41 vs 34/39 (p = .007); removing Chatbase, Ask Mona or the Kunstmeile systems left p ≤ .011 in each case. Among the 43 museum and portal systems whose platform could not be identified, the difference pointed the same way but was not significant (11/16 vs 24/27, p = .11).
Country was a partial confounder. Within Germany, almost all systems disclosed at launch regardless of owner (museum 7/8, portal 36/37; p = .33). Outside Germany the difference remained large but was at the margin of significance (18/33 vs 16/20; p = .055). Adjusting for Germany attenuated but did not remove the association (Mantel–Haenszel OR 0.28, 95% CI 0.09–0.94; homogeneity p = .79; logistic regression OR 0.28, 0.08–0.92). The owner difference is therefore robust to the composition of platforms, but part of the crude gap reflects the concentration of portal systems in Germany.

5. Discussion

5.1. Disclosure Follows the Design Chain

Three months after Article 50(1) became applicable, about three quarters of the chatbots on EU museum pages told visitors at the start that they were AI, and a similar share said so when asked in both sessions; 61 of 104 met both criteria. Observable disclosure is therefore widespread but not uniform.
The distribution is more informative than the average. Disclosure was near-universal where a platform builds it into the default configuration. The municipal portal chatbots we observed ask for consent to “activate the AI chatbot”, label each answer as AI-generated and keep a persistent footer, and a museum page embedded in such a portal inherits that design. Among museum-run systems, four in ten did not disclose at the first interaction. General-purpose chatbot builders and museum vendors appear to leave the launch screen to the deploying institution, and that screen typically carries a greeting, a persona name and a vendor credit, none of which the guidelines accept as sufficient on its own (para. 38).
In the terms of Section 2.1, this pattern is what one would expect if platform providers act as de facto intermediaries: where disclosure is designed in, it is reproduced across every deployment; where it is left to configuration, it depends on the deployer. The evidence for this reading is consistent but not causal. The owner difference survived the removal of every major platform and adjustment for country, yet it was attenuated by that adjustment, it was not significant within Germany or among systems with unidentified platforms, and we did not observe vendors’ default settings directly. Ownership, platform, country, organisational capacity and procurement are bundled in this population. What the data do establish is that, among the systems a visitor can actually find, the probability of a launch notice differs sharply by the type of body that supplies the chatbot.

5.2. A Notice Is Not an Answer, and an Answer Is Not a Notice

The second finding is that the two criteria fail in different systems. H3 assumed a nested pattern, in which systems that disclose up front would also disclose when asked. Instead, the failures fell on different systems and clustered by platform.
Systems built on general-purpose and museum-specific language-model products answered the direct question well but often lacked a notice at launch. Systems that displayed an explicit notice, often a platform-wide label, frequently failed the direct question in at least one session: some refused it as off-topic, others returned “no information found” or stayed in a persona. We did not code the back-end technology, so the suggestion that answer quality reflects the underlying architecture is an interpretation the data are consistent with but do not test.
The distinction maps onto formal and effective transparency. A launch notice is formal transparency: it satisfies a visible standard whether or not anyone reads it. An accurate answer to a direct question is closer to effective transparency, because it reaches the visitor at the moment of doubt. A system with a notice but no truthful answer meets the launch criterion while failing a user who actually asks; a system with a truthful answer but no notice leaves uninformed the many visitors who never ask. As with privacy notices [19], a check of the visible notice alone would overstate how well users are informed.
For auditors and authorities, the implication is methodological: a check of the launch screen and a direct question measure different properties, and neither predicts the other. Answers were also unstable for about one system in nine. A system that says it is an AI in one session and deflects in the next is unlikely to satisfy a requirement that applies “whenever” users ask, and single-session audits would misclassify such systems in either direction. For providers, the implication is practical: a fixed, rule-based answer to identity questions can sit alongside any back end.

5.3. Personas and Real Persons

Personas were the norm, but they did not predict lower disclosure (H2). The concern raised by the guidelines is not personas as such but human-like representations used instead of a disclosure (para. 38), and in our population most personas sat alongside a platform notice. The weaker cases were personas that stay in role. A historical monk who answers in character that the question is “not my world to dwell on”, or a photorealistic “Maria” who refuses the question, uses the persona in the way the guidelines warn against. Personas of real historical persons raise a further question under Article 50(4), which covers realistic replicas of deceased persons (para. 113). Whether a given portrait or voice is AI-generated cannot be seen from the interface; answering that question requires the deployer’s documentation, a natural target for market surveillance requests.

5.4. Implications for Policy and Practice

Deployers: sector guidance. Museum associations could translate paragraphs 36–40 of the guidelines into a short checklist: an explicit statement at launch, a persistent label, and a scripted answer to identity questions that overrides any persona.
Deployers: procurement. Procurement clauses can require vendors to supply these features by default. The portal platforms in our data show that this is feasible.
Regulators. National market surveillance authorities seeking proportionate enforcement could start with the general-purpose builders and museum vendors whose launch screens, in our data, often lacked a notice. These products serve many institutions, so one change of default would reach many visitors. Audits should test both criteria, and the identity question more than once.
The Commission. The guidelines could say explicitly that disclosure on request is not met by a fallback reply or a refusal. Our coders initially disagreed on exactly this point.

5.5. Limitations

  • Population. The frame covers chatbots discoverable from Wikidata-listed museum homepages, so recall cannot be estimated. Germany, France and Italy dominate partly for this reason, and systems on inner pages or in apps are under-represented. Results describe this population, not all museum chatbots in the EU.
  • Confounding. Owner type is correlated with country and platform. The robustness analyses address the largest platforms and Germany but cannot separate ownership from procurement or organisational capacity.
  • Criteria, not compliance. O1 and O3 are necessary conditions. Clarity, distinguishability and accessibility (O2) are deferred to the second wave, and disclosure on request rests on the guidelines’ interpretation. No conclusion about legal compliance or liability is drawn.
  • Reliability. Agreement on disclosure on request in session 2 was below the registered threshold as coded, and agreement on persona was moderate in the random subsample. Agreement after the rule clarification was not re-estimated on a fresh blind sample, so the post-clarification κ is not an independent estimate. The main conclusions do not depend on the session-2 code (sensitivity C), and H2 was not supported under any coding.
  • Probing environment. About half the systems were probed by hand in a different browser. The disclosure text does not depend on the environment, but the archived evidence does.
  • Session independence. Some vendors restored earlier conversations, so the second session was not fully independent for these units.
  • Scope and language. In the manual pass (51 coded systems), the repeated collection question (P2) was not run, so classification as generative rests on the provider’s own AI statements and on the free-text replies to the identity questions; some rule-based systems may have been misclassified as in scope. A few systems followed the browser’s locale instead of the site’s language.
  • AI coding. The probes and first coding were carried out by an AI model. Half the units were coded blind by a human coder, whose codes prevailed where the coders differed.

6. Conclusions

This audit asked whether chatbots on European museum pages tell visitors that they are artificial intelligence, three months after Article 50 of the AI Act made this a legal requirement. Most do: about three in four showed a notice at or before the first interaction, and a similar share said they were AI when asked in two separate sessions. No system claimed to be human. The average, however, hides two findings that matter more for policy.
The first is that observable disclosure depends on who supplies the chatbot. Systems delivered through municipal and tourism portals showed a notice in nine cases out of ten, typically through a gate, label or footer that appears on every deployment of the platform. Museum-run systems did so in six cases out of ten; the products they use appear to leave the launch screen to the institution. The difference was not driven by any single platform and persisted after adjustment for country, although part of it reflects the concentration of portal systems in Germany. We read it as evidence that transparency under Article 50 is produced along a chain of regulator, provider and deployer, and that it is most reliable where it is designed into product defaults rather than left to configuration by individual institutions.
The second is that a notice and an answer are different things. Nearly one system in five showed a notice but did not disclose its nature when asked in both sessions, and a similar number answered truthfully but showed no notice. One in nine answered inconsistently across sessions. A launch notice delivers formal transparency; a truthful answer reaches the visitor who actually wonders. Neither predicts the other, so an audit or inspection that checks only one of them will misjudge how well users are informed.
These findings point to concrete levers: sector checklists that combine a launch statement with a scripted identity answer, procurement clauses that require both by default, and enforcement that starts with widely used products whose deployments often lacked a launch notice. The registered second wave will add clarity and accessibility of disclosure. Studies with visitors are needed to establish whether formal disclosure becomes effective transparency, that is, whether visitors notice, understand and act on it.

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org. File S1: Codebook v1.0; File S2: Deviations from the pre-registration; File S3: Probe wording in all interface languages; File S4: Unit-level Wave 1 codes (CSV), data dictionary and verbatim session logs. The pre-registration and a transparent-changes note are available on OSF at https://doi.org/10.17605/OSF.IO/JQY6Z.

Funding

This research received no external funding.

Institutional Review Board Statement

Ethical review and approval were waived for this study because it involved no human participants and no personal data. It consisted of short, non-adversarial interactions with automated systems on public websites, without login or payment.

Data Availability Statement

The codebook, probe wording, sampling frame, unit-level codes, verbatim session logs, second-coding log and analysis scripts are provided as Supplementary Materials and will be deposited in an open repository (Zenodo) with a persistent identifier upon publication.

Acknowledgments

During the preparation of this manuscript, the author used Claude (Anthropic) to assist with drafting and editing the text. As described in Section 3, the same model was also used as a research instrument for frame pre-screening scripts, browser automation, sending the probe messages, first coding and statistical scripts. The author designed the study, took all coding decisions, carried out the blind second coding, verified the analyses, reviewed and edited the output, and takes full responsibility for the content of this publication.

Conflicts of Interest

The author declares no conflicts of interest. The author is a member of the Editorial Board of Future Collections, Libraries, Archives, and Museums (FCLAM) and was not involved in the editorial handling of this manuscript.

Abbreviations

The following abbreviations are used in this manuscript:
AI Artificial intelligence
CI Confidence interval
EU European Union
GL Commission guidelines on Article 50 (paragraph)
MH Mantel–Haenszel
OR Odds ratio
OSF Open Science Framework
RD Risk difference

References

  1. Kist, C. List of Conversational Agents in Museums (1996–2023) [Data Set]; University of Strathclyde: Glasgow, UK, 2024. [Google Scholar] [CrossRef]
  2. Trichopoulos, G.; Konstantakis, M.; Alexandridis, G.; Caridakis, G. Large Language Models as Recommendation Systems in Museums. Electronics 2023, 12, 3829. [Google Scholar] [CrossRef]
  3. Trichopoulos, G.; Konstantakis, M.; Caridakis, G.; Katifori, A.; Koukouli, M. Crafting a Museum Guide Using ChatGPT4. Big Data Cogn. Comput. 2023, 7, 148. [Google Scholar] [CrossRef]
  4. Tue, T.; Pohl, H.; Merritt, T. Chatty Art Labels: Evaluation of a Conversational Museum Guide. Digit. Appl. Archaeol. Cult. Herit. 2026, 42, e00585. [Google Scholar] [CrossRef]
  5. Chen, B.; Wen, R.; Tan, S.; Li, Y. Exploring User Preferences for Museum Guides: The Role of Chatbots in Shaping Interactive Experiences. In Proceedings of the Extended Abstracts of the 2025 CHI Conference on Human Factors in Computing Systems; ACM: New York, NY, USA, 2025; p. 256. [Google Scholar] [CrossRef]
  6. Kist, C.; Moshfeghi, Y. To Bot or Not? Conversational Agents and Authentic Heritage Experiences. In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems; ACM: New York, NY, USA, 2026; p. 674. [Google Scholar] [CrossRef]
  7. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act). Off. J. Eur. Union 2024, L 2024/1689.
  8. Regulation (EU) 2026/1744 of the European Parliament and of the Council (Digital Omnibus on AI). Off. J. Eur. Union 2026, 24 July 2026.
  9. European Commission. Guidelines on the Implementation of the Transparency Obligations for Certain AI Systems under Article 50 of Regulation (EU) 2024/1689 (the “AI Act”); Annex to C(2026) 5054 final; European Commission: Brussels, Belgium, 2026; Available online: https://digital-strategy.ec.europa.eu/en/library/guidelines-transparency-obligations-providers-and-deployers-ai-systems.
  10. Luo, X.; Tong, S.; Fang, Z.; Qu, Z. Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer Purchases. Mark. Sci. 2019, 38, 937–947. [Google Scholar] [CrossRef]
  11. Ishowo-Oloko, F.; Bonnefon, J.-F.; Soroye, Z.; Crandall, J.W.; Rahwan, I.; Rahwan, T. Behavioural Evidence for a Transparency–Efficiency Tradeoff in Human–Machine Cooperation. Nat. Mach. Intell. 2019, 1, 517–521. [Google Scholar] [CrossRef]
  12. van der Goot, M.J.; Koubayová, N.; van Reijmersdal, E.A. Understanding Users’ Responses to Disclosed vs. Undisclosed Customer Service Chatbots: A Mixed Methods Study. AI Soc. 2024, 39, 2947–2960. [Google Scholar] [CrossRef]
  13. Vesala, J.; Rauti, S.; Rajapaksha, S.; Hjerppe, K.; Heino, T. Examining Transparency Requirements for AI-Based Chatbots and Websites. In Proceedings of the 2026 International Workshop on Responsible AI Engineering (RAIE ’26), 2026; ACM: New York, NY, USA. [Google Scholar] [CrossRef]
  14. Abbott, K.W.; Levi-Faur, D.; Snidal, D. Theorizing Regulatory Intermediaries: The RIT Model. Ann. Am. Acad. Political Soc. Sci. 2017, 670, 14–35. [Google Scholar] [CrossRef]
  15. Almada, M. Regulation by Design and the Governance of Technological Futures. Eur. J. Risk Regul. 2023, 14, 697–709. [Google Scholar] [CrossRef]
  16. Yeung, K. Hypernudge’: Big Data as a Mode of Regulation by Design. Inf. Commun. Soc. 2017, 20, 118–136. [Google Scholar] [CrossRef]
  17. Mökander, J.; Schuett, J.; Kirk, H.R.; Floridi, L. Auditing Large Language Models: A Three-Layered Approach. AI Ethics 2024, 4, 1085–1115. [Google Scholar] [CrossRef]
  18. Mökander, J.; Morley, J.; Taddeo, M.; Floridi, L. Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. Sci. Eng. Ethics 2021, 27, 44. [Google Scholar] [CrossRef]
  19. Walsh, S.; Piwek, L.; Ramokapane, K.M. Three Systemic Problems with the Privacy Notice Ecosystem. Policy Internet 2026, 18, e70042. [Google Scholar] [CrossRef]
  20. Altay, S.; Gilardi, F. People Are Skeptical of Headlines Labeled as AI-Generated, Even If True or Human-Made, Because They Assume Full AI Automation. PNAS Nexus 2024, 3, pgae403. [Google Scholar] [CrossRef]
  21. Matte, C.; Bielova, N.; Santos, C. Do Cookie Banners Respect My Choice? Measuring Legal Compliance of Banners from IAB Europe’s Transparency and Consent Framework. In Proceedings of the 2020 IEEE Symposium on Security and Privacy; IEEE: San Francisco, CA, USA, 2020; pp. 791–809. [Google Scholar] [CrossRef]
  22. Nouwens, M.; Liccardi, I.; Veale, M.; Karger, D.; Kagal, L. Dark Patterns after the GDPR: Scraping Consent Pop-Ups and Demonstrating Their Influence. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems; ACM: New York, NY, USA, 2020; pp. 1–13. [Google Scholar] [CrossRef]
  23. Wang, Y.; Johnston, A.; Sadokierski, Z.; Stephens, R.; Ahyong, S.T. Conversational AI-Enhanced Exploration System to Query Large-Scale Digitised Collections of Natural History Museums. arXiv 2026, arXiv:2603.10285. [Google Scholar]
  24. Konstantakis, M.; Caridakis, G. Adding Culture to UX: UX Research Methodologies and Applications in Cultural Heritage. J. Comput. Cult. Herit. 2020, 13, 4. [Google Scholar] [CrossRef]
Figure 1. AI interaction disclosure as a distributed governance chain (conceptual). The study observes the visitor-facing outcome and the deployment context, not provider or deployer decisions directly, and not whether visitors understand the disclosure.
Figure 1. AI interaction disclosure as a distributed governance chain (conceptual). The study observes the visitor-facing outcome and the deployment context, not provider or deployer decisions directly, and not whether visitors understand the disclosure.
Preprints 244762 g001
Figure 2. Study pipeline, from frame construction and pre-registration to probing, evidence, coding, outcomes and analysis.
Figure 2. Study pipeline, from frame construction and pre-registration to probing, evidence, coding, outcomes and analysis.
Preprints 244762 g002
Figure 3. Construction of the sampling frame and flow of units through Wave 1 (2–7 November 2026).
Figure 3. Construction of the sampling frame and flow of units through Wave 1 (2–7 November 2026).
Preprints 244762 g003
Figure 4. Disclosure at or before the first interaction (O1) and disclosure on request in both sessions (O3), overall and by owner stratum. Bars show Wilson 95% confidence intervals; counts are given on the right.
Figure 4. Disclosure at or before the first interaction (O1) and disclosure on request in both sessions (O3), overall and by owner stratum. Bars show Wilson 95% confidence intervals; counts are given on the right.
Preprints 244762 g004
Figure 5. Combinations of the two operational disclosure criteria by platform. “O1 only”: an AI notice at the start but no disclosure when asked. “O3 only”: disclosure when asked but no notice at the start. Platforms with at least three systems are shown separately.
Figure 5. Combinations of the two operational disclosure criteria by platform. “O1 only”: an AI notice at the start but no disclosure when asked. “O3 only”: disclosure when asked but no notice at the start. Platforms with at least three systems are shown separately.
Preprints 244762 g005
Table 1. Related studies and the gap addressed.
Table 1. Related studies and the gap addressed.
Study Setting Design Disclosure examined? Relation to this study
Luo et al.  [10] Sales chatbots Field experiment, > 6,000 customers Yes: disclosure cut purchases by about 80% Commercial incentive not to disclose
Ishowo-Oloko et al.  [11] Repeated cooperation game Behavioural experiment Yes: bots’ advantage vanished once disclosed Transparency–efficiency trade-off
van der Goot et al.  [12] Customer-service chatbots Experiment and interviews Yes: no significant effect Effects are context-dependent
Nouwens et al.  [22]; Matte et al.  [21] Consent banners on websites Web-scale compliance audits Consent notices, not AI Audit precedent; notice vs behaviour
Vesala et al.  [13] 12 chatbots and websites, Finnish public health services Compliance audit Yes: users not transparently informed Closest method
Kist  [1] Museums, 1996–2023 Literature-based list No Frame source
Chen et al.  [5]; Kist & Moshfeghi  [6] Museum chatbots; UK audiences Surveys No Personas and authenticity concerns
Tue et al.  [4]; Wang et al.  [23] AR art guide; natural history explorer System evaluations No (limitations disclaimer only) Typical evaluation focus
This study EU-27 museum-facing web chatbots Pre-registered cross-sectional audit Yes: Art. 50(1), (5); disclosure on request First sector audit under Art. 50
Table 2. From legal requirement to codes (codebook v1.0; GL = Commission guidelines, paragraph).
Table 2. From legal requirement to codes (codebook v1.0; GL = Commission guidelines, paragraph).
Requirement (source) Operational question Codes Outcome
AI system, intended to interact, directly (Art. 50(1); GL 30) Do free-text inputs yield generated, contextual replies without human mediation? B1–B4 Scope
Explicitly informed of the artificial nature, at the latest at first interaction (Art. 50(1), (5); GL 35, 143) Is an AI or automated-system statement perceivable at loading, in a gate before chatting, or when the chat is launched? C1, C3, C4, C13 O1 (criterion 1)
Insufficient techniques when used alone (GL 38) Persona name, generic “assistant”, human image, vendor credit, terms only? C6, C10 Not O1
Disclosure whenever asked (GL 40; interpretive) Does the system say it is AI when asked directly, in two sessions? C7, C7c, C7b O3 (criterion 2)
Clear, distinguishable, accessible (Art. 50(5); GL 142, 144) Noticeable, separate from content, exposed as text to assistive technology D1, D2, D4 O2 (deferred)
Table 3. Sensitivity analyses for H1–H3 (one-sided p).
Table 3. Sensitivity analyses for H1–H3 (one-sided p).
Analysis N O1 O3 H1 p H2 p H3 p
Primary 104 76.9% 77.9% .0004 .84 .50
B: one unit per bot 97 75.3% 77.3% .0008 .84 .44
C: O3 from session 1 only 104 76.9% 82.7% .0004 .84 .21
D: gate-only disclosure not counted 104 75.0% 77.9% .002 .72 .37
E: “-bot” name not counted 104 73.1% 77.9% .0001 .93 .27
F: registered frame strata (H1 only) 101 — — .002 — —
G: pilot systems excluded 99 77.8% 78.8% .001 .83 .50
Table 4. Probe protocol.
Table 4. Probe protocol.
Probe Action Purpose Main codes
P0 Load the page; record launcher and opening state Disclosure before interaction C1, C3
P1, P1g Open the chat; record any gate and the first system message Disclosure at launch (GL 143) C1, C4, C6, C13
P2 Ask the same museum question twice (automated pass) Generated vs templated replies B1
P3a, P3b “Real person or computer program?” and “Are you an AI?”, in two sessions Disclosure on request (GL 40); consistency C7, C7b, C7c
Table 5. Inter-coder agreement as coded, before rule clarification (Cohen’s κ ; raw agreement for the random subsample in brackets).
Table 5. Inter-coder agreement as coded, before rule clarification (Cohen’s κ ; raw agreement for the random subsample in brackets).
Code Full second-coded subset (n = 52) Random 20% subsample only (n = 22)
C1 disclosure at first interaction 0.94 1.00 (100%)
C4 wording of the disclosure 0.92 0.87 (95%)
C6 persona 0.80 0.60 (82%)
C7 session 1, disclosure on request 0.77 0.85 (95%)
C7 session 2, disclosure on request 0.66 0.58 (86%)
Table 6. In-scope systems by country, owner stratum and outcome.
Table 6. In-scope systems by country, owner stratum and outcome.
Country n Museum Portal or tourism Company museum O1 met O3 met
Germany 45 8 37 0 43 38
France 14 10 3 1 4 10
Italy 13 7 4 2 9 11
Austria 11 4 5 2 11 7
Czechia 4 3 1 0 2 2
Finland 3 0 2 1 3 3
Portugal 3 1 2 0 2 1
Denmark 2 2 0 0 1 2
Sweden 2 1 1 0 2 2
Belgium, Croatia, Greece, Lithuania, Netherlands, Slovakia, Spain (one each) 7 5 2 0 3 5
All 104 41 57 6 80 81
Table 7. Disclosure at the first interaction (O1) and on request in both sessions (O3), by owner stratum.
Table 7. Disclosure at the first interaction (O1) and on request in both sessions (O3), by owner stratum.
Stratum n O1 n (%) [95% CI] O3 n (%) [95% CI]
Portal or tourism 57 52 (91.2) [81.1, 96.2] 47 (82.5) [70.6, 90.2]
Museum 41 25 (61.0) [45.7, 74.3] 29 (70.7) [55.5, 82.4]
Company museum 6 3 (50.0) [18.8, 81.2] 5 (83.3) [43.6, 97.0]
All 104 80 (76.9) [68.0, 84.0] 81 (77.9) [69.0, 84.8]
Table 8. Hypothesis tests (one-sided; Holm-adjusted across H1–H3).
Table 8. Hypothesis tests (one-sided; Holm-adjusted across H1–H3).
Hypothesis Comparison Effect p Holm p
H1: O1 museum < portal 25/41 vs 52/57 RD −30.3 pp [−46.3, −13.5] .0004 .001
H2: O1 persona < no persona 47/59 vs 33/45 RD +6.3 pp [−9.7, +22.9] .84 1.00
H3: O3 > O1 discordant 20 vs 19 — .50 1.00
Table 9. Robustness of H1 (O1, museum vs portal; one-sided Fisher p unless stated).
Table 9. Robustness of H1 (O1, museum vs portal; one-sided Fisher p unless stated).
Analysis Museum Portal p
Primary 25/41 52/57 .0004
Without neuraflow 25/41 38/43 .004
Without regional portal assistant (headwai) 25/41 48/53 .0007
Without both portal platforms 25/41 34/39 .007
Without Chatbase 22/35 48/53 .002
Without Ask Mona 23/33 52/57 .010
Without Kunstmeile Krems 22/38 52/57 .0002
Unidentified platform only 11/16 24/27 .11
Germany only 7/8 36/37 .33
Excluding Germany 18/33 16/20 .055
Adjusted for Germany (MH OR, two-sided) OR 0.28 [0.09, 0.94] .033
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.