Children of AI: A Protocol for Managing the Born-Digital Ephemera Spawned by ChatGPT

1. Introduction

In recent months there has been wide-spread public attention regarding the use of artificial intelligence (AI) in various fields. The public releases of the image generator DALL-E and of the generative AI language model ChatGPT (Chat Generative Pre-trained Transformer) in early 2022 caught the public imagination. Since then, free-ranging debate emerged regarding the present and potential future abilities of generative AI, the dangers its may pose and the ethics of its usage. ChatGPT is type of deep learning model that uses transformer architecture to generate coherent and contextually relevant, human-like responses based on the input it receives [1].

Since its initial release in 2018, ChatGPT has undergone several revisions, mainly focusing on increased capabilities in providing longer segments of coherent text and contextual answering of questions including the addition of human preferences and feedback. The release of ChatGPT 2.0 in September 2019 relied on a training data set with 1.5 billion parameters, while the ChatGPT 3 (June 2020) was trained (by human trainers) on 175 billion parameters. ChatGPT 3.5 was released to the general public to encourage experimentation [2], with a temporal cut for the addition of its training data in September 2021.

ChatGPT has been shown to be capable of producing poetry [3], short stories and plays [4,5,6], English essays [7], as well as writing lines of code [8]. A growing number of studies has been looking at the effects of generative AI on education and academia, examining the level of knowledge and capabilities of ChatGPT as reflected in its responses to several field of academic endeavor, such as agriculture [9], archaeology [10], chemistry [11], computer programming [12], cultural heritage management [13], diabetes education [14], digital forensics [15], medicine [16,17,18,19,20,21,22,23], medical education [24], nursing education [25], and remote sensing [10]. ChatGPT is the typical double-edged sword presented by many new technologies [26]: both useful a potential tool to enhance student learning (e.g., [27,28,29]) given its ability to aid students in assignment writing and the associated potential misconduct (e.g., [30,31,32]).

2. The Problem

A primary mandate of academic publishing is ethical academic conduct. To ensure the integrity, transparency, and reproducibility of published research, the Committee on Publication Ethics (COPE) has issued a set of guidelines on the deposition and management of data and datasets that were used to explain and substantiate findings that are reported in academic publications [33]. In brief, this entails the deposition of research data in a common readable format in a curated, public, institutional or governmental data depository; the publication of the data and their collection methodology as a stand-alone data publication; or the appendage of the data as supplementary material to the article hosted on the publisher’s servers. This mandate is followed by and large, with some disciplines being more compliant than others (e.g. medical research).

At this point in time, academic research into ChatGPT, its abilities and limitations is expanding at rapid rate across most disciplines. The question arises how the research data associated with the publications are being managed.

The nature of ChatGPT as a generative AI tool means that each response to a given task will be different. While responses may be structurally and conceptually similar [34], they are not identical. Thus, it is not possible to recreate an identical or near identical response. Furthermore, all conversations with ChatGPT are deleted after 30 days to maintain server space [35]. Irrespective of this, all conversations with ChatGPT are virtual artefacts (sensu [36]) which, even when will eventually disappear due server upgrades or data warehouse restructuring. Consequently, the original conversations are equivalent to and should be treated like an experiment’s raw data and should be archived. At present, academic papers that have been written in relation to ChatGPT have taken five different approaches:

• articles that include the entire conversation in the paper [9,10,11,21,24,34]
• articles which quote extracts of the conversation in the text and provide the full text of the conversation as a supplementary document [13] or an appendix [37]
• articles which quote extracts of the conversation in the text but does not provide access to the full text [15,22,23,25,29,31,38,39]
• articles which discuss but not quote specific conversation(s) and do not provide access to the full text [30]; and
• articles which discuss ChatGPT but do not refer to specific conversations but discuss the topic at a more abstract level [26,37].

Setting aside the first group, where the conversation makes up the core of the paper, and the second group where the full text is supplied as an appendix or a supplementary file, the other three groups do not allow readers to understand the full context of the conversation and cannot independently assess the validity of the author’s interpretation of the interaction.

3. Towards a Solution

3.1. Functional Considerations

While the conversations with ChatGPT have great similarity with formal interviews in anthropological, and ethnographical and sociological settings [40], they differ in on key respect. Human-to-human interviews are conducted in a linear fashion, with one question following on from, and building on, an answer. While, in theory, a human respondent could be asked to re-answer the question, this normally occurs with transitioning phrases [41] and a concomitant second-guessing by the respondent trying to taking cues from the interviewer why the previous answer was insufficient (else it would not have been asked in exactly the same way). ChatGPT, on the other hand, allows the human participant to request ChatGPT to answer the question again, generating a new response to the question. Thus any archiving of a conversation with ChatGPT needs to archive all “regenerations” of a question if they were conducted, while the paper needs to identify which instance of regeneration was being used or cited.

ChatGPT as a service is not static but is both being continually upgraded in terms of functionality and server performance and due to its ability to ‘learn.’ The latter is reinforced through user feedback which is solicited when a user tasks ChatGPT to regenerate a response and ChatGPT delivers the second version (Figure 1) and by OpenAI staff monitoring selected conversations (“Conversations may be reviewed by our Al trainers to improve our systems”).

Thus it can be posited that a ChatGPT is time-dependent, and that therefore the date and time of conversation should also be recorded, akin to the practice of stating the access date of web pages in standard referencing.

3.2. Ethical Considerations

Standard ethnographic research practice mandates that interviews are based on a mutual understanding of trust that the conversation is confidential and any interpretation of the conversation is carried out with expressed and informed consent, even though power dynamics and their changing nature need to be considered [42]. Irrespective, notebooks and interview transcripts were often deemed personal data, ‘owned’ by the respective researcher.

In recent years research ethics to mitigate against falsified research findings have led to the mandate to archive and make accessible the original or raw research data. In the space of qualitative research, this posed the ethical conundrum to allow access while at the same time maintaining the confidentiality of information provided in the interviews [43,44,45,46]. While this can be overcome by anonymizing or de-identifying the respondents, contextual information in the interviews may well allow for a re-identification of the informants [47,48,49]

In the current understanding, works created by generative AI do not accrue copyright for the AI system [50] as they fail to meet the human authorship requirement, but can generate copyright for the human actor in the interaction if the latter has substantive guiding involvement [51]. In the same vein, an ‘interview’ with ChatGPT differs from sociological or ethnographic interviews because generative AI is, at least at this point in time, not a sentient entity and thus cannot provide informed consent. It follows that, at lkeast at this point of legal understanding, ‘conversations’ with ChatGPT can be archived.

3.3. Protocol

We propose the following five-step process for ChatGPT data collection and archiving:

Step 1) record the metadata, comprised of version and version date, which can be found at the bottom of the interface (Figure 2), as well as the date and time the conversation occurred, using GMT as the standard.

Step 2) carry of the conversation as required.

Step 3) add the end time of the conversation to the metadata entry.

Step 4) Copy the text of the conversation into a text editor or word processor and save the file(s), making sure that all iterations of task generation (if any) are captured and identified as such (e.g. Regeneration 1/3, 2/3 etc.).

Step 5) Generate a complete data document that contains the metadata and text of each conversation.

Step 6) Submit the data document to a data repository of append it to the publication as a supplementary file.

4. Conclusions

Over the past decade, the development of generative artificial intelligence systems has accelerated dramatically, resulting in the recent public release of the generative AI language model ChatGPT. ChatGPT has captured the public imagination with widespread experimentation by academia and the general public alike. Numerous academic disciplines experimented with the capabilities of ChatGPT in relation to their research directions, examining its ability to provide accurate responses. The number of publications on the capabilities of ChatGPT and the practical and ethical implications of the use and abuse generative AI has been growing exponentially.

This unprecedented growth in scholarship related to generative AI, in particular ChatGPT, occurs in a large unregulated space, wherein in most cases the raw data, that is the text of the original ‘conversations,’ are not being made available to the audience of the papers. In consequence, they cannot be drawn on to assess the veracity of the arguments made in the publications and the conclusions drawn therefrom. This paper has provided a protocol for the documentation and archiving of these raw data.