The Monumental Impact of OpenAI´s ChatGPT on Scientific Research: Enabling Scholars to Engage in High-Value Intellectual Activities of Systematic Literature Reviews

1. Introduction

Scientific research is characterized by the enduring stability and gradual evolution of research methods, methodologies, and techniques [1]. Periodically, specialized software programs tailored for computer-assisted qualitative [2] or quantitative [3] data analysis emerge, significantly enhancing the capabilities of academic and scientific institutions. Departing from conventional paradigms, OpenAI's ChatGPT has assumed a pivotal role in reshaping the scholarly paradigm [4,5,6]. This article aims to explore the transformative potential of ChatGPT within academic domains, anticipating that future academics will distinguish themselves by adapting to and harnessing Artificial Intelligence (AI) capabilities to streamline non-value tasks.

Amidst the transformative changes in academia facilitated by the integration of AI, a discernible lacuna persists in the comprehensive understanding of the most efficacious methods through which scholars can harness ChatGPT to enrich and streamline their research endeavors. To underscore this assertion, a preliminary exploration of Scopus substantiates the exponential surge in ChatGPT utilization, reflecting the scholarly community's earnest quest to discern the optimal practices for employing this technology within the academic sphere. Consequently, this research aims to address the critical gap by elaborating on the dynamics between academics and AI, particularly within the context of systematic literature reviews, renowned for their inherently standardized and analytical characteristics [7].

This investigation will expressly concentrate on the convergence of high-value intellectual activities inherent to scientific research that demand human engagement and the low-value tasks amenable to machine execution. The primary objective of this inquiry is framed within a well-defined research question: How can academics strategically leverage the capabilities of ChatGPT to optimize their research efforts, with a specific emphasis on enhancing the efficiency of systematic literature reviews? This question serves as the focal point for unveiling how scholars can strategically deploy ChatGPTs to amplify the effectiveness and expediency of their scholarly pursuits.

The preliminary findings substantiate the prevailing insights in scholarly literature, affirming the proficiency of AI in executing mechanical and analytical tasks [8]. Concurrently, these results reiterate the specialized role of academics, uniquely suited for engaging in the empirical and empathetic dimensions of the investigative process, especially during data collection methodologies such as interviews and focus groups. This dichotomy underscores the complementarity between AI's efficiency in certain domains and the nuanced capacities of academics in the multifaceted aspects of empirical research.

Section 2 explores the contemporary literature, relevant to the thematic focus of this article. Section 3 emphasizes the materials, methods, and methodologies employed throughout this article. Section 4 identifies high- and low-value intellectual work in SLRs, it explores the potential automation of low-value activities through the integration of Python with ChatGPT and concludes by examining the feasibility of fully automating systematic and bibliometric reviews in scientific research. Section 5 presents the conclusions, offering reflections on the theoretical and managerial contributions, acknowledging limitations, and proposing suggestions for future research endeavors.

2. Literature Review

In the domain of AI advancement, its far-reaching impact is notably conspicuous in academic and scientific service sectors [9]. This influence is particularly pronounced in specialized English proofreading tailored to distinct scientific domains [10], as well as its substantial contributions to academic research and scientific discovery. Consequently, articles on AI's pivotal role in reshaping scientific-academic services and facilitating breakthroughs in scientific discovery have gained prominence [4,9]. Importantly, the prospect of academics engaging in higher-value intellectual activities is steadily becoming a reality.

Understanding the ramifications of AI's potential in scientific research requires exploring the phenomena, like the theory of job displacement [11], especially in fields linked to mechanical and analytical intelligence [8]. With the progression of AI, the diminishing significance of analytical skills becomes evident, ushering in the prominence of intuitive and empathetic skills. In this context, empathy plays a pivotal role in empirical scientific studies. Consider the instance of conducting interviews, where possessing empathy is essential for effective communication and comprehending the interviewee's intended message. Beyond verbal interactions, there are facets tied to intuitive understanding, often intertwined with the field of non-verbal communication and other-oriented, intersubjective modes of comprehension. The question of how extensively these skills can be harnessed in the arena of scientific discovery will remain open to humans. The displacement of jobs by AI is discernible at the task level, initiating tasks classified as "lower" in terms of intelligence. The narrative foresees a future where AI capabilities extend to tasks involving intuition and empathy, opening innovative prospects for collaboration between humans and AI and posing a fundamental challenge to conventional structures. As AI advances to higher levels of intelligence, the role of human intelligence experiences diminution. This strategic approach is geared towards optimizing the benefits of AI while ensuring its seamless integration with human intelligence, thereby averting the exclusion of humans from the scientific discovery process.

Given the current complexity and premature nature of scientifically analyzing the relationship between AI and humans in intuitive and empathetic tasks, our focus will shift towards the application of large-scale language models in tasks classified as "lower" in terms of intelligence in scientific research. To this end, we will concentrate on exploring the use of AI in SLR, a domain celebrated for its inherently standardized and analytical characteristics. Among the contemporary tools harnessing large-scale language models is ChatGPT, a pre-trained AI model for natural language conversations, which incorporates advanced techniques from Natural Language Processing (NLP), supervised learning, and reinforcement learning [12]. Thus, ChatGPT emerges as a potentially valuable tool in the domain of scientific research as exemplified by Adesso [12], who underscores the significance of effectively integrating AI capabilities with human intelligence. This integration, according to Adesso [12], offers a glimpse into the promising future of human-AI collaboration in scientific exploration.

3. Methodology

This article seeks to address a gap in the existing scholarly literature by exploring the diverse applications of Python in conjunction with ChatGPT for the automation of non-value-added tasks associated with SLRs. The primary aim is to mechanize routine activities, allowing researchers to concentrate on higher-value tasks. Our approach is twofold:

Firstly, we conducted a series of semi-structured interviews with experts actively engaged in researching and regularly publishing SLRs. Our objective was to compile valuable and comprehensive insights that would shed light on the key distinctions between high- and low-value intellectual contributions within the domain of SLRs. These interviews, held in June 2023 at a scientific conference organized by this article's author(s), provided a platform for discussions with 12 renowned SLR experts over three days. Our selection of experts deliberately encompassed a diverse geographical representation, including participants from Canada, Brazil, Germany, Italy, Portugal, Spain, and the United States of America. Each interview session spanned a duration of 45 to 60 minutes and was dedicated to exploring and evaluating intellectual endeavors within the SLRs. The insights derived from these interviews constitute a foundational comprehension of the intellectual contributions within the context of SLRs.

In addition to conducting interviews, our research methodology encompassed the integration of additional data collection sources. This encompassed direct observation [14] wherein a field diary served as a tool for methodically documenting informal discussions [15] following the presentation of conference papers that contributed to SLRs. Furthermore, we organized researchers into concise focus groups [16], fostering deliberate discussions centered on this research subject. This deliberate approach not only facilitated the generation of innovative insights but also validated findings derived from the primary data collection source. The interview data underwent a dual-stage coding process. Initially, a manual approach was employed, involving the verbatim transcription of recorded interviews [17]. During this phase, we tallied instances of repeated words, cross-referencing the transcriptions with field notes (i.e., direct observation and focus groups). The analysis relied on a computer-assisted data analysis package – NVivo 12 [18]. NVivo 12 was utilized for an integrated audio-textual analysis of interviews and field notes, resulting in a comprehensive document spanning 924 pages. Leveraging qualitative data analysis software like NVivo facilitated the management of the substantial data volume through an interactive process of coding and categorizing. This approach allowed for the identification of consistent patterns and relationships between variables, serving the purpose of data reduction and interpretation. The analytical process unfolded in four stages [19]: firstly, the establishment of a hierarchy of categories and subcategories; secondly, the association of interview excerpts with the respective categories and subcategories, with the incorporation of new ones as needed; thirdly, the identification of emerging patterns and ideas; and fourthly, the revision of previous categories, incorporating adjustments until redundancies and contradictions were resolved, rendering the results easily interpretable. Consistent with the practices in other studies [20], we employed diverse sources of evidence, encompassing interviews, direct observation, and focus group evidence to bolster construct validity. Moreover, participants were actively engaged in the process, reviewing all transcriptions, and providing additional data through follow-up emails, thereby contributing to the improvement of the reliability of our interpretations.

Secondly, we introduce a conceptual framework that delineates the Python ecosystem, facilitating the execution of low-value tasks through ChatGPT. This framework is designed around libraries and frameworks essential for continuous data manipulation, advanced analytics, and optimization techniques [21].

To construct this framework, our research method involved a conventional approach, drawing insights from a diverse array of sources. These sources encompassed peer-reviewed scientific articles, conference proceedings, and the official Python website [22]. The intentional emphasis on peer-reviewed journal articles stems from their capacity to provide substantive and dependable insights within specific disciplinary domains [21,23]. This methodological choice ensures a comprehensive understanding of the applications of Python programming intertwined with ChatGPT.

It is essential to underscore that the incorporation of a versatile, high-level programming language such as Python [24] for enhancing SLRs remains relatively underexplored within academic discourse. However, the selection of Python was motivated by the dynamic attributes inherent in software development within this language, marked by its rapid pace and the proliferation of open-source software originating from diverse entities, including academic institutions, industrial entities, start-ups, and expansive open-source communities. Hence, our research presents a foundational exploration that elucidates and delineates the as-yet-undiscovered potential of Python within the domain of SLRs. Table 1 below presents a summary of this section.

Within the literature review, we discover that tools such as Rayyan.com have been instrumental in accelerating and simplifying SLRs through web-based functionalities [25]. Rayyan.com [26] stands out for its efficiency, offering a platform that significantly aids in managing large volumes of academic articles while fostering collaborative efforts among research team members. While existing tools have demonstrated commendable capabilities, our proposal advocates for the creation of tailored software designed to meet the specific needs of researchers. This envisioned software aims to elevate the depth of analysis by seamlessly integrating the advanced capabilities of ChatGPT [27]. By doing so, it aspires to enhance the overall research process and scholarly inquiry, providing researchers with a more refined and efficient tool for their endeavors.

4. Findings

4.1. High- and Low-Value Intellectual Work in SLRs

SLRs are thorough investigations involving the systematic collection, analysis, and synthesis of existing research on a particular topic. Like any established methodology, these reviews encompass intellectual tasks of varying significance. Within the domain of SLRs, intellectual efforts can be classified as either high- or low-value activities, depending on their impact on the overall quality and rigor of the review process. Activities with low intellectual value typically involve routine and standardized tasks, while those with high intellectual value are associated with critical thinking, creativity, and complex undertakings. To facilitate the analysis of defining high- and low-value intellectual work within SLRs, we followed the methodology outlined by Xiao and Watson [28]. This article offered crucial guidance for initiating interviews with domain experts, enabling us to categorize the distinct types of tasks associated with each of the eight steps elucidated by Xiao and Watson.

Steps 1 to 3 are identified as inherently intellectual (Table 2), requiring profound analytical skills and a comprehensive understanding of the research domain. These initial stages are characterized as critical reasoning activities firmly within the domain of human capabilities. Step 4 introduces tasks suitable for automation, such as duplicate removal, which is classified as low value. Automation tools like Python and ChatGPT are recognized for their efficiency in handling these mechanical aspects. However, it emphasizes that the subsequent step, validating preliminary results, remains intellectually demanding and high value. Steps 5 and 6 underline the researcher's crucial role in validating results and assessing machine outputs. While ChatGPT is acknowledged for its efficacy in conducting assessments using predefined criteria, the researcher ensures accuracy and fidelity by employing traditional methods. The intellectual centrality at this stage lies in the human validation process. In Step 7, ChatGPT's proficiency in justifying graphs from scientific databases is highlighted. Step 8 recognizes ChatGPT's utility in English proofreading during the concluding phase. However, the intellectual effort in these steps is attributed to the researcher, involving reasoned conclusions and discussions on the implications of the findings for theory, practice, and future research endeavors.

Table 2 distinctly emphasizes the collaborative nature of the SLR process, where human researchers and automated tools like ChatGPT complement each other. While automation is employed for certain low-value tasks, the intellectual contributions of the researcher remain crucial in decision-making, validation, and drawing meaningful conclusions. It is crucial to recognize the evolving nature of SLR methodology, marked by the continuous integration of both human expertise and machine capabilities. The integration of automated tools is seen as enhancing efficiency in specific tasks but does not replace the indispensable intellectual contributions of human researchers in framing problems, validating results, and interpreting findings.

4.2. Automating the Low-Value Activities by Integrating Python with ChatGPT

This section offers a concise overview of the prospective automation of low-value activities and introduces a tailored Python software application that integrates ChatGPT to automate Step 4, thereby optimizing the efficiency of low-value tasks.

In the proposed automation workflow (Table 3), Step 4 entails the customization of software to facilitate the automated identification and removal of duplicate entries within the literature review section of documents obtained from databases such as Scopus, presented in CSV format. ChatGPT assumes an important role in this step by contributing to the formulation of algorithms for the initial screening and prioritization of pertinent articles. Moreover, ChatGPT possesses the capability to discern thematic elements and conduct preliminary analyses, thereby evaluating the relevance of multiple articles to the overarching research question. Figure 1 shows an example of a Python script to remove duplicate entries.

The above Python script utilizes the pandas library [29,30] to remove duplicate entries from an Excel file sourced from Scopus. The central function 'remove_duplicates' requires two parameters: 'input_file' and 'output_file '. To begin, the script imports the pandas library, aliased as 'pd'. The primary operation involves loading an Excel file specified by the input_file path into a DataFrame using the pd.read_excel function. A DataFrame serves as a two-dimensional tabular data structure [31]. Following this, the script identifies and removes duplicate rows based on a specified column. In this instance, duplicates are eliminated based on the 'Title' column using the drop_duplicates method. By employing the argument keep='first', only the initial occurrence of a duplicate is retained. The inplace=True parameter modifies the DataFrame directly. Subsequently, the altered DataFrame is saved to a new Excel file specified by the output_file path using the to_excel method. The parameter index=False indicates that the DataFrame index should not be included in the output Excel file. The script incorporates a conditional block, if __name__ == "__main__": to check whether the script is being run directly (as opposed to being imported as a module). Within this block, the user is required to provide the paths for the input and output Excel files by assigning values to the variables input_excel_file and output_excel_file. The remove_duplicates function is then called with these file paths as arguments, initiating the duplicate removal process. Finally, a message is printed indicating the completion of the duplicate removal process, along with the path where the modified Excel file is saved. It's worth noting that users need to customize the input_excel_file and output_excel_file variables according to their specific file paths.

Proceeding to Step 5, the focus shifts to quality assessment, wherein algorithms or decision trees are deployed to automate the preliminary evaluation of research quality. This step aims to flag studies that may necessitate closer scrutiny by researchers. Step 6, which is linked to Step 4, involves the extraction of data. During this phase, complete manuscripts may be submitted to ChatGPT, thereby enabling the generation to undertake the initial coding and produce succinct summaries. Emphasis is placed on the conventional methods, such as NVIVO, with this approach to augment the validity and reliability of the research endeavor. In Step 7, the analysis and synthesis of data, the recommendation is made to apply NLP and ChatGPTs for accessing online content and generating succinct textual explanations for diverse Scopus/WoS graphs. Transitioning to Step 8, the final phase, report findings give the researcher the option to either input text directly from the manuscript for a comprehensive review by ChatGPT or configure the OpenAI API key to elicit a response from ChatGPT. In summary, the workflow underscores the potential advantages of automation in streamlining various aspects of the research process.

This section concludes with a conceptual framework that succinctly describes the content covered herein. Notably, we have delineated five steps amenable to automation, juxtaposed with three intellectual activities expressly reserved for human creativity and originality. We draw attention to the need to find a balance between automation and human examination to mitigate risks such as the inadvertent inclusion of irrelevant studies. The central message is that excessive reliance on automated tools in research reviews should be eschewed to ensure optimal accuracy and comprehensiveness.

4.3. Complete Automation of Systematic Reviews in Scientific Research – Future Reality or a Myth?

Having made it this far in the article, the reader might be wondering: is full automation of SLRs in scientific research really on the horizon, or is it just wishful thinking? While the suggested workflow points out the potential benefits of automation, it also stresses the need for a balanced approach. It warns against relying too much on automated tools, pointing out the risks involved, like accidentally including irrelevant studies or missing important information. Even though academic and scientific work is not about making predictions, the idea of completely automating systematic and bibliometric reviews still seems like a long shot with its own set of challenges and risks.

Striking a careful balance between automation and human review seems to be the smart move for ensuring accuracy and thoroughness. In a nutshell, based on our data collection and analysis, there is no evidence or any signs suggesting that full automation could become a reality in the short term or become a major focus of development. However, there is a need to explore deeper into the topic as it could be. Potential direction for future scientific research. Recognizing that research methods are always evolving, incorporating both human experience and machine capabilities, and stressing the importance of a balanced approach to reap the benefits of automation while minimizing the risks seems to be a more practical analysis.

5. Concluding Remarks

5.1. Theoretical Contributions

The information presented in this article delineates several critical theoretical advancements concerning the influence of OpenAI's ChatGPT on academic research. These contributions collectively signify a potential paradigm shift in academic practices, placing a spotlight on collaboration between human researchers and AI tools. This article suggests that ChatGPT enables academics to streamline routine, low-intellectual-value tasks through AI adaptation. At its core, the theoretical significance lies in positing that the success of future scholars hinges on their adeptness at utilizing AI to delegate low-value tasks, thereby allowing a more concentrated focus on high-value intellectual pursuits and reshaping the academic panorama. Furthermore, an additional theoretical contribution involves crafting customized software that integrates Python and ChatGPT to automate low-value tasks, particularly in SLRs.

The research also explores the complexities of defining added value in both high- and low-value intellectual activities within scientific inquiry. This exploration serves as a guide for scholars seeking to integrate AI into academic research methodologies. Overall, the paper introduces a conceptual framework and automation workflow proposing the integration of Python and ChatGPT to automate low-value tasks in SLRs, highlighting potential advantages while cautioning against over-reliance on automation. The conclusions underscore the complementary relationship between the efficiency of AI and the distinct capabilities of academics. The article advocates for a balanced approach where human researchers and automated tools synergize in the research process.

While acknowledging potential benefits, the article maintains a balanced perspective on the complete automation of systematic reviews, urging caution against over-optimism. It underscores the importance of a balanced approach, emphasizing the risks and challenges associated with full dependence on automated tools. In summary, these theoretical contributions collectively contribute to discussions surrounding the transformative impact of AI on scientific research methodologies, and the evolving role of academics in leveraging technology to enhance research efficiency.

5.2. Managerial Contributions

From a pragmatic standpoint, this article serves as a foundational resource aiding researchers in exploring their methodological and procedural choices. Historically, and to the best of our current knowledge, the delineation of tasks suitable for machine execution has not been definitively established. However, the present work provides researchers with preliminary guidelines in this regard. The article's significance extends beyond the knowledge it imparts, as ongoing contributions from the academic community may augment and refine the methodologies discussed herein.

Furthermore, it is crucial to recognize that AI technologies exhibit rapid cycles of technological innovation. Consequently, the findings presented in this article are expected to undergo evolution over time. A noteworthy practical contribution emerges in the successful integration of ChatGPT with Python. This integration lays the groundwork for the creation of customized software, empowering academics—especially those engaged in prolific SLRs—to automate routine tasks efficiently, thereby enhancing productivity and mitigating low-value efforts.

The aptitude of ChatGPT in facilitating search processes through predefined criteria significantly enhances the SLR workflow. This underscores the synergistic relationship between machine capabilities and human validation, emphasizing their complementary roles in advancing the efficiency and effectiveness of SLR procedures.

5.3. Limitations and Future Research Directions

While this research provides valuable insights, it is essential to acknowledge its limitations. The study predominantly concentrates on SLRs, and additional investigation is required to determine the applicability of the proposed framework to other research methodologies. The assessment of high- and low-value tasks is contingent on the context and may differ across disciplines. Moreover, the field is dynamic, and technological advances can impact the effectiveness of the proposed automation workflow over time. Another limitation lies in the inherent challenge of universally defining high- and low-value tasks. Despite attempts to categorize activities based on their intellectual demands, individual perspectives on what constitutes high value may vary.

Numerous prospective avenues warrant exploration in forthcoming research endeavors. A strategic approach to improve existing limitations involves extending the findings of this article across diverse research methodologies. Moreover, a comprehensive examination of ethical considerations is essential, necessitating an exploration of the ethical ramifications associated with the integration of AI into SLR. It is, therefore, essential to develop comprehensive guidelines for the responsible utilization of AI within academic settings. The establishment of such guidelines is pivotal since it may foster a framework for responsible AI utilization within scholarly pursuits. In essence, these guidelines should delineate ethical parameters, ensuring the conscientious and equitable deployment of AI technologies in the pursuit of scientific knowledge. In the broader context, as scholars persist in unraveling the dynamics between human intelligence and artificial capabilities, a commitment to reflective, ethical, and collaborative research practices emerges as paramount.