Reconsidering Google Scholar Regarding PRISMA Guidelines

1. Introduction

In an article first online 10 November 2018—cited by 1000 according to a 19 November 2025 Google Scholar search of its author, Michael Gusenbauer—the opening sentences state, “Information on the size of academic search engines and bibliographic databases (ASEBDs) is often outdated or entirely unavailable. Hence, it is difficult to assess the scope of specific databases, such as Google Scholar” [1]. Thus, this article, which compares the sizes of 12 academic search engines, commences by singling out Google Scholar, preempting the research results. It is not surprising, therefore, that the title begins with the question “Google Scholar to overshadow them all?” It is a query alerting readers that Google Scholar is decidedly problematic.

Almost a year later, Gusenbauer (in collaboration with Haddaway) asks in an article cited 2493 times according to the Google Scholar search, “Which academic search systems are suitable for systematic reviews or meta-analyses?”—investigating Google Scholar as the first database, followed by PubMed, “and 26 other resources”. This article states emphatically, “our findings demonstrate why Google Scholar is inappropriate as a principal search system. We call for database owners to recognize the requirements of evidence synthesis and for academic journals to reassess quality requirements for systematic reviews” [2]. As such, within the year, Gusenbauer progressed to the point of making a clear statement advising academic journals to no longer accept Google Scholar as a primary database for systematic reviews.

In an August 2021 publication (cited 29 times according to that Google Scholar search), Gusenbauer asserts the following to researchers. “I claim that this googling mentality is only adequate for lookup searching, the type of searching Google and Google Scholar are predominantly geared towards. For exploratory and systematic searching, however, fast contentment with high-ranking results might be problematic as it reduces the quality of search outcomes” [3].

By 2024, in an article assessing Search Smart—a free website offering guidance to researchers regarding the databases most likely to be productive for their search—the reader is reminded by Gusenbauer “that some databases, like Google Scholar, are ranked at the very bottom because although they may cover a given type of record, the extent to which they cover it is unknown” [4]. Later in that publication, the author laments, “Google Scholar, for example, has barely improved its functionality in recent years.”

In a span of less than six years, Gusenbauer advanced his advice from cautioning researchers about Google Scholar to recommending that it be a supplementary database, to stating that it is inappropriate, and then ranking it at the bottom as a search engine in key respects. The most recent of these publications has fewer citations than the others (10), according to the Google Scholar search of 19 November 2025. Nevertheless, Gusenbauer’s views on Google Scholar as a poor database for systematic reviews have only intensified. An example is his most recent work with Gauster—there, Gusenbauer has offered the following advice to researchers on conducting a literature search for systematic reviews and meta-analyses. “One database that performs particularly poorly at keyword searches is Google Scholar (GS), where misinterpretations of Boolean search queries lead to relevant results being omitted” [5]. As of 19 November 2025, citations of this work are 43 times.

Gusenbauer is not the first researcher to judge Google Scholar as deficient. Released as a database in November 2004 [6,7], one of the first articles on Google Scholar was by librarians. It opened with these comments. “Handwringers worry that Google Scholar will drive our users away from libraries. They worry that students will gravitate to Google Scholar due to its name recognition and ease of use. They worry that Google Scholar will become their sole source of research information and that students will bypass libraries and librarians altogether” [7]. This 2005 article concludes with the comment, “Many individuals have raised the question, “what’s in it for Google?””. An additional 2005 article by two other librarians is objective regarding the limitations of Google Scholar. “We are in agreement with those who have critiqued GS’s lack of authority control, lack of choices for sorting results, lack of controlled vocabulary, and lack of version control; Google’s unwillingness to specify what exactly GS indexes; the currency and completeness of its contents; and the uncertainty regarding the frequency with which it is updated. On the other hand, we were impressed by the degree to which GS manages to weed out non-scholarly materials, by its potential for resource discovery, by the quality and number of relevant results generated by GS in comparison with results generated in some research databases (for certain topics), and by its usability” [8].

A 2008 publication highlights the immediate recognition of Google Scholar’s creation by librarians [6]. By then, three years later, the view was that “citation counts reported by Google Scholar are not entirely reliable”. This 2008 publication notes that “Google staff have been unwilling to provide much information about Google Scholar’s selection algorithms, sorting (relevance) algorithms, content sources, or organizational partnerships.” Nevertheless, even with these cautions, the 2008 author was able to conclude objectively that “Google Scholar provides excellent coverage of at least one social science field, in both absolute and comparative terms. The results presented here suggest that the unconventional methods used to build the GS database have not adversely affected its coverage of the scholarly literature.”

Yet by the next decade, many studies of Google Scholar focused on its limitations rather than its positive features. Although not the first of his works on Google Scholar, by the 2010s Jacsó produced a series of publications whose titles focused on what he considered the irrationality of academic use of Google Scholar: “Metadata mega mess in Google Scholar” [9], “Google Scholar duped and deduped–the aura of “robometrics”” [10], “Google Scholar Author Citation Tracker: is it too little, too late?” [11], “Grim tales about the impact factor and the h-index in the Web of Science and the Journal Citation Reports databases” [12], and “Using Google Scholar for journal impact factors and the h-index in nationwide publishing assessments in academia – siren songs and air-raid sirens” [13]. Jascó, as a library scientist, is clear on his distrust of Google Scholar. One quotation from a 2012 article provides an example. “Standardization of metadata has been one of the highlights in the field of library and information technology. Unfortunately one of the basic principles in the development of Google Scholar was to create metadata elements from the data collected by its crawlers and processed by its parsers instead of using existing metadata that are provided by practically every publisher of scholarly journals. Their accuracy and consistency is incomparably better than the metadata crafted by Google Scholar” [13]. Another author from this period goes even further than Jascó in his criticism of the database, stating that, “Regrettably, Jacsó’s work does not reveal all the errors in Google Scholar, although it does expose the most notorious and flagrant, a fact that has led to an improved service” [14].

In contrast, other scholars during the 2010s were more positive in their assessment of Google Scholar. Gehanno et al. titled their research article on Google Scholar, “Is the coverage of google scholar enough to be used alone for systematic reviews” [15]. They conclude the following. “The coverage of GS for the studies included in the systematic reviews is 100%. If the authors of the 29 systematic reviews had used only GS, no reference would have been missed. With some improvement in the research options, to increase its precision, GS could become the leading bibliographic database in medicine and could be used alone for systematic reviews.” Still, it was the generally negative evaluations of Google Scholar as a database that provided the foundation for the influential Gusenbauer articles—those relegating the database to supplementary status regarding preferred reporting items for systematic reviews and meta-analyses, PRISMA.

The PRISMA Statement, including its extensions, is a minimum set of evidence-based recommendations for systematic reviews encouraging comprehensive transparency in reporting [16]. Initially published in 2009, following a three-day meeting of 29 stakeholders in 2005 [17], the guidelines now also include scoping reviews and review protocols aiming to ensure accuracy and unambiguous reporting for all aspects of systematic reviews. PRISMA helps authors describe the exclusion criteria, the search process, and the outcome, or what researchers are planning to do in the case of a review protocol [18]. They represent the most-cited reporting guidelines in biomedical literature [19]. An updated version, PRISMA2020, was published in 2021 [20]. The number of systematic reviews following the PRISMA guidelines has increased substantially over the years [16,21]. Because the PRISMA guidelines have become integral to the systematic review process, it is significant if Google Scholar—considered the most comprehensive [22]—is judged inadequate as a primary database. Since the publication of Gusenbauer’s critiques, some researchers have excluded Google Scholar from their search process [23,24], while others have asked, “Does Google Scholar count as a database or as an additional information source for the PRISMA 2020 flow diagram?” [25]. This uncertainty about the sufficiency of Google Scholar as a database concerning the PRIMSA process presents the rationale for undertaking this study.

Two approaches investigate this matter. The first is to examine and enumerate the problems Gusenbauer identified in his papers that led to the determination that Google Scholar should be considered a supplementary database for systematic reviews. The second is to present a comprehensive analysis of the results of scoping reviews undertaken by this author between 2023 and 2025 to determine the value of Google Scholar as a database. The outcomes of these two investigation methods are that the evidence is irrelevant to conclude that Google Scholar should be a supplementary database when conducting a systematic review following the PRISMA guidelines. Support for this conclusion comes from examining the author’s previous scoping reviews and finding that, except for one, Google Scholar returns the most and the most relevant results. As such, the recommendation is to reconsider Google Scholar as a primary database in following the PRISMA guidelines.

2. Materials and Methods

The materials of this study are from two source types. The first is the comments in Gusenbauer publications [1,2,3,4,5] suggesting that Google Scholar should be considered a supplementary database in following PRISMA guidelines. The second source of materials is the scoping reviews by this author published since 2023 [26,27,28,29,30,31,32,33].

The methodology initially involves searching each of Gusenbauer’s publications for his comments on Google Scholar and listing each of them. There is then a categorization of these comments by their types. This categorization compares these remarks with the requirements of the PRISMA guidelines stated in [18,20]. As such, this part of the study aims to present all of Gusenbauer’s negative comments concerning the database. The second part of the methodology involves examining this author’s published scoping reviews for the database search results. The returns of each database searched per publication are combined into a table to create these materials.

There was no use of generative artificial intelligence (GenAI) in this paper (e.g., to create text, data, or graphics, or to assist in study design, data collection, analysis, or interpretation).

3. Results

This section divides the results of the two aspects of this study into the following subsections.

3.1. Limitations of Google Scholar, According to Gusenbauer’s Five Publications

The results of Table 1 arise from searching each of [1,2,3,4,5] for “Google Scholar”, and including all 43 statements that represent the database negatively. In three of these publications [1,2,4], Gusenbauer acknowledges that Google Scholar is the largest and most used database by scholars. However, rather than being considered a positive attribute of Google Scholar, these advantages are presented to demonstrate why understanding and acknowledging its limitations as a database is imperative for researchers. Table 1 reveals that Gusenbauer’s criticisms of Google Scholar were, in some cases (such as 7), beyond rational. The table presents that the two earlier publications by Gusenbauer make the most disparaging comments about Google Scholar—unfortunately, these are also the most cited publications, as mentioned in the Introduction section of this study.

Table 2 accompanies Table 1 to explain the marks beside each numbered quotation. From Table 2, there are ten distinct Gusenbauer themes regarding the limitations of Google Scholar. They are listed in Table 2 by the order in which they appear chronologically. Four themes are unique to specific publications: (1) Difficult to assess the scope [1], (2) Not all documents are fully available [2], (3) Not curated [2], and (4) Only allows searches of up to 256 characters [2]. Two comments (1) that the database is imprecise [1,2], and (2) there has been no improvement in Google Scholar in recent years [3,4], are mentioned by two publications. Three works stipulate problems with three issues: (1) Google Scholar is trying to convince users of its capabilities [1,2,3], (2) it is secretive [1,2,4], and (3) it doesn’t support Boolean search functionality [2,3,5]. Most of the negative comments are in the early publications alone; however, there are four criticisms of the database that extend to the recent publications: (1) Tries to convince users of its capabilities, (2) Secretive, (3) Doesn’t support Boolean search functionality, and (4) No improvement in recent years. There are three judgments of Google Scholar that Gusenbauer repeats frequently in individual publications and in more than one: (1) Secretive, (2) Imprecise, and (3) Doesn’t support Boolean search functionality.

In considering the implications of Gusenbauer’s disparaging claims about Google Scholar for systematic reviews, it is relevant to present the PRISMA guidelines for database suitability. Of the well-cited publications directly associated with the PRISMA 2020 guidelines [18,20], there are analyses, outcomes, sources, data, and studies that are primary; however, there is no description of primary databases. According to these publications, the PRISMA guideline requirements are as follows: the description of the search strategy for each database is sufficient to allow another researcher to reproduce the results. This sufficiency requires identifying the database used, the search date, and the inclusion and exclusion criteria. Similarly to “primary”, “supplementary” is not a term used in relation to databases in [18,20]. What relegates a source to supplementary status is that it is a study not found in any database. These sources include hand searches, trial registries, or websites.

In recognizing the nonexistence of a description of a primary database in the PRISMA guidelines, Gusenbauer’s conclusion, relegating Google Scholar to supplementary status, is baseless. However, those guidelines, as presented in [18,20], do necessitate that the search be comprehensive, reproducible, and transparent regarding its details. The point to consider is whether any of the ten types of comments Gusenbauer makes in his five publications that mention the limitations of Google Scholar identify the database as lacking in comprehensiveness, reproducibility, or transparency. There is no such mention for comprehensiveness. In fact, three of the Gusenbauer publications [1,2,4] note that it is the most comprehensive academic search engine. Concerning reproducibility, the adequacy of Google Scholar search parameters for search replication is the point. Gusenbauer does not state that Google Scholar is lacking regarding search replication. The problem noted in [1] is that the returns are inconsistent and limited, causing a lack of validity. Yet, concurrently, this 2018 publication by Gusenbauer estimates Google Scholar accesses 389 million records. This amount compares with just over 72 million for Scopus and almost 68 million for Web of Science—two databases Gusenbauer views as consistent, ample, and valid. Given that Google Scholar is a half order of magnitude larger than these two databases, the comparison with them is misleading. By his ruling that Google Scholar is secretive, imprecise, inappropriately persuasive, and lacking support for Boolean search functionality, Gusenbauer implies that Google Scholar lacks transparency. However, according to PRISMA guidelines, transparency regards the search itself—not the database. It is the researcher who must be transparent by stating the name and date of the database searched, along with the inclusion and exclusion criteria. As such, the limitations of Google Scholar are irrelevant to following the PRISMA guidelines—meaning the use of Google Scholar in this regard is comparable to other databases—ones Gusenbauer judges primary.

3.2. Results of the Author’s Scoping Reviews

The demonstration is that the studies by Gusenbauer [1,2,3,4,5] are unable to identify deficiencies of Google Scholar that leave it a supplementary database according to PRISMA guidelines. However, it may be that Google Scholar is unable to return results for systematic reviews because it lacks Boolean functionality, for example. An examination of eight scoping reviews by the author following PRISMA guidelines tests whether Google Scholar returns results for systematic reviews. Table 3 provides the outcome of this examination. Not only did Google Scholar return results for each of the eight scoping reviews, in all but one instance [33], it returned the most.

Still, it is possible that, following the PRISMA exclusion process, the returns for assessment did not include those from Google Scholar. Table 4 provides the included results. Confirmed is that for all but one [33], Google Scholar was the database with the included results. Additionally, without the Google Scholar search for [28] and [29], there would have been no included results. For [27], there would have been only one included result from Web of Science. In every case but one, where Google Scholar was a source of included results [32], it produced the most included results.

4. Discussion

Of Gusenbauer’s criticisms of Google Scholar, the most persistent and the only one evident in his latest publication is that the database does not support Boolean search functionality. Although the PRISMA guidelines do not require this type of functionality, the implication is that Google Scholar lacks a relevant means of search refinement. What Google Scholar does support is the use of quotation marks around terms to search for the exact phrase and eliminate automatic synonym/variation searching for a single word [34]. Table 3 identifies [28] as including two Google Scholar searches. One returned 290 results, and the other 25 results. The keywords searched that produced 290 results were [Burnout psychological flow Csikszentmihalyi employees OR healthcare providers “COVID-19” since 2021]. Those that returned 25 results were [“burnout” “psychological flow” “Csikszentmihalyi” employees OR healthcare providers “COVID-19” since 2021]. The results of both searches were retained for the study, as only 17 of the returns were common between them. This result indicates that using quotation marks is a meaningful way to refine a Google Scholar search process. The Boolean operator “OR” was included in this search, although it was not utilized as such by the database search process.

The power of using or not using quotation marks is relevant regarding a Google Scholar search. As an additional example, the most recent scoping review published by this author [33] was the only one to have no results included from Google Scholar (see Table 4). The Google Scholar search for this study used quotation marks around each concept [“self-directed learning” “mobile technology” “co-creation” “virtual worlds”, “2021-2025”, “no citations”]. The outcome of 12 returns for this search (see Table 3) was from 30 March 2025. This one was the only Google Scholar search for that study. However, performing the search on 4 December 2025 with no quotation marks among the keywords [self-directed learning mobile technology co-creation virtual worlds 2021-2025 no citations] produced “About 15,000 results” for publications since 2021. Placing quotation marks around only “self-directed learning”, a Google Scholar search on the same day produced “About 8,340 results”. If quotation marks were around “mobile technology” alone, the returns were “About 2,020 results”. With quotation marks around just “co-creation”, there were “About 1,450 results”. Thus, quotation marks make a significant difference, and their type of use is relevant.

What can be identified from the assessment of the author’s scoping reviews regarding Google Scholar is that, similar to the results of “Is the coverage of google scholar enough to be used alone for systematic reviews” [15], the 2013 conclusion is the maintained. As previously quoted in the Introduction above, “The coverage of GS for the studies included in the systematic reviews is 100%. If the authors of the 29 systematic reviews had used only GS, no reference would have been missed. With some improvement in the research options, to increase its precision, GS could become the leading bibliographic database in medicine and could be used alone for systematic reviews.”

A limitation of this study is that it was a sole effort by the author. Researcher [35], confirmation [36] or cognitive bias [37] may be the outcome. These potential biases are mitigated in two ways: (1) a presentation of the direct quotations from Gusenbauer, and (2) the support of the supplementary files attached to each of the author’s publications referenced. The supplementary files provide the exact scoping review exclusion process that produced the included results. As such, any researcher can fully examine the author’s assessment. They have complete and transparent documentation.

Future research directions on this topic include reviewing the evidence presented by Gusenbauer and others who view Google Scholar as a supplementary database for systematic reviews. Regarding PRISMA guidelines, the aim is to confirm that there is no reason for Google Scholar to be judged unacceptable as a primary database for systematic reviews.

5. Conclusions

The claim by well-cited articles that Google Scholar is inappropriate as a primary database for following the PRISMA guidelines for systematic reviews is unsupported by an examination of quotations of these well-cited articles. Additionally, the comparison of Google Scholar to primary databases in scoping reviews demonstrated not only its adequacy but its superiority. Examining eight author-published scoping reviews accomplished this outcome. The suggestion is to accept Google Scholar as a primary database, comparable in all relevant ways to other primary databases.