Analysis of Gross Methodological Errors in Webometrics Ranking Data (July 2025) Using Yemeni Universities as a Case Study

1. Introduction: The Impact of Global Rankings and the Importance of Data Integrity

Global university rankings have become profoundly influential tools in the higher education sector, affecting policy decisions, funding allocations, prospective student choices, and the academic reputation of institutions [2]. Among these, the Webometrics Ranking of World Universities, produced by the Cybermetrics Lab, stands out for its extensive scope, covering nearly 32,000 higher education institutions worldwide. The ranking aims to promote open access to the knowledge generated by universities and provides unique coverage of many institutions in the "Global South" often overlooked by other rankings [3].

However, web-based rankings, including Webometrics, have not been immune to academic criticism. The literature suggests that such rankings may contain methodological biases, tending to favor older, well-resourced institutions, while universities in developing nations are negatively affected by factors like the digital divide and limited internet access. Concerns have also been raised about their reliance on quantitative metrics that may overlook qualitative aspects of academic excellence, as well as a lack of transparency in data collection and verification methodologies [4].

In an effort to address challenges of data ambiguity and accuracy, the academic community has increasingly moved towards adopting Persistent Identifiers (PIDs) as a standard solution. A prominent example is the Research Organization Registry (ROR), an open, community-led infrastructure designed to provide a unique and persistent identifier for every research organization in the world. The ROR ID disambiguates institutions with similar names and tracks them through structural or name changes, ensuring clean and consistent metadata. The adoption of ROR IDs by essential scholarly infrastructure systems like Crossref and DataCite makes their correct usage a hallmark of sound data management. Webometrics’ decision to incorporate ROR IDs is not merely a technical update but an implicit declaration of commitment to accuracy and modern standards of data integrity, aimed at bolstering its credibility against existing methodological critiques.

This study’s central thesis is that although the Webometrics ranking has adopted ROR IDs to enhance accuracy, a detailed analysis of its published data for the July 2025 edition reveals a widespread methodological failure in their application. Using Yemeni universities as a ’critical case’ to probe for systemic weaknesses, this paper presents empirical evidence of gross methodological errors. This shifts the critique from the ranking’s metrics to its fundamental data governance, demonstrating that this failure is not a mere technical glitch but a failure to meet the very standards the ranking claims to uphold.

2. Analytical Framework and Methodology

This study was designed to be replicable and transparent, relying on public data sources and a rigorous verification process. The methodological framework aims to document errors irrefutably and present them within a clear classification.

2.1. Data Sources

The analysis was based on two primary data sources:

• Primary Data Source: The dataset under review is the "Ranking Web of Universities (webometrics.info). July 2025 edition (Version 1)," published by Isidro F. Aguillo via the "figshare" platform. This dataset is precisely identified by its Digital Object Identifier (DOI): https://doi.org/10.6084/m9.figshare.29588921.v1 [1].
  A notable structural limitation of the published dataset is the absence of a dedicated ’Country’ column for each listed institution. This omission complicates data sorting and verification, and can introduce ambiguity, especially when dealing with universities that have similar names across different regions.
• Reference Verification Source: The official Research Organization Registry (ROR) was used as the primary reference for verifying the correctness of identifiers. To ensure comprehensiveness and accuracy, verification was conducted by analyzing a full ROR database dump, publicly available via Zenodo [5]. Additional cross-verification was performed using other persistent identifier databases such as Wikidata to ensure consistency.

2.2. Scope of the Study

The scope of the analysis was intentionally limited to the sample of Yemeni universities listed in the Webometrics data file. This approach provides a focused and in-depth case study capable of revealing systematic patterns of errors in a manageable and fully documentable manner.

2.3. Classification of Errors

To systematically analyze the nature of the errors, a classification of four main types was developed based on initial observations of the data. Each type is defined as follows:

• Type I: Chained Errors: The most complex pattern, where the ROR ID of university (B) is assigned to Yemeni university (A), while the correct ID for university (A) is assigned to a third university (C).
• Type II: Misidentification: The direct assignment of an incorrect ROR ID belonging to another institution, often one with a similar name, to a Yemeni university.
• Type III: Omission of Existing IDs: A university that officially possesses a ROR ID is listed, but its identifier field is left blank in the dataset.
• Type IV: Complete Omission: The total absence of a university from the dataset despite its existence and possession of an official ROR ID.

3. Results: A Typology of Data Integrity Failures in the Yemeni University Sample

A meticulous examination of the Webometrics ranking data for the July 2025 edition reveals serious and systemic methodological flaws in the handling of ROR IDs within the Yemeni university sample. These errors have been categorized into four distinct types, illustrating the systemic nature of the problem rather than isolated, random mistakes.

3.1. Type I: Chained Errors

This error type represents the most egregious failure in data processing, creating a complex web of incorrect assignments that links unrelated institutions across continents. Table 1 illustrates this chain of errors, documented directly from the published data file.

3.2. Type II: Misidentification

This error occurs when two different institutions are confused due to name similarity—precisely the type of problem ROR IDs are designed to solve.

• Al Jazeera University, Ibb: According to the data file [1], this university was assigned the ID https://ror.org/00basmr24. Upon verification, this ID officially belongs to Aljazeera Private University in Syria, a clear case of institutional misidentification.

3.3. Type III: Omission of Existing IDs

In these cases, the a shows that some Yemeni universities with official, registered ROR IDs were included in the ranking without their identifiers being listed, indicating a failure to perform a basic check against the ROR database.

• Al-Nasser University: Its official ID https://ror.org/02rsbbb97 was omitted.
• Emirates International University: Its official ID https://ror.org/03j6pc929 was omitted [1].

3.4. Type IV: Complete Omission from the Ranking

This type of error represents a failure at the initial stage of compiling the list of institutions, where universities were entirely excluded from the dataset despite their existence and possession of an official ROR ID.

• Iman University: This university, which holds the official ROR ID https://ror.org/04ajy5s58, is completely absent from the Webometrics dataset published by Aguillo (2025) [1].

Table A1 in the Appendix presents the dataset with manually corrected ROR identifiers to accurately match each institution. It is crucial to note that the ranking scores shown remain the original data published by Webometrics. A definitive correction of the ranks would require a re-calculation by the Cybermetrics Lab based on these accurate institutional identities.

4. Discussion: Implications for Ranking Credibility and Fairness

The methodological failures documented in the results are compounded by a recent and alarming lack of transparency regarding the ranking’s online presence. The official Webometrics portal (https://webometrics.info), which historically published detailed ranking data, has ceased its updates since the July 2025 edition, with no notice of maintenance, migration, or discontinuation. Compounding this ambiguity, a new and seemingly unaffiliated website has emerged at https://webometrics.org/. This new portal uses the same branding but carries a disclaimer explicitly stating it is an "independent university ranking platform and is not affiliated with the former Webometrics[.]info website or the Consejo Superior de Investigaciones Científicas (CSIC)." This schism creates profound confusion and further erodes the trustworthiness of the Webometrics brand, leaving users and institutions unable to determine the official source of data or the status of the ranking itself.

The patterns of errors documented in this analysis transcend mere individual oversights to signify deep methodological failures in the data collection and validation processes at Webometrics. These findings, derived directly from the ranking’s official data, have serious implications for the credibility and fairness of the ranking, both locally and globally.

The existence of "chained errors," in particular, strongly suggests something beyond human data entry mistakes. The interconnected nature of these errors, where IDs are swapped between institutions in Yemen, Japan, Peru, and Spain, implies a catastrophic failure in an automated process, such as an inaccurate name-matching algorithm or the merging of spreadsheets without using key fields for verification. This pattern of error reveals an absence of basic quality controls and sound data governance mechanisms at the Cybermetrics Lab; the severity and systemic nature of these flaws make it difficult to dismiss them as mere "teething problems" of a new system.

It is deeply ironic that a system like ROR, designed specifically to eliminate ambiguity and ensure accuracy, has been used in a way that creates new layers of chaos and distortion. This superficial adoption of persistent identifier technology, without a commitment to the underlying principles of data integrity it represents, not only devalues the technology but also misleads the academic community, who might assume that the use of ROR is a guarantee of accuracy. The problem is not a scarcity of correct data, but a clear failure to use it. ROR IDs are universally accessible via APIs or full database downloads. The fact that this readily available reference data was not used to validate the assignments indicates either a significant lack of technical competence or a gross neglect of accuracy standards.

The damage caused by these errors is not theoretical but tangible, affecting the institutions involved. When a Yemeni university is incorrectly linked to an institution in Pakistan, all associated bibliometric and cybermetric indicators are skewed, artificially inflating or deflating its perceived research performance. This digital misrepresentation harms the universities’ reputations, their ability to attract talented students and researchers, and their standing in the global academic community.

Although this study focuses on a specific sample from Yemen, the systemic nature of the identified flaws makes it highly probable that similar errors exist in the data for universities in other countries, especially those in the Global South that may lack the resources to audit and challenge their data. These findings, therefore, do not just question the accuracy of the ranking for Yemeni universities but cast a shadow of doubt over the integrity and reliability of the entire global Webometrics ranking. If such fundamental errors exist in a simple process like ID matching, how can the more complex calculations that the ranking relies on be trusted?

5. Conclusion and Recommendations

This critical review demonstrates, with direct and documented evidence from the analysis of the Yemeni university sample in the published Webometrics data for the July 2025 edition, that there are serious and varied methodological flaws in the handling of ROR IDs. These failures, ranging from misidentification to complex chained errors, fundamentally undermine the ranking’s credibility and raise serious questions about its data validation mechanisms.

Based on these findings, the following recommendations are made:

Recommendations for the Webometrics Ranking Administration (Cybermetrics Lab):

• Immediate Retraction and Correction: The flawed July 2025 dataset should be immediately retracted, and a corrected version should be published after a thorough verification of all ROR IDs.
• Comprehensive Audit: A full and transparent audit of all global data in the ranking should be conducted to identify similar errors in ROR ID assignments for universities in other countries.
• Methodological Transparency: A detailed report should be published explaining precisely how data is collected, validated, and how persistent identifiers are integrated, outlining the quality assurance procedures in place to prevent such errors from recurring.
• Process Re-engineering: Robust automated validation protocols should be adopted. This includes using the ROR API to programmatically verify each ROR ID and implementing fuzzy name matching algorithms with country-level filtering to flag potential mismatches before publication.

Recommendations for Affected Academic Institutions:

• Official Challenge: Yemeni universities and other institutions that suspect inaccuracies in their data should use the evidence documented in this study to formally contact the Webometrics administration and demand immediate correction of their data.
• Proactive ID Management: Universities are encouraged to claim and regularly update their official profiles in the ROR registry to ensure their metadata is accurate and authoritative at the source.

Recommendations for the Academic Community:

• Critical Scrutiny: Researchers, policymakers, and funding bodies are called upon to exercise a higher degree of critical scrutiny of all university rankings and to demand higher standards of transparency and data integrity from ranking providers.
• Promote PID Adoption: Emphasize the importance of consistent use and advocate for the adoption of open persistent identifiers like ROR across all academic systems to build a more reliable and interconnected research ecosystem.