Mining and Mineral Processing Journals in the WoS and Their Rankings When Merging SCIEx and ESCI Databases—Case Study Based on the JCR 2022 Data

1. Introduction

Journal rankings are commonly used to assess publishing houses and writers. They have become an essential tool for research and library administration, and can be generated in a variety of ways, ranging from a group agreement among specialists to a journal impact index based on citations [1,2,3]. The Web of Science (abbr. WoS) maintains several databases including its Core Collection, the Science Citation Index-Expanded, Social Sciences Citation Index, Art & Humanities Citation Index, Emerging Sources Citation Index, Conference Proceeding Citation Index, Book Citation Index. Another derivative base from the Core Collection is the Journal Citation Reports (abbr. JCR) for the journals ranked based on the Journal Impact Factor (abbr. JIF) and divided into quartiles (abbr. Q). Nowadays, the JCR is published once per year, usually in late June and is considered the most prestigious measure of the international impact of a journal. Until 2022, only a part of the journals selected in the WoS had been selected in the JCR and had have calculated the JIF. Those were all the journals, in the field of natural and technical sciences, reviewed and selected in the following databases: Current Contents (abbr. CC), Science Citation Index (abbr. SCI) and Science Citation Index – Expanded (abbr. SCIEx), as the largest single database including the first two (vice versa is not valid).

However, from 2015 onward, the WoS developed a fourth database named the Emerging Sources Citation Index (abbr. ESCI), where Clarivate (the owner of the WoS) included all the journals that applied for the WoS inclusion, passed the evaluation process, but did not receive the final editorial approval for inclusion in the SCIEx. For the ESCI journals (self) citations, h-index, and number of publications per year had been calculated, but the JIF, and consequently quartiles were not, so they were omitted from the annual JCR report. This was changed in 2022, when the ESCI journals have been included in the JCR, but without rankings in the quartiles based on the impact factor[4].

Almost at the same time, Clarivate introduced the Journal Citation Indicator (abbr. JCI) in the WoS, as an additional ranking measure, which has been calculated from 2021 for all the journals in the WoS, regardless of their database, and by which all the journals have been immediately ranked into the quartiles based on the JCI value. So, the JIF is a relatively simple measure, and the JCI is a measure of the average Category Normalized Citation Impact (CNCI) of the citing items (articles and reviews) published by a journal over the last three-year period. Clarivate introduced the JCI to enable evaluation of the journals based on the other metrics besides the JIF, but also following the Scopus database (by Elsevier) that has two weighted and normalized measures for the journal ranking [5], namely the Scimago Journal Rank (abbr. SJR) and the Source Normalized Impact per Paper (abbr. SNIP). It is important to mention that in December 2016, Scopus launched the Citescore as an alternative to the Clarivate’s JIF measure [6,7].

Such normalization of the impact factors and similar measures based on database subject categories is a statistically demanding process [8] and is out of the scope of this paper. Consequently, the study included the publication, ranking and citation patterns of the journals selected in the JCR subcategory Mining and Mineral Processing. Previously, some of the current authors managing / edit(ed) such journals and published citation analysis for the journal published by “small publisher” in the Publications [9].

The JCR is a very large database that includes 254 categories in 21 groups, where the number of categories decreases from 59 categories in the Clinical Medicine group to 7 categories in the Agricultural Sciences as well as in the Philosophy and Religion category. The number of journals ranges from 7441 journals in Clinical Medicine to 423 in the Agricultural Sciences [10]. The analysed data is a part of the Engineering group where 41 categories and 3,556 journals exist in the JCR 2022. The data have been derived for the subcategory Mining and Mineral Processing (both belonging to the Engineering and Geosciences) which includes 20 journals that are part of the SCIEx (median Impact Factor (IF) 2.2), and 12 of the ESCI (median IF 1.2).

2. Analysis of the Mining and Mineral Processing journal’s rankings in the JCR 2022

2.1. Journal Impact Factor

The journal impact factor (JIF) is a scientometric variable calculated by Clarivate as a journal-level metric. From the moment of its introduction, it has been used as the most popular measure for the assessment of the journals’ prestige in the academic field (or the WoS subject group), often used by universities or funding bodies in decision processes of different applications [11,12].

The calculation of the JIF is a simple mathematical expression (Equation 1) where for any year, the JIF is the ratio between the number of citations received in a selected year and the total number of publications (or citable items) in that journal in the preceding years [13]:

$${IF}_y=\frac{{Citations}_y}{{Publications}_{y-1}+{Publications}_{y-2}},$$

(1)

where y is the selected/observed year, y-1, y-2 are the two preceding years regarding the selected one.

So, officially, the JIF is a two-years’ measure, although it can be calculated for any given number of the preceding years. Consequently, the JCR includes information about a five-year JIF [14].

2.2. Journal Citation Index

The Journal Citation Indicator has been introduced by Clarivate in 2021 as a new journal metric that can be compared across scientific fields (Clarivate’s groups and (sub)categories) and accounts for the specific characteristics of different fields and their publications [15]. Consequently, it is a complex and weighted measure, as opposed to the JIF as a simple citation counting tool.

The new Journal Citation Indicator meets this requirement for journal evaluation, providing a single number that represents the specific characteristics of different fields and their publications [15]. The calculation of the JCI is a simple derivative of another Clarivates’ measure [16]; - the Category Normalized Citation Impact (CNCI) (Equations 2, 3 and modified 4):

$$CNCI=\frac{c}{e_{ftd}},$$

(2)

$$CNCI=\frac{\sum \frac{c}{e_{f\left(n\right)td}}}{n},$$

(3)

$${CNCI}_{journal}=\frac{\sum {CNCI}_{paper}}{p}$$

(4)

where e is the expect citation rate or baseline, c is the number of times cited, n is the number of subjects to which a paper is assigned, p is the number of papers, f is the field or subject area, t is the year, d is the document type, journal (or “i" in original) is the entity being evaluated (journal, institution, country/region, person, etc.).

Equation 3 is optionally applied when the document belongs to multiple categories and the CNCI is firstly calculated for each category and the final value is an average of such values. If a category is filtered, the single CNCI will be given just for that category. Generally, the CNCI value of 1 represents something like a “world average”.

Moreover, the JCI is just journals’ value defined as the mean CNCI(s) for all the articles and reviews (as citable documents) in the three preceding years. For example, the 2023 JCI will be calculated from the CNCIs of all the documents published in the period 2019-2022 and it will be interpreted in the same way (“CNCI value of 1.0 means that, across the journal, published papers received a number of citations equal to the average citation count in that subject category” [16].

The JCI has been introduced to increase information obtained using the JIF, but mostly trying to “weight” the fact that the number and period of receiving a citation is highly dependent on the subject field. Besides equation(s), the next major difference between the JIF and the JCI is the observed period of citing papers, which, in the case of the JIF is two and in the case of the JCI, is three years.

2.3. Merging of the Mining and Mineral Processing journals rankings based on the JIF and JCI values

In 2022, Clarivate announced that Journal Citation Reports (JCRT) would extend the JIF to all journals in the Web of Science Core Collection (WoS CC) [17], including those indexed in AHCI and the multidisciplinary ESCI. In late June 2023, the Journal Citation Report update was released, which included an expansion of JIF coverage to all journals in the (WoS CC), resulting in the inclusion of more than 9,000 journals from more than 3,000 publishers for the first time. This is, of course, very important for small publishers such as universities and scientific institutes. However, it should be noted that the AHCI and the ESCI journals will not be ranked, or receive a quartile, or percentile until 2024 [18].

In Table 1 a comparison of the JIF and JCI rankings of journals in the Mining and Mineral Processing subcategory in JCR 2022 is shown. The table shows that five journals from the ESCI database have the JIF score above the median for the category, which is 1.25. In addition, one of the ESCI journals with a JIF value of 8 is in second place in the imaginary JCR 2022 ranking presented in Table 2. Some ESCI journals have higher IFs than SCIE journals but are not included in the SCIE database, probably because they do not fulfil some or all additional citation criteria (citations of authors, editorial board members and/or content). It is obvious that several ESCI journals have higher IFs than SCIE ones but are currently not included in the SCIE database, which requires a more urgent, faster and transparent procedure for the Clarivate calculations of provisional impact factors and quartiles for the entire ESCI database.

Table 2 shows the imaginary JCR 2022 ranking in quartiles with the merging of journals from the ESCI and the SCIE bases. The ranking in quartiles is based on the JIF 2023 calculated with three decimal places, although according to the JCR, the JIF now has only one decimal place instead of three. The JIF with only one decimal place will probably result with more rank position ties in many categories. According to Clarivate, this change encourages the comparison of journals considering additional indicators and descriptive data in the [18].

In addition to the analysis of the journal rankings by the JIF, a comparison of WoS and Scopus scholarly metric scores was analysed among journals in the Mining and Mineral Processing subcategory in JCR 2022. Unlike WoS, Scopus does not have a category that covers only the mining and mineral processing topics. However, of the 32 journals listed under the category Mining & mineral processing in WoS, 31 are indexed in Scopus under the subcategory Geotechnical Engineering and Engineering Geology, which is part of the main category Earth and Planetary Sciences. The journal metrics from WoS (the JIF and the JCI) and Scopus (the SJR, the CiteScore and the SNIP) for the year 2022 are shown in Table 3.

Based on the citation indexes of WoS and Scopus in 2022, Table 4 presents the hypothetical quartile ranking of the journals in the JCR Mining and Mineral Processing subcategory. For each citation index, the distribution of journals by quartiles was made in the range Q1 to Q4. In Table 4, in addition to the quartile ranking, the relative change in the journal's position according to the different citation indexes is given, indicated by a number in brackets (n).

The number (n) denotes the relative difference between the journal's rank according to the JCR, the CiteScore, the SJR, the SNIP and its rank according to the JIF in the Mining and Mineral Processing subcategory. A negative value of the number (n) for a particular citation index indicates that the journal is placed n places lower than its JIF ranking position. The data is intended to show how citation indexes affect the rank of a journal.

3. Discussion

The obtained results should be carefully interpreted, considering all the uncertainties that resulted from the presented analysis. The inclusion of the ESCI journals brought an additional 12 items to the primary set of 20 journals, i.e. the database had been increased by 60%. The addition of the entire WoS Core Collections journals to the JCI in 2021, increased its coverage to more than 21,000 scholarly publication titles adding around 7,000 journals from the ESCI (about a 50% increase of the database), especially in some disciplines, bringing deeper regional or specialty area coverage. Consequently, the extension of the JCR in the Mining and Mineral Processing subcategory is similar to the entire database.

Moreover, in the new ranking, of the first 15 places, 13 were taken by the journals belonging to the SCIEx, and the last places 29^th-32^nd were taken by the journals from the ESCI. There is always a discussion what the ESCI really represents in terms of quality. If Figure 1 is discussed analysing the SCIEx vs. the ESCI, the four values are available for interpretation: the mean value (μ) marked also with red line, standard deviation (σ) and the percentage of journals with the Journal Citation Indicator above 1.0 and 1.5.

It is evident that all the variables favour the journals belonging to the SCIEx with mean (JCI) 0.89 (vs. 0.38 in the ESCI). Even wider standard deviations indicate that a higher percentage of the journals will be with significantly higher mean values in the SCIEx than in the ESCI, which is demonstrated with 10.5% of the SCIEx journals with the JCI>1.5 (vs. only 1.7 % in the ESCI). Due to lower achievements, the impact of the ESCI is significantly lower and this is the main reason why ESCI’s journals mostly cannot fulfil the last criterion of the Clarivate editorial evaluation during the journal assessment – the impact. The sub-criteria of the impact are defined to select “the most influential” journals in each scientific field and move them toward the SCIEx. In last few years the Clarivate policy was to move a given journal from the ESCI to the SCIEx if it reaches Q2 in provisionally calculated JIF, so the Q2 was the measure of the impact. That put citation activity as the primary indicator of the impact [10], with the journals being continuously monitored and re-evaluated when the citations indicate that the (citation) impact can be reached.

The criteria, based on “impact”, are designed to select the most influential journals in a given field of research, using the citation activity as the primary indicator of the impact. If a journal does not pass this step, its performance will be monitored. The journals are periodically re-evaluated when the citation activity indicates that the impact criteria may be met. The introduction of the JIF for all the journals in WoS CC gives rise to the question of the significance of maintaining the two databases, the SCIEx and the ESCI, bearing in mind that until June 2023, the JIF was exclusively reserved for the SCIEx. Namely, Clarivate currently uses a single set of 28 criteria to evaluate journals, including the 24 quality criteria designed for editorial rigor and the best practice, and four impact criteria designed to select the most impactful journals in their fields. Journals that meet the quality criteria enter the ESCI while the journals that meet the additional impact criteria enter the SCIE, the SSCI, and the AHCI. This criterium is furthermore divided in (observing) of the following sub-criteria [10]:

• Comparative Citation Analysis – counting citations of the journals from the most selective databases (SCIEx, SSCI and AHCI) in given field(s), where the number and sources of citations as well as the stability of citation activity is observed;
• Author Citation Analysis – following journals’ publication history in the WoS and citation networks, especially in the journal’s category;
• Editorial Board Citation Analysis – analysing board members’ publication history in the WoS and citation networks, especially in the journal’s category;
• Content Significance – it must be of interest and importance intended for the WoS readership and subscribers.

The complexly answered dilemma (“which came first - the chicken or the egg?”) remains unanswered. Did the ESCI journals collect lesser citations and consequently mostly reached (in 2022) the Q3/Q4 because they were not listed in the JCR and attract the attention of the authors? Staying non-attractive for the long active and funds-seeking large researching groups, the ESCI journals just could not compete with most of the SCIEx journals, especially those in the Q1/Q2. Eventually, they did not meet the impact criteria of the Clarivate selection committee and the “circle is closed”.

On average, the journals have a higher CiteScore (4.59) than the JIF (2.53). The average Scopus based impact factor is 88.4% higher than the one based on the WoS data. Although other authors have also reported higher CiteScore than the JIF values [19], such a significant discrepancy between Scopus and WoS requires a more detailed analysis. It should be noted that only one subcategory of Engineering category in WoS, which contains 32 journals, was considered. On the other hand, the difference between average values of the JCI and the SJR is minimal (Table 5).

Based on the Pearson correlation coefficient, a significant bias was found in the ranking of journals according to the WoS and Scopus citation indexes. As expected, the highest dependencies were found between the citation indexes of the same database, for JIF-JCI R2 0.967 and SJR-CiteScore R2 0.967 (Table 6). According to Torres-Salinas et al. (2022) [20], the correlation coefficient between the JCI and the JIF is 0.904 in the case of Science journals and 0.857 in the case of Social Sciences journals. It was also found that in the Engineering category, the Pearson coefficient is between 0.95 and 1, which is also true for the Mining and Mineral Processing subcategory in Engineering. Moreover, a significant dependence was found between the JIF and the Scopus-based citation indexes such as CiteScore, SJR and SNIP.

It is also noteworthy to analyze how different citation indices affect the ranking of quartiles of journals within the same category. To investigate this, all journals in the Mining and Mineral Processing subcategory were ranked according to each citation index and divided accordingly into the quartiles shown in Table 4. Regardless of the citation index, almost half of the journals in this category (47%) did not change their quartile rank. For other journals, there was a change within one quartile, while a change of two quartiles was only observed for the two journals. For both journals, there was a significant difference in ranking between the JIF and CiteScore, by 11 and 12 places respectively. When comparing the ranks according to the JIF and the SJR, the difference is even greater, leading to a change in rank of 20 places.

To determine the mutual influence of the index indicators on the ranking of the journals, the Spearman's rank coefficient was used. Table 6 shows that the greatest dependence between the index indicators is from the same databases (JCI-IF, Citescore-SJR). It is interesting to note that there is a significant dependency between the JIF and CiteScore with a Spearman’s rank coefficient of 0.90 and between the SJR and the JCI with a somewhat lower coefficient of 0.80. This can be taken as a strong argument in favour of considering both citation databases as equally legitimate for assessing journal impact factors.

Based on the provided analyses, potential areas for future research can be identified. These areas can help further explore and understand the impact of different citation indices on journals’ rankings and provide valuable insights for the researchers, publishers, and the academic community. Future research could investigate the reliability and validity of different citation indices. This could involve assessing the accuracy of the rankings and their correlation with the journals’ quality, as perceived by experts in the respective fields. Such research could help answer the questions which indices are more reliable indicators of the journals’ impact. So, future research in this area can contribute to a deeper understanding of how citation indices affect journal rankings and their broader implications for academia. Such directions can help guide academic institutions, researchers, and publishers in making informed decisions and promote transparency and fairness in the evaluation of scholarly work.

4. Conclusions

Based on the completed analyses, conclusions were formulated in the following points:

• The inclusion of the ESCI journals in the JCR, merging with the SCIEx ones is a step in the right direction and, for sure, will increase of the quality of journals’ content in the WoS. However, this action needs to be evaluated using both citation indicators – the JIF and the JCI.
• Most of the added journals fit into the Q3/Q4. Their JCI’s values were significantly lower before the inclusion, and the same can be assumed as valid for the provisional JIF calculated for most of them in the past.
• That is also proven by the date given in Table 2, from which it is evident that only 3 (of 12) journals, or merely 25%, would be listed in the JCR 2022 in the Q1/Q2 in the case that Clarivate decided to rank the ESCI journals last year. A similar outcome can probably be expected in the JCR 2023.
• Based on the Pearson and Spearman rank correlation coefficient, a significant bias was found in the ranking of journals according to the WoS and Scopus citation indexes.
• Under the subcategory Mining & Mineral Processing, 32 journals are listed in WoS, of which 31 are indexed in Scopus under the subcategory Geotechnical Engineering and Engineering Geology. Looking at the ranking of the journals in the Mining & Mineral Processing subcategory according to the Scopus index indicators, it can be seen that almost half of the journals (47%) have not changed their quartile rank. For other journals there was a change within one quartile, while a change of two quartiles was only observed for two journals.
• Looking at the data of the WoS and the Scopus citation index, it is reasonable to conclude that both bases are equally relevant in the evaluation and ranking of journals, even though the rankings of a few journals changed significantly.