The Intersection of Health Literacy and Public Health: A Machine Learning-Enhanced Bibliometric Investigation

1. Introduction

Health literacy, as put by the [1] and by [2], has had many meanings throughout time with different emphasis and varying levels of specificity. In this article, we approach the field of health literacy connected to public health, aiming to identify key trends in the literature and assess the development of academic research in the area. We have opted to implement bibliometric analysis for this study to show how the field has advanced through time, how authors have networked, and emerging trends.

We have chosen to pair health literacy with terms related to public health as a way to focus our analysis on the social impact of health literacy, looking at articles that articulate health literacy with its societal impacts. This was done through filters applied in the queries to limit the scope of the corpus obtained.

However, it is important to note that health literacy is a broad concept with different meanings, as put by [2]. It refer to different ways that an individual deals with their health and health information, with functional health literacy, interactive health literacy and critical health literacy all measuring different aspects of the concept [3,4,5]. The functional health literacy dealing with basic understanding of information, interactive health literacy measuring the capacity to extract information and interpret different forms of communication and critical health literacy referring to the capacity to critically analyze health information and to use it to obtain greter agency over one’s life.

This scenario exacerbated the importance of providing proper guidance and information to the population to solve public health problems. However, the relationship between health literacy and public health is relevant outside of a health crisis and the infodemic scenario. Health literacy focuses on the capacity of individuals to access and understand health information and services [2], which makes it possible for them to make appropriate decisions regarding their health [6]. The importance of health literacy, however, is broader than the individual level, as low health literacy is associated with higher mortality [7], increased hospitalizations [8], lower vaccination rates [9], which results in higher health care costs[10], and lower productivity [2].

Despite being usually thought of as something beneficial, it is important to point out potential negative aspects of health literacy. There is often an over-emphasis of the individual, be it their own capacity to self-manage, without considering social networks and different sources of structural social support [11]. This higher emphasis on the individual can also lead to problems given that overconfidence in self-management capabilities can lead to individuals not seeking professional medical attention when needed [12]. High health literacy can also have unintended consequences such as a higher critical health literacy leading to vaccination hesitancy [13].

We have applied bibliometric analysis technique, through the use of the Bibliometrix package on R [14,15] to determine relevant authors, journals and topics inside the corpus of articles that we have selected. These aspects are capable to delineate the effects of the pandemic and its consequences on the literature.

We have also used machine learning in order to selection to identify the most important keywords to predict the paper’s number of citations per year. This approach was also implemented by [16] in the Interbank Financial Networks literature and provides relevant insight on the best practices to those aiming to publish in the area.

Through this article, we hope to shed light on the state-of-the-art scientific research on health literacy and its association with public health as well as showing most relevant themes and keywords on the area and stimulate further research.

2. Materials and Methods

2.1. Data

We obtained the data used in this article from the Scopus and Web of Science and Pubmed databases. We employ as queries "health literacy" along with terms relating to public health with the results restricted to academic articles in English. We opted for this query to select articles that focused on the broader impact of health literacy and its positive effects on society.

We can observe the specific queries used in each database below:

• Scopus: TITLE-ABS-KEY ALL=("Health Literacy" AND ("Public Health" OR "Health Care Policy" OR "Health Services" OR "Health Care Quality" OR "Health Policy" OR "Health Promotion" OR "Public Health Service")) AND ( EXCLUDE ( PUBYEAR,2023) ) AND ( LIMIT-TO ( DOCTYPE,"ar" ) ) AND ( LIMIT-TO ( SRCTYPE,"j" ) ) AND ( LIMIT-TO ( LANGUAGE,"English" ))
• Web of Science: ALL=("Health Literacy" AND ("Public Health" OR "Health Care Policy" OR "Health Services" OR "Health Care Quality" OR "Health Policy" OR "Health Promotion" OR "Public Health Service")) and 2023 (Exclude – Publication Years) and Article (Document Types) and English (Languages) and Article (Document Types)
• Pubmed: Search: ("Health Literacy" AND ("Public Health" OR "Health Care Policy" OR "Health Services" OR "Health Care Quality" OR "Health Policy" OR "Health Promotion" OR "Public Health Service")) Filters: Abstract, English, from 1992 - 2022

These queries yielded a total of 9925 articles, once the duplicates were removed. The Scopus database resulted in 5505 articles, and the Web of Science resulted in 5150 articles and the Pubmed resulted in 7102.

2.2. Methodology

2.2.1. Citation likelihood

We also applied a white-box linear regression estimated with OLS, similar to the one employed by [16], using the output of Random Forest Algorithm applied to dummy variables representing the the presence of keywords in order to predict the average citations per year:

$$y_i=\alpha +{\beta }_{1}Age+{\beta }_{2}SingleAuthore{d}_i+{\beta }_{3}QtyAuthor{s}_i+{\beta }_{4}Keyword{s}_i+ϵ$$

(1)

Where ’i’ refers to the paper’s ID, ’$y_i$’ is the average citations per year of the paper ’i’, ’${\beta }_1$’ refers to the age of the paper,’${\beta }_2$’ is a dummy variable representing whether paper ’i’ was written by a single author ’${\beta }_3$’ is the number of authors in the paper ’i’, and ’${\beta }_4$’ are dummy variables that represent whether each of the top 20 keywords for predicting average citations per year, as estimated by the Random Forest model, are present in paper ’i’. We use robust error clustering at the paper level and show a version with fixed effects for the age of the paper. This way, we can make our model robust for unobserved aspects regarding individual differences amidst the papers that could impact the dependent variable, and we can also show the model taking the age variable into account and controlling for it.

2.2.2. Bibliometric

This study applies bibliometric analysis to the data gathered from Scopus and Web of Science regarding the literature on health literacy. In order to evaluate the state-of-the-art on the theme, we have employed the Bibliometrix 4.1.2 package on the statistical programming language R.

The bibliometric approach allows for a reproducible, systematic, and transparent study [14,15]. More specifically, we have used the functions of the Bibliometrix package to explore the peculiarities of the vast corpus in question. This package allows the charting of descriptive data regarding the scientific production on the chosen topic and other bibliometric methods such as Lotka’s Law.

Lotka’s Law refers to a mathematical model capable of measuring the productivity of authors, assessing the contribution of different researchers to the progress of science, and evaluating the distribution of scientific production [17].

According to [17], the number of authors who make ’n’ contributions in a specific field of scientific knowledge is approximately $1/n^2$ of those who make only one. Lotka’s Law can be formally represented as follows [17]:

$$x^{n}y=const$$

(2)

where ’y’ is the frequency of authors who have published ’x’ amount of articles, ’n’ represents the degree of inequality in the distribution of productivity, and "const" represents a constant value that remains the same as x and y vary, being the total amount of articles observed.

We apply this formula to quantify the distribution of scientific production in a specific field. Our main aim in applying Lotka’s Law is to determine how many researchers are highly productive in the health literacy area and how many have published a low number of articles in this specific area.

3. Results

The term health literacy gained popularity in the decade of 1990 [18] and has been more researched as time went on. This can also be observed in the case of health literacy being studied with a focus on public health, as can be seen in Figure 1.

Figure 1 shows how the research on health literacy and public health has been expanding. This growth has been more pronounced in recent years. The mean amount of articles published from 2000 to 2010 being 67.91 and from 2011 to 2022 that mean grows to 773.92. In 2022 alone 1735 articles have been published on the theme.

3.1. Citation likelihood

For our Random Forest Model, we began by dividing the papers in two sets, a testing set with 20 % of papers and a training set with the remaining 80%. We have applied the model selection procedure to the training set to determine the optimal parameters that maximize the Root Mean Squared Error (RMSE),a metric known for its sensitivity to large errors and ability to provide a direct interpretation in the context of the original scale of the data [19,20]. With the optimal parameters we have estimated the model on the testing set.

For the first step of selecting the parameters, we have, similarly to [16], defined the amount of trees as 500 and then we have tested the resulting RMSE for each mtry, as can be seen on Figure 2, which exhibits the results of a repeated k-fold cross-validation procedure. The mtry selected was 4, as it was the value that minimized the RMSE.

The trained model was then used to identify the most important keywords to predict high or low number of average citations per year. The top 20 keywords were then selected as the $Keywor{d}_i$ variable for the estimation of the model of Eq. 1. The resulting coefficients are presented in Table 1, with both the model presenting $Ag{e}_i$ as an independent variable, on the first column, and the model using fixed effects to control for the age of the paper being estimated, on the second column.

On Table 1 first coefficient, on the first column, shows that the age of the paper is relevant to its average citations per year, as can be expected given that established papers will be cited and through its citations will be read by more people and also due to the fact that seminal authors will be widely cited in the literature.

Regarding the effect of the amount of authors on the citations, unlike the results found by [16] on the Interbank Financial Networks literature, whether the paper is single authored is not a significant predictor of citations. However, the amount of authors has a positive effect, which indicates that collaboration amidst authors tends to yield positive results in the area of health literacy.

Now regarding the keywords being analyzed, eight of them show a positive and significant coefficient, with three of those having the p-value above 0.05. Those keywords are: Article, Cross sectional study, Questionnaire, Male, Behavior (with a p-value > 0.01), Covid 19, Public health (with a p-value > 0.05) and Mental health (with a p-value > 0.05).

It is note worthy that the Covid 19 keyword has shown the highest coefficient in both models, despite being a relatively recent phenomena. This makes sense due to the fact that the Covid-19 pandemic and the public health crisis subsequent to it was an event closely related to health literacy as the lack of information and guidelines was a big problem, especially in the first moments of the crisis. Many people were overwhelmed by both accurate and inaccurate information, which were difficult to distinguish apart, especially given the unfamiliarity of the situation [21,22,23,24].

Other keywords with a positive coefficient that can give insight on the literature is the ’Questionnaire’ and the ’Cross sectional study’ which indicates that this literature cites empirical studies more frequently. Especially due to the fact that health literacy is often measured using questionnaires, such as the Health Literacy Questionnaire [25], the European Health Literacy Survey Questionnaire [26,27] and the Mental Health Literacy Scale [28].

Four keywords show a significant and negative coefficient. Those keywords are: ’Human’, ’Health knowledge attitudes practice’, ’Female’ and ’Surveys and questionnaires’.

It is interesting to note that articles with the ’Male’ keywords had more citations than those with the ’Female’ keyword, indicating a possible bias in the scientific community. Another noteworthy aspect here is the positive coefficient for the ’Questionnaire’ and negative coefficient for the ’Surveys and questionnaires’ keyword, which could indicate that the first keyword is more relevant due to it being more rigorous on the instrument being used, despite the fact that many surveys use questionnaires in them.

Regarding the model itself, due to the nature of citation, there are aspects many of which can’t be easily captured or quantified in a model, such as whether the authors are established authors on the literature, or even the quality of the article itself. This nature of the data being predicted explains the relatively low $R^2$ of the model.

3.2. Sources

This expansion of research on the topic also co-occurs with a significant change in the dynamics of prominent journals. In Figure 3, the sudden growth of the International Journal of Environmental Research and Public Health is clear from 2017 onward, reaching more than 500 articles published and being the most relevant source in the area by 2022. The Patient Education and Counseling was the most important source from 2006 to 2017 being surpassed by the BMC Public Health from 2018 to 2019, which was then surpassed by the International Journal of Environmental Research and Public Health in 2020. The International Journal of Environmental Research and Public Health presented over 600 articles published in the topic by 2022. We can also see the recent growth in the case of BMJ Open, which surpassed both Plos One and the Patient Education and Counseling 2022. By 2022 the most important sources were, in descending order, the International Journal of Environmental Research and Public Health, BMC Public Health, BMJ Open, Patient Education and Counseling and Plos One, with all of them, except for Plos One, having more than 200 articles on the theme.

3.3. Authors

We have used the functions of bibliometrix [14,15] to identify the most prolific and most cited authors in the field. We have also examined the distribution of the production of these authors throughout time.

For this section, author disambiguation was done manually as a way to prevent erroneous representation of authors with similar initials and surnames.

Regarding the authors researching the topic, Figure 4 shows the most prolific authors. Here we can see the authors with the highest amount of publications in our corpus. All of those authors range from 118 to 39 articles.

The highest five are Kirsten McCaffery, Michael Wolf, Richard Osborne, Danielle Muscat, Anthony Jorm. Kirsten McCaffery is a prolific author discussing themes such as over diagnosis and patient empowerment [29,30,31,32,33]. Michael Wolf is a researcher that focuses on health literacy and its impact in treatment adherence and decision making [34,35,36,37]. Richard Osborne is a researcher known for the development of the Health Literacy Questionnaire (HLQ) [25] and is active on several other empirical articles on health literacy [38,39,40,41]. Danielle Muscat is a researcher focused on health literacy and socially disadvantaged populations [42,43,44]. Anthony Jorm is one of the precursors of the research on mental health literacy [45,46], discussing problems like stigma [47] and being a reference on the development of instruments measuring mental health literacy such as the Mental Health Literacy Scale[28].

In Figure 5, we can see how the production of the most prolific authors has been distributed over time. From this figure, it is possible to see that all the authors were still active in 2022. The authors that have been publishing in this field the longest are Anthony Jorm, which started his publications on the theme by 1997, Don Nutbeam, which had his first publication on the area by 2000 and Dean Schillinger who published in 2001.

Another essential aspect we exhibit in Figure 6 is the authors with the highest impact, as measured by their H index. Comparing Figure 5 with Figure 6 we can see that the authors with the highest h-index are also the ones that have started publishing on the theme for the longest time. The fact that Anthony Jorm, Dean Schillinger, Michael Wolf and Richard Osborne were the four authors with a higher h-index and were also the ones that have been producing articles on the theme for a longer amount of time, as shown in Figure 5, indicate this correlation as a possible explanation.

A large portion of the authors in Figure 6 and Figure 4 are responsible for helping the development of health literacy assessment methodologies. Richard Osborne has took part in developing Health Literacy Questionnaire (HLQ) to assess patient-reported outcomes related to health literacy [48]. Michael Wolf has worked on the Rapid Estimate of Adult Literacy in Medicine (REALM) [49,50]. Orkan Okan focused on the adaptation of the European Health Literacy Survey (HLS-EU) to children [51,52,53]. Anthony Jorm has focused on different tools to evaluate mental health literacy and dementia [45,54,55].

We have applied Lotka’s Law to evaluate how the amount of publications on health literacy is divided among authors. The graph illustrating the curve of Lotka’s Law can be seen in Figure 7. The results show that 75.2 % of authors have published just one article on the theme, 13.4 % have published two articles 11.4 % have published three or more articles. This shows the few prolific authors on the theme of health literacy, with a majority of authors having few contributions.

3.4. Region

Another important aspect of the literature is the countries where the topic is being researched. Here we have mapped the countries of the corresponding author of each paper. It’s important to note that our corpus consists only of English articles, which reduces the amount of articles on countries where English is not the native language.

Figure 8 shows a map where each country with articles is shaded blue, the intensity of the color represents the amount of articles published by corresponding authors in that country. The countries with the most articles are the USA, China and Australia. In Europe the countries with most publications are the United Kingdom and Germany, in South America Brazil has the most articles, in Africa the most prolific country is South Africa.

On Figure 9 the five countries with the highest amount of publications, the USA, Australia, China, Canada and Germany, are shown with the amount of articles by year. The USA is an outlier with more than 8400 articles published by 2022. Australia shows over 3500 articles by 2022, while China, Germany and the United Kingdom have under 1900. This shows how the production of scientific articles on theme is concentrated in a few specific prolific countries. Another aspect that can be seen is the recent growth of China, which became the third country with most articles in 2021.

3.5. Topics

In order to evaluate the topics being studied by the authors, we have opted to analyze the author’s keywords. We present the most frequent keywords in Figure 10. Here we can see mostly terms related to the groups being studied, such as Male or Female and Adult, Adolescent, Middle Aged and Aged. We can also see Surveys and Questionnaires, as those are the most common health literacy measuring tools.

Figure 11, shows the time frame that each topic has been primarily addressed. Here it is possible to visualize recent topics such as Covid-19, Vaccine Hesitancy, Social Media, Digital Health Literacy and Mental Health Literacy arising. It is also possible to notice how some of the topics have started receiving less attention, such as those regarding organization and administration of health facilities.

Covid-19 and vaccination hesitancy are topics that have gained attention lately due to the pandemic and infodemic situation that increased vaccination hesitancy around the world[56]. Another topic that relates to the pandemic and co-occurring infodemic are the ones of social media [23,57] and mental health [58,59,60].

We have also opted to visualize the most frequent words used in the articles’ titles, analyzing them isolated from other adjacent terms (as unigrams). This is another way to look into most studied topic that might not have been indexed as keywords.

Figure 12 shows us some specific terms that indicate aspects being studied by health literacy researchers. Covid and Mental show here as well as on Figure 11, indicating that while being more recent topics of study, they already show on a substantial amount of articles. Cancer also shows as a theme being present in many article titles, which goes in line to research showing that health literacy is important in order to promote cancer screening [61] and adequate patient decision making and treatment comprehension, which can lead to better outcomes [62].

4. Discussion

This research used the bibliometric analysis methods present in bibliometrix to look at the emerging trends and patterns in the scientific production regarding health literacy. Our analysis sets itself apart due to its use of the Random Forest technique to estimate the impact of keywords on average citations per year and the focus on public health.

Other studies using bibliometric methods have been made on health literacy but with different focuses such as the academic production on the theme of health education and health literacy [63]. Or with regional emphasis, focusing on the studies done on the theme in Europe [64] that also find a predominance of the USA on the theme by 2008 and decides to focus on the specificity of Europe. There is also the systematic review of the health literacy measurement instruments coupled with bibliometric techniques by [65].

We have identified that health literacy as a field has grown with exceptional intensity in the last few years. The themes of COVID-19, mental health and social media are relevant to this expansion. Another evident aspect was the prominence of the International Journal of Environmental Research and Public Health in the latest years, similarly to what was found in the case of the field of health literacy and health education [63].

These recent changes in the field, coupled with the fact that all of the most prolific authors are still active, show that this is a growing theme that is expanding and changing focus as time progresses. We have also found that, when looking at the authors with the highest H index, a number of those authors that have developed or adapted tools and methodologies for evaluating health literacy, which then were incorporated in other researches.

Using the Random Forest Model we have estimated, through the use of OLS regression exhibited in Eq. 1, we have shown that, as expected, the age of the article is relevant for its average citation per year. That the amount of authors is also relevant for that, with a positive relation to the citations. Another aspect that we have shown is that ’Covid-19’ has a significant impact in citations despite being a recent topic of study. We have also found a positive coefficient for the ’Male’ keyword and a negative one for the ’Female’ keyword, that indicates a possible bias in the literature. We have also found a positive coefficient for the ’Questionnaire’ and the ’Cross sectional study’ keywords, that indicates an interest in empirical studies.

The model we have used, however, does not take into account other important aspects. Our findings presented in Figure 6 show that the authors involved in the creation and adaption of health literacy evaluation tools receive more citations. This serves to point that there are other qualitative variables that account for the citations per year that are not possible of being modelled.

Further studies can be made using different corpora, tailoring them for each corpus to encapsulate different dimensions of health literacy, be it the type of health literacy, such as functional, interactive or critical [3,4,5], or specific themes, such as mental health literacy or digital health literacy [28,66]. Our research also indicated a possible gender bias in the literature, future research focusing on the gender divide in the literature is warranted in order to properly understand this issue.

5. Conclusions

The field of health literacy and public health has grown in the number of publications in the last two decades, with the bulk of its growth happening since 2015. However, we have also shown a change in the most studied topics. This is an ongoing change, and it remains to be seen how this field of study will grow with time if the new emerging themes and authors will become the most cited ones.

For the time being, the themes related to Covid-19 remain highly researched topics. We have also found a gender discrepancy in citation likelihood, with the keyword ’Female’ having a negative coefficient and the keyword ’Male’ having a positive one, which indicates a possible bias in the literature. Empirical studies with the keywords ’Questionnaire’ or ’Cross Sectional Studies’ have also shown a positive coefficient for annual citation.

We have also found a recent change in the most relevant journals on the theme, with the International Journal of Environmental Research and Public Health surging in the amount of articles published from 2018 onward, being in 2022 the most prolific journal in the literature. We have also seen a growth in production from China in the last decade, being the third most productive country, behind the USA and Australia. These changes, coupled with the new themes rising indicates that the literature is evolving, incorporating new topics with the entrance of new authors, reflecting the growth and diversification of the field.

This article provided a state of the art of the field of the intersection between health literacy and public health, and introduces a predictive model spotlighting the most pertinent keywords. This reflects the themes considered relevant by the literature and also offers potential guidelines for authors in the field. However, like all fields, how the observed dynamics and the immense impact of the Covid-19 pandemic will persist throughout time remains to be seen.