Open Government Data Topic Modelling and Taxonomy Development

1. Introduction

1.1. Background and Motivation

Open Government Data (OGD) constitutes a vital source of publicly released data originating from the government sector. The primary goals of OGD are to foster transparency, enhance accountability, and create added value [1]. In recent years, the production and analysis of data by public sector organizations have expanded considerably, resulting in increased research attention on OGD [2,3,4,5,6]. As government-generated data continues to grow, efforts to make it accessible to the public remain a priority. Initially, the emphasis was on promoting transparency in governance [7]. However, since 2010, expectations surrounding open data have evolved significantly [8,9]. Today, OGD is valued not just for its role in transparency but also for its potential to drive innovation [10], foster a data-driven economy [11], and contribute to societal goals. Achieving these objectives hinges on the release of high-impact data to the public—a priority highlighted by Ubaldi [5].

Despite its transformative potential, our understanding of the precise impacts of OGD remains limited, largely due to the scarcity of systematic and comprehensive studies in this field [12,13,14,15,16,17,18]. The Open Data Maturity Report [19] offers insights into how OGD generates impact across four domains such as governance, society, the economy, and the environment. The report illustrates that OGD can either be consumed in its raw form or processed further to create enriched datasets, innovative solutions, and value chains. However, generating impact depends on reusers finding practical applications for the data and implementing processes to transform it into actionable outcomes. Today, measuring or even identifying the impact of OGD remains a formidable challenge due to its diverse applications and the complexities involved in defining universal metrics. Additionally, the benefits of OGD are often indirect, further complicating efforts to quantify its impact [20]. The Open Data Maturity Report [19] assesses OGD impact using surveys with over 150 queries across areas like policy, impact, data portals, and quality. While such surveys provide valuable insights, they may reflect subjective perception of the respondents and are constrained by limited government resources, including funding and staffing for open data initiatives, which often compete with other high-priority projects [14]. Another resource that the EU mandates Member States to submit annually are case studies or use cases on OGD use, which offer a promising avenue for deriving actionable insights. Young and Verhulst [21] emphasize the importance of leveraging OGD use cases to understand its applications and benefits. Submissions on the Data Europa portal [22], supervised by the Publications Office of the European Union, serve as an invaluable dataset for examining the real-world impact of OGD and present an opportunity to deepen our understanding of its transformative potential. However, before classifying OGD impact areas, it is crucial to recognize that classification, as an established data mining problem, often requires preprocessing steps to make raw data suitable for training classifiers [23]. Experimental results on benchmark datasets demonstrate that integrating a taxonomy significantly improves classification accuracy while maintaining manageable computational effort, highlighting the importance of taxonomy in effectively categorizing OGD impact areas.

A key element in understanding and measuring the impact of OGD is therefore the development of a taxonomy of impact areas, akin to the approaches already utilized for mapping OGD research areas [24,25] and creating OGD solutions [26] such as interactive maps and dashboards. A taxonomy provides a structured framework for categorizing the diverse domains where OGD creates value and facilitates systematic analysis of OGD. Furthermore, a taxonomy enables the development of domain-specific metrics, guiding policymakers and researchers in designing targeted interventions to maximize OGD's benefits [27].

To address the challenges of understanding the impact of OGD, this study aims to systematically classify and assess the impact areas of OGD initiatives. Specifically, we seek to answer the following research question:

How can open government data impact areas be systematically classified using machine learning techniques and taxonomy development?

To achieve this, we employ a combination of topic modeling and taxonomy development methodologies to create a structured framework that organizes and categorizes OGD impact areas.

2. Literature Review

Our work is related to three main research pillars in OGD impact assessment: (1) benchmarking open government data, (2) OGD impact evaluation benchmarks, and (3) impact assessment with machine learning.

2.1. Benchmarking Open Government Data

In the context of OGD impact research, most efforts gravitated to benchmarking OGD practices, which involves systematically evaluating and comparing OGD initiatives, programs, or systems against established standards, metrics, or best practices. This includes identifying key indicators, collecting data, and analyzing how different governments, organizations, or regions perform relative to these benchmarks. As discovered by Hao-En [28], data quality and usage/usage patterns are the most frequently discussed topics in academic research. Zuiderwijk et al. [29] expands on this by highlighting the variety of metrics and methodologies used to benchmark OGD progress. Their research reveals that different benchmarks prioritize different aspects of OGD: some focus exclusively on data publication, such as the Open Data Index [30], Open Data Economy [31] and Open Data Inventory Network (ODIN) [32], while others examine both data publication and usage, or the potential value that could be created from the data, like the Open Data Readiness Assessment (ODRA)[33], Open Data Barometer [34], and Open Data Maturity [35] benchmarks. Zuiderwijk et al. [29] identified a significant challenge in the benchmarking process as many benchmarks fail to measure the actual impact of OGD, particularly in terms of citizen participation and the real-world collaborations between data providers and users. Furthermore, authors point out that although several newer benchmarks, such as the WJP Index and EIU, claim to assess the impact of OGD, they often do so from a theoretical perspective, assuming that impact is possible under certain conditions, rather than measuring actual outcomes. Research by Zuiderwijk et al. [29] underscores the multifaceted nature of OGD and the complexity of accurately assessing its progress and suggests that a comprehensive evaluation of OGD requires combining multiple benchmarks, each focusing on different facets, to gain a fuller understanding of open data’s impact and effectiveness. As additionally discovered by Hao-En [28] and Farhadloo et al. [36], there is a lack of benchmarking efforts specifically targeting the social and economic impacts of OGD, indicating a gap in current research.

Where impact has been observed, different and non-consolidated sources of data or methodologies have been used, neither of them followed a consolidated impact classification scheme or taxonomy. Instead, research papers usually followed impact areas defined by evaluation projects that were used as the foundation of their research [37] or extracted impact areas based on small and country specific dataset [38]. Zeleti’s [38] findings also suggest that impact categories can guide practitioners and governments in reviewing and evaluating open data initiatives. By considering both the demand and supply sides of open data, these categories help expand the focus beyond individual projects and specific user communities. The study acknowledges that while the four impact categories offer a framework for analysis, there are likely additional impact areas that were not explored due to insufficient evidence.

2.2. OGD Impact Evaluation Benchmarks

In recent years, several key global evaluation projects have been conducted to assess the impact of OGD initiatives across different levels, from municipalities to entire countries. These evaluation projects aim to measure the transparency, usability, and real-world impact of OGD, as well as its role in promoting governance, economic development, and social inclusion. Among the most significant evaluation frameworks are the Global Open Data Index [30], Open Data Barometer [34], European Open Data Maturity Report [35], Open Useful Reusable Government Data [39] and Open Data Inventory [32]. These evaluations have become one of the essential tools in understanding the maturity of open data practices across the globe.

The Global Open Data Index (GODI)[30], conducted by the Open Knowledge Foundation, focuses solely on the publication of open data by national governments. It assesses how governments make data available across six categories: Finance and Economy, Politics and Election, Spatial Data, Law, Environment, and National Statistics. However, it does not evaluate the use or impact of the data, concentrating primarily on its availability. In contrast, the Open Data Barometer (ODB) [34], conducted by the World Wide Web Foundation, takes a more comprehensive approach by including three main indexes: Readiness, Implementation, and Impacts. The ODB evaluates the political, economic, and social impacts of open data through expert surveys, which assess factors like government transparency, economic growth, and social inclusion. It uses sources from online media, academic publications, and expert input to estimate the extent of these impacts.

The European Open Data Maturity Report [35], by the Publications Office of the European Union, evaluates the openness and quality of government data across European countries. Its goal is to identify best practices and provide guidance to enhance the open data landscape in Europe. Meanwhile, the Open Useful Reusable Government Data (OURdata) [39] evaluation, led by the Organisation for Economic Co-operation and Development (OECD), focuses on the accessibility and reusability of open government data. It emphasizes fostering innovation and creating economic value by promoting the reuse of open data. Similarly, the Open Data Inventory (ODIN) [32], conducted by Open Data Watch, focuses on evaluating official statistics and the practices of national statistical offices, with a strong emphasis on improving governance and economic development.

Despite their differences, these evaluations collectively provide important insights into the political, economic, and social impacts of open data. The ODB, for example, identifies the political impact of open data in terms of government efficiency, accountability, and transparency. In the economic domain, evaluations like ODB and OURdata show that open data can drive innovation, support entrepreneurship, and contribute to economic growth. Socially, these evaluations also highlight the role of OGD in fostering environmental sustainability and social inclusion, particularly in marginalized communities.

Despite their contributions, these evaluations have limitations. For instance, GODI focuses exclusively on data publication and does not account for the broader impacts of open data on society or governance. Similarly, ODB's reliance on expert surveys and secondary sources may not fully capture the nuances of OGD’s real-world effects.

2.3. Impact Assessment – Methods Used

Recognizing the significance of OGD impact areas, particularly within major European organizations, particularly Publications Office of the European Union with ODM, we have identified an opportunity to automatically detect the areas of OGD from textual documents through text mining techniques. This process can empower governments to develop targeted and higher-quality datasets. In this context Artificial intelligence (AI) methods are highly valuable and represent a field of active research and rapid advancement [40]. Within the field of AI and Machine Learning (ML), text mining stands out, referring to the extraction of valuable and meaningful information from unstructured text [41]. This research focuses on Natural Language Processing (NLP) methods, a subset of text mining [42], with specific emphasis on keyword extraction and topic modeling.

While text mining is a common practice in literature, its application to identify OGD impact areas is relatively rare. One relevant application of NLP in this context is the work of [43], where the authors conducted topic modeling using Latent Dirichlet Allocation (LDA) to uncover research areas related to OGD and Freedom of Information (FOI). The study employed a corpus comprising journal articles, conference proceedings, and book chapters. The authors categorized the topics into four major groups: Transparency Collaboration/Participation, Technology, Economic/Social/Innovation, and Citizen Engagement. Their findings indicated a predominant focus on technology and related topics, as well as issues related to citizen engagement while only a scarce amount of research has employed the selected methodology to discern the impact of OGD in the existing literature. Tinati et al. [44] examined the impact of open data on the United Kingdom government through 22 semi-structured interviews with various stakeholders, owing to the scarcity of quantitative and qualitative data sources. Meng [45] assessed the relationship between civil society and the social impact of open data in eight Latin American countries using the Most Similar Systems method and fuzzy logic, relying on survey data. Jetzek et al. [46] measured the social and economic impact of OGD using the Partial Least Squares (PLS) method, incorporating various indicators and indexes. Their model was subsequently expanded by Jetzek et al. [47] to introduce contextual factors for sustainable value generation. Bilkova et al. [48] further extended the model to encompass additional enabling factors, along with an expanded sample size and attributes supported by the literature review from Machova & Lnenicka [49]. Their structural equation modeling (SEM) model incorporated various impact areas, including economic, educational, environmental, health, politics and legislation, social, trade, and business.

The previous research efforts in (OGD impact area assessment have provided valuable insights into various dimensions of OGD, yet they fall short in contributing to a comprehensive recognition and measurement of impact areas. While studies have explored the broader impacts of OGD, particularly in terms of transparency, citizen engagement, and economic or social benefits, they have not consistently incorporated methodologies that systematically assess and track these impacts across different OGD initiatives. A key limitation in the existing literature is the focus on isolated impact categories without a cohesive framework for capturing the full spectrum of impacts in a structured and measurable way. Existing studies, while contributing significantly to understanding OGD’s potential, have not fully addressed the need for a more systematic, scalable, and automated approach to recognizing and measuring the impact of OGD across various domains. Their reliance on limited sample sizes, qualitative methods, or context-specific data means they do not provide a comprehensive, universally applicable framework that can be used by governments to assess the true impact of OGD initiatives. Furthermore, the inability of these studies to integrate real-time data and contextual shifts within the open data ecosystem prevents them from being dynamic enough to reflect ongoing developments in open data practices.

In conclusion, while previous research has made significant contributions to understanding certain aspects of OGD impact, it has not yet provided an integrated, systematic, and scalable approach to identifying and measuring these impacts comprehensively. The existing studies, which mainly rely on qualitative methods or sector-specific models, fail to establish a universally applicable framework that can be used by governments to assess the broader and long-term effects of OGD initiatives. Furthermore, their focus on limited sample sizes, geographic contexts, or non-dynamic datasets leaves a gap in real-time impact evaluation.

The application of text mining techniques, particularly NLP, presents a promising solution to address these challenges. By leveraging AI and Machine Learning, governments can automate the analysis of textual documents to identify and evaluate OGD impact areas on a broader scale. This method offers the potential to create a more scalable, dynamic, and real-time approach to assessing OGD’s impact. To fully capitalize on this opportunity, there is a need for the development of a comprehensive taxonomy of OGD impact areas. Such a taxonomy would provide a standardized framework for categorizing and defining the various dimensions of OGD impact, allowing for more accurate, consistent, and universally applicable assessments. It would also facilitate comparative analyses and support targeted interventions to maximize the benefits of OGD initiatives across different regions and contexts. The goal of this research is, therefore, to develop such a taxonomy, which will enhance the overall evaluation and measurement of OGD’s impact and contribute to more informed policymaking and better data-driven decisions.

3. Materials and Methods

The research methodology employed in this study is rooted in Design Science Research (DSR), as described by Hevner et al. [50]. DSR is a well-established framework that focuses on the creation and evaluation of artifacts, such as models, methods, and taxonomies, to address real-world problems. The DSR framework consists of the relevancy, rigor and design cycle. While the relevancy and rigor cycle refer to problem identification, existing knowledge review and research gap identification covered in introduction, the design cycle provides the framework for artefact development and evaluation. Our research is focused on development of a taxonomy for open government data topics – the artefact, which is guided by the principles of the CRoss-Industry Standard Process for Data Mining (CRISP-DM). CRISP-DM is a six-phase methodology designed to offer a systematic and flexible approach to data mining projects and is, as outlined by Azevedo & Santos [51], widely adopted., Its comprehensive structure ensures the applicability of its methods across industries, providing a clear pathway for extracting meaningful insights from diverse data sets.

For this study, we applied the CRISP-DM framework to analyze 697 unstructured OGD use cases, sourced from the European Data Portal [22]. The one page use case documents, ranging from 431 to 1353 words, typically begin with a header that displays the title of the use case, often accompanied by relevant logos or branding elements. This section is followed by an executive summary or abstract that provides an overview of the document's purpose, context, and main outcomes. Next section then describes the challenges or needs that prompted the initiative and link the project to broader strategic goals or policies. Final sections define the objectives and a delineation of the project’s boundaries.

Given the unstructured nature of the data, our first step was to preprocess the text, ensuring it was suitable for machine learning analysis. This preprocessing involved several key tasks, including data extraction, cleaning, and normalization, as outlined in the following section. Following the data preprocessing phase, we employed topic modeling techniques to uncover the thematic structure within use case documents. Specifically, we utilized five distinct machine learning methods for topic modeling, each offering unique advantages in identifying latent patterns within the text. These methods allowed us to capture the diversity of topics and group them into coherent themes.

It is important to note that while our approach utilizes the CRISP-DM framework to systematically extract topics from unstructured data, the development of the taxonomy itself is not a direct outcome of CRISP-DM. Considering that the primary artifact of this study is the taxonomy, the design cycle also includes a specific methodology dedicated to taxonomy development by Nickerson et al. [52]. In this context, based on the insights derived from the topic modeling analysis, we iteratively developed and refined a comprehensive taxonomy of OGD topics, which serves as an organized classification system, foundational for understanding the impact areas within the OGD domain and supports further machine learning-driven analysis of OGD use cases.

The entire CRISP-DM framework process, results, and the detailed methodology and construction of the taxonomy are further explained in sections that follow.

3.1. Data Extraction and Pre-Processing

Unstructured text constitutes a prevalent form of data and, indeed, may constitute a substantial portion of the information accessible to a given research or data mining project [53]. When extracting data from unstructured documents, it was imperative to ensure the versatility of our code to handle various document formats beyond PDFs and this is why we utilized the Textract package [54], which provides a single interface for extracting content from any type of file, without any irrelevant markup.

After data extraction, a preprocessing pipeline was also developed and executed. This involved converting the entire corpus to lowercase, lemmatization, removal of stop words, and elimination of noise. Stop words were obtained from the Natural Language Toolkit (NLTK) library [55] and a customized noise removal function was applied. This function systematically processed incoming documents, eliminating special characters, punctuation, numerical values, URLs, email addresses, and redundant white spaces through regular expressions. Finally, a process of text normalization was implemented, comprising sentence tokenization, word frequency calculation using NLTK library, identification of near-identical words, and the removal of redundant words from the text.

3.2. Topic Modelling

Topic modeling, a statistical modeling approach, leverages unsupervised Machine Learning to discern clusters or groups of comparable words within a text corpus. This method in text mining utilizes semantic structures within the text to comprehend un-structured data without the need for predefined tags or training data [56].

In this study, five distinct machine learning methods for topic modeling were employed, ranging from modern and sophisticated approaches like OpenAI's Chat GPT to more established and simpler methods such as Latent Dirichlet Analysis (LDA). Detailed descriptions and parameterizations for each method are presented in the following sections.

3.2.1. Generative Pre-Trained Transformer

We first applied topic modeling using the OpenAI Generative Pre-Trained Transformer (GPT) model [57]. These models demonstrate the ability to produce coherent and contextually relevant responses when presented with a prompt or input text [58]. GPT’s extensive pre-training on large text datasets enables it to understand complex language patterns and generate accurate responses, even in ambiguous contexts [59]. The GPT model is based on a transformer architecture that includes an encoder, decoder, feed-forward network, and cross-attention layer [60]. The encoder consists of multi-headed self-attention layers that process the input sequence, passing information between layers through interconnected blocks [61]. The decoder also uses multi-headed self-attention layers, focusing on generating the output sequence. Attention, first proposed by Bahdanau et al. [62], involves computing a context vector for each decoder step, which captures the most relevant information from all encoder states using a weighted average. The contribution of each encoder state is based on its alignment score, which measures its relevance to the previous decoder state. Self-attention applies this mechanism to every position in the input sequence, generating three vectors: query, key, and value; at each position. Using the query vector, the attention mechanism transforms the input sequence into a new sequence, with each element incorporating both the original input and its contextual relationship with other positions. These computations are performed across the entire sequence by grouping the queries, keys, and values into matrices [60]:

$$Attention\left(Q,K,V\right)=softmax\left({\displaystyle \frac{{QK}^T}{\sqrt{d_k}}}\right)V,$$

(1)

GPT’s transformer architecture also includes a feed-forward neural network (FFNN) that processes both input and output sequences. This network consists of several layers of interconnected neurons that allow information to flow through the model. At its core, the feed-forward network takes input values, multiplies them by corresponding weights, adds a bias term, and applies an activation function to produce the output. Additionally, a cross-attention layer connects the encoder and decoder. After the FFNN processes the input, the decoder’s multi-head attention block applies the same tokenization, word embedding, and attention mechanisms to generate attention vectors, which map the relationship between the source and target text. This information is then processed through another feed-forward layer to produce the final output.

We utilized the text-davinci-003 and gpt-3.5-turbo variants of GPT-3.5 model and during experimentation, we explored different temperature settings, which define how creative (higher value) or deterministic (lower value) the answer is. We specifically tested values of 0.2 and 0.7, in accordance with the recommendations provided in the OpenAI documentation [63]. The prompt, which is required for both variants we used was: “What domain is this text about in five keywords:“.

3.2.2. Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) is a technique used to factorize a non-negative matrix X into two non-negative matrices, W and H, with the aim of approximating X by their product [64]. As per Lee & Seung [64], NMF is also a recognized mathematical method for dimensionality reduction.

$$X\approx WH,$$

(2)

NMF has been widely recognized as a powerful tool for topic modeling. Studies by Carbonetto [65] and Purpura [66] have demonstrated its potential in enhancing topic model performance, while Lee & Seung [64] originally proposed NMF as a method for uncovering latent structures, including its application to topic modeling.

In the code we created, we performed topic modeling using a Term Frequency-Inverse Document Frequency (TF-IDF) vectorizer, which converts a set of documents into a matrix of TF-IDF features. Non-negative matrix factorization was then applied to extract topics, with the number of topics set to 1 and the number of words per topic limited to 5.

3.2.3. Latent Dirichlet Allocation

Latent Dirichlet Allocation (LDA) is a generative probabilistic model commonly used in natural language processing to uncover latent topics within a collection of documents. Developed by Blei et al. [67], LDA provides a framework for understanding the thematic structure of large text datasets. LDA assumes that each document is represented as a probability distribution over latent topics, with a shared Dirichlet prior for all documents. Similarly, each topic is modeled as a probability distribution over words, also governed by a Dirichlet prior.

As described by Blei et al. [67], for a corpus D of M documents, each with Nd words (d ∈ {1, ..., M}), LDA's generative process involves the following steps:

• Choosing a multinomial distribution ϕ_t for topic t(t ∈ {1, ..., T }) from a Dirichlet distribution with parameter β;
• Choosing a multinomial distribution θ_d for document d(d ∈ {1, …, M}) from a Dirichlet distribution with parameter α;
• For a word w_n (n ∈ {1, …, N_d}) in document d, select a topic z_n from θ_d and select a word w_n from ϕ_zn.

In this generative process, words in documents are observed variables, while other variables (such as ϕ and θ) and hyperparameters (α and β) are latent. The probability of the observed data D is computed for a corpus as follows:

$$p\left(D|\mathsf{\alpha },\mathsf{\beta }\right)=\prod _{d=1}^M\int p\left({\theta }_d\right|\mathsf{\alpha })\left(\prod _{n=1}^{N_d}\sum _{z_{dn}}p\left(z_{dn}|{\theta }_d\right)p\left(w_{dn}|z_{dn},\beta \right)\right)d{\theta }_d,$$

(3)

Several studies have highlighted LDA's effectiveness in topic modeling. Ostrowski [68] demonstrated its utility in identifying sub-topics and classifying Twitter data. Christy et al. [69] proposed a hybrid model using LDA for feature extraction and selection, improving document clustering accuracy. Muchene et al. [70] employed a two-stage approach, combining LDA for per-document topic probabilities with hierarchical clustering for final topic clusters in scientific publications.

In our code, we tokenized individual documents, created a dictionary of tokenized words, and generated a bag-of-words representation. An LDA model was then instantiated with a predetermined number of topics, set to 1 in our instance, with the number of words per topic limited to 5.

3.2.4. Hierarchical Dirichlet Process

The Hierarchical Dirichlet Process (HDP), proposed by Teh et al. [71], is an extension of the Dirichlet Process that provides a flexible framework for hierarchical Bayesian modeling. Unlike models that require the number of topics or clusters to be set in advance, HDP adapts to the data and automatically determines the number of topics. In HDP, random probability measures G_j are assigned to each group, and these are derived from a global random probability measure G₀, which is sampled from a Dirichlet process with concentration parameter γ and base probability measure H:

$$G_0|\mathsf{\gamma },\mathrm{H}~\mathrm{D}\mathrm{P}(\mathsf{\gamma },\mathrm{H}),$$

(4)

Each G_j is then conditionally independent given G₀ and follows another Dirichlet process with concentration parameter α₀:

$$G_j|{\alpha }_0,G_0~\mathrm{D}\mathrm{P}({\alpha }_0,G_0),$$

(5)

The hyperparameters include the baseline measure H and the concentration parameters γ and α₀​. H acts as a prior for the parameters θ_ji ​, while the variability around G₀​ is controlled by γ, and the deviations of G_j from G₀ are governed by α₀​. In some cases, different concentration parameters α_j​ can be used for different groups to account for varying levels of variability. HDP is typically used as a prior distribution for grouped data, with each observation x_ji corresponding to a parameter θ_j2​, sampled from G_j​:

$$\begin{gathered} {\theta }_{ji}|G_j~G_j \\ {x}_{ji}|{\theta }_{ji}~F\left({\theta }_{ji}\right), \end{gathered}$$

(6)

HDP is preferred over LDA for several reasons. First, HDP can automatically infer the number of topics, addressing a major limitation of LDA [72] Second, HDP is more flexible and can incorporate supervision [72,73]. Lastly, HDP’s nonparametric nature and ability to handle an unknown number of mixture components make it ideal for online data clustering [74].

In our code, we implemented HDP for topic modeling by tokenizing documents and generating a dictionary from the tokenized words. A bag-of-words representation was created, and an HDP model was built using this representation. The number of topics and words per topic can be specified, in our case we again defined these values to 1 and 5.

3.2.5. Latent Semantic Analysis

Latent Semantic Analysis (LSA), introduced by Deerwester et al. [75], is a technique used in natural language processing and information retrieval to uncover the underlying structure and meaning in large text corpora. LSA represents words and documents in a high-dimensional space that captures latent semantic relationships. The key method used in LSA is Singular Value Decomposition (SVD), which is applied to a term-document matrix A, where a_ij represents the frequency of term i in document j.

SVD decomposes matrix A into three matrices U, Σ and V^T where:

• U containing the left singular vectors,
• Σ a diagonal matrix with singular values, and
• V^T the transpose of the matrix with the right singular vectors.

Next, dimensionality reduction is performed by retaining only the top k singular values and their corresponding vectors to obtain reduced matrices U_k, Σ_k, and V_k^T. The reduced matrices represent terms and documents in a lower-dimensional semantic space where each document is represented as a vector in this semantic space (V_k^T), capturing the latent topics.

Dimensionality reduction is then performed by retaining only the top k singular values and their corresponding vectors, producing reduced matrices U_k, Σ_k, and V_k^T. This process results in a lower-dimensional representation of terms and documents, where documents are expressed as vectors in this semantic space, capturing the latent topics:

$$A_k~U_k{\Sigma }_k{V}_k^T,$$

(7)

LSA has been widely used for topic modeling in various studies. Valdez et al. [76] demonstrated its effectiveness in identifying thematic patterns in large text collections. Gupta et al. [77] emphasized its role in revealing hidden patterns in big datasets, while Alghamdi & Alfalqi [78] provided a comprehensive review of different topic modeling methods, including LSA.

In our code we implemented LSA on a collection of documents by first creating a Term Frequency-Inverse Document Frequency (TF-IDF) matrix. We tokenized each document, extracted article titles, and applied Truncated Singular Value Decomposition (SVD) to reduce the dimensionality of the TF-IDF matrix. Top 5 terms for each topic were extracted once more.

3.3. Taxonomy Development

Taxonomies provide a systematic way to classify and organize complex data into meaningful categories, enabling stakeholders to identify key areas of focus and assess the effectiveness of observed topic [79,80]. This is particularly critical for government and European Commission officials, who could benefit from efficient resource allocation and prioritization of impact areas with the purpose of maximizing the societal and economic benefits of open data. As per Nickerson et al. [52], taxonomy in mathematical representation consists of n dimensions D_i (where i = 1, …, n), each comprising k_i (with k_i >= 2) mutually exclusive and collectively exhaustive characteristics C_ij (where j = 1, …, k_i). This structure ensures that each object being analyzed is associated with only one characteristic C_ij for each dimension D_i. In other words, we can express the taxonomy as:

$$\mathrm{T}={\{D}_i,i=1,\dots ,n|D_i=\{C_{ij},j=1,\dots ,k_i;k_1\ge 2\left\}\right\}$$

(8)

In this context, the taxonomy development process proposed by Nickerson et al. [52] offers a rigorous methodological approach to constructing a well-defined classification system. Nickerson et al. [52] method combines both empirical and conceptual approaches, ensuring that the taxonomy is grounded in real-world data while also aligning with theoretical frameworks. This iterative process allows for continuous refinement of categories based on empirical evidence and expert input, making it particularly suitable for the dynamic and evolving nature of OGD impact areas.

In the following sections, we will outline the steps taken in the taxonomy development process, including the identification of meta-characteristics, ending conditions and the iterative classification of impact areas. This structured approach ensures that the taxonomy is not only comprehensive but also adaptable to future developments in the field of open government data.

3.3.1. Meta-Characteristic

Following Nickerson et al. [52] approach we first had to define a meta-characteristic, which is a foundational concept in taxonomy development and serves as the primary guiding principle. It defines the scope and focus of the taxonomy, ensuring that the dimensions and characteristics are consistent and aligned with the overall purpose of the classification system. It is essentially the key attribute or perspective that all other dimensions and characteristics of the taxonomy will be based on.

Selecting an appropriate meta-characteristic is one of the most important steps in taxonomy development, as it influences the entire structure. According to Nickerson et al. [52], there are a few key considerations to consider when choosing the meta-characteristic:

• Purpose of the taxonomy: The meta-characteristic should align with the objective of the taxonomy and must reflect the core purpose or objective of the taxonomy.
• Audience and use cases: The meta-characteristic should be relevant to the stakeholders and use cases, guiding the classification in a way that meets their needs.
• Scope of the domain: The meta-characteristic should capture the essential scope of the domain being studied. It should be broad enough to encompass all relevant dimensions but specific enough to ensure focus.
• Mutually exclusive and collectively exhaustive: The meta-characteristic should help guide the development of dimensions and characteristics that are mutually exclusive (no overlap between categories) and collectively exhaustive (covering all relevant aspects)

Given that our goal is to classify the impact areas of OGD, the meta-characteristic defined for our research is: "The potential impact areas influenced by Open Government Data initiatives and their use cases."

3.3.2. Ending Conditions

Next, conditions that end the process must be determined. So called ending conditions are criteria that define when the iterative process in development of a taxonomy should be stopped. According to Nickerson et al. [52], taxonomy development in general is an iterative process involving empirical and/or conceptual steps, and it is essential to have clear criteria for when to stop iterating. Nickerson et al. [52] suggest that both objective and subjective ending conditions must be met before concluding the taxonomy development process. This ensures that the taxonomy is not only structurally complete (objective) but also practical, usable, and aligned with its intended purpose. Ending conditions that were proposed by Nickerson et al. [52] and will be used in this research are presented in Table 1.

3.3.3. Approach Selection

As first defined by Bailey [81] and later expanded and proposed in their model by Nickerson et al. [52], two different approaches can be utilized for taxonomy development: empirical-to-conceptual and conceptual-to-empirical. In the empirical-to-conceptual approach, the researcher selects a subset of objects to classify, typically based on familiarity or accessibility, potentially through literature reviews. This subset may be sampled randomly, systematically, or conveniently. The researcher then identifies common characteristics of these objects, which must logically derive from the meta-characteristic while also effectively discriminating among the objects. The researcher may rely on their knowledge or consult with experts. In the conceptual-to-empirical approach, the researcher starts by conceptualizing the dimensions without examining actual objects. This deductive process relies on the researcher’s understanding of similarities and differences among objects, drawing on existing knowledge and judgment to determine relevant dimensions. Each dimension must have characteristics that logically stem from the meta-characteristic. The appropriateness of these dimensions is assessed by checking if they encompass objects with the proposed characteristics. As in the empirical approach, each dimension must also be mutually exclusive and collectively exhaustive.

As stated by Nickerson et al. [52] one can utilize both approaches while creating a taxonomy, depending on the specific goals of the research and the nature of the subject matter being studied. In developing our taxonomy, we have chosen to utilize both the empirical-to-conceptual and conceptual-to-empirical approaches. This dual methodology allows us to leverage the strengths of each approach to create a comprehensive and robust classification system.

3.3.4. Validation

Besides the validation by using ending conditions Nickerson et al. [52] explains that currently there is no method to provide clear sufficient conditions for determining a taxonomy’s usefulness, other than stating that it is useful if it is actually used by others - a tautological condition. This aligns with design science research, which prioritizes utility over truth [50] and if this is the only sufficient condition, then the only way to assess a taxonomy’s usefulness is by observing its use over time.

Ideally, we would prefer sufficient conditions that are easier to apply and could be evaluated before the taxonomy is put into use. However, such conditions are likely to vary depending on the intended purpose of the taxonomy.

3.3.5. Process Schema

Drawing from the methodology outlined by Nickerson et al. [52], we introduce a high-level process schema (Figure 1) that details the essential steps involved in developing a taxonomy. Each step in this schema will also be referenced in every iteration of the taxonomy development in the Results section to simplify and improve reader's understanding of the process.

4. Results

In Results section, we present each iteration of the taxonomy development process in own section in a way that each section corresponds to a specific iteration and outlines the steps defined in the methodology section, providing a structured overview of the iterative process.

We initially performed a hierarchical clustering on the preprocessed use cases to identify potential high-level impact areas and possibly align them with the ones proposed by the . We first generated document embeddings using the Multilingual BERT model [82] and employed cosine similarity to calculate distances between rows in the dataset, following recommendations from previous studies addressing text mining challenges such as text classification, summarization, and information retrieval [83,84,85]. The results of the hierarchical clustering revealed four major groups in the data. Upon manual inspection of each cluster, we confirmed that most use cases were multi-domain, with three distinct clusters focused primarily on the environment, transport, and education, while the fourth and largest cluster encompassed all other domains. Defining this fourth area as either society or economy would lead to a disproportionate taxonomy, with the majority of taxons falling into this category. Therefore, while the clustering analysis did not show a direct correlation with the impact areas outlined by the Publications Office of the European Union, a manual review of the clusters better aligns with the initial four impact areas defined in the Open Data Maturity Report [19].

4.1. Iteration 1 - Initial Taxonomy Draft Based on Predefined Domains

The first iteration aimed to establish a preliminary taxonomy using predefined categories from the Open Data Maturity Report [19] and Data Europa use case sectors [22] before introducing machine learning techniques. Actions taken in this iteration, that correspond to the Nickerson et al. [52] methodology, are outlined in Table 2.

This iteration resulted in Table 4, which presents the initial taxonomy structure, mapping OGD use approaches, impact dimensions and corresponding use case sectors. However, it became evident that relying solely on predefined classifications limited the granularity and flexibility of the taxonomy. To address these shortcomings, the next iteration applied machine learning techniques to derive impact areas directly from OGD use cases.

4.2. Iteration 2 - Refinement Using GPT-Based Topic Modeling

To refine the taxonomy, GPT-based topic modeling was applied to extract emerging themes and impact areas from the dataset. Actions conducted in this iteration are outlined in Table 5 and refined dimensions with subcategories are presented in Table 6.

This iteration resulted in the updated taxonomy with seven impact dimensions and two additional use approaches. While this structure significantly improved the taxonomy, some extracted terms were too abstract or lacked practical relevance, necessitating further refinement through additional machine learning techniques.

4.3. Iteration 3 - Refinement Through Additional Topic Modeling Techniques

In Iteration 3, multiple topic modeling techniques were applied to validate the findings of GPT-based topic modeling from Iteration 2. The goal was to ensure that the identified impact areas were consistently represented across different machine learning approaches and to identify any additional emerging topics that may have been overlooked in the previous iteration. Actions and outcomes of the Iteration 3 are repesented in Table 7.

Each technique used in this iteration offers unique advantages and limitations, influencing the quality of extracted topics and their relevance to OGD impact classification. The comparison of these techniques is summarized in Table 8, highlighting their strengths and weaknesses.

Given these findings, the taxonomy structure was retained, but terms were manually refined to remove redundancies and clarify definitions before finalizing the taxonomy in Iteration 4. While additional machine learning techniques contributed to validating existing categories, they did not introduce significantly new impact dimensions, reinforcing the structure established in the previous iteration. Instead, few additional characteristics were identified, thus making taxonomy more complete.

This iteration confirmed that the refined taxonomy was robust and comprehensive, requiring only minor refinements in wording and classification logic before being finalized.

4.4. Iteration 4 - Final Refinements and Conceptual Validation

The fourth and final iteration focused on structural validation using Nickerson et al.'s [52] ending conditions.

Table 9. Key actions and outcomes of the fourth and final iteration of taxonomy development.

Step	Action	Outcome
3ce	Approach: conceptual-to-empirical	Extend taxonomy based on findings derived from other topic modelling techniques
4ce	Conceptual validation	Evaluated against taxonomy development criteriaNo additional approaches for using OGD were identified
5ce, 6ce	Conceptual validation	No additional characteristics or dimensions were identified
7	Assessed taxonomy against ending conditions	Verified coverage, exclusivity, simplicity, practicality, and extendibility

Table 9. Key actions and outcomes of the fourth and final iteration of taxonomy development.

Step	Action	Outcome
3ce	Approach: conceptual-to-empirical	Extend taxonomy based on findings derived from other topic modelling techniques
4ce	Conceptual validation	Evaluated against taxonomy development criteriaNo additional approaches for using OGD were identified
5ce, 6ce	Conceptual validation	No additional characteristics or dimensions were identified
7	Assessed taxonomy against ending conditions	Verified coverage, exclusivity, simplicity, practicality, and extendibility

The final taxonomy structure is detailed in Table 10, which maps the application of identified OGD use areas, and Table 11, which provides comprehensive descriptions of each dimension along with their corresponding subcategories and characteristics. At this stage, the taxonomy was considered structurally complete, fulfilling both objective and subjective ending conditions.

5. Discussion

The results of this research highlight the significant role of taxonomy development in categorizing OGD impact areas. By employing a combination of empirical-to-conceptual and conceptual-to-empirical methodologies, along with machine learning-driven topic modeling techniques, we successfully created a structured taxonomy that classifies OGD impact into various domains.

One of the key findings is that OGD impact areas extend beyond traditional domains of governance and transparency, affecting sectors such as education, infrastructure, and health. Our taxonomy reveals that OGD initiatives influence societal dynamics, economic conditions, and environmental sustainability. These findings align with prior research on OGD impact, which underscores the need for structured classification systems to measure and evaluate open data initiatives effectively [13,14]. Compared to previous studies, which often focused on single-use cases or specific geographies, our approach applies a scalable methodology by leveraging machine learning for topic identification. This contrasts with previous research that either relied on predefined classifications [37] or manually developed taxonomies [38].

While our taxonomy provides a strong foundation for assessing OGD's impact, it remains subjective and will require validation through domain expert reviews or surveys. Additional limitations apply for our dataset as well. Specifically, the limited size of our dataset constrained the scope of our conclusions and the use cases analyzed were relatively brief, which limited the depth of extracted insights to some extent. Another notable limitation is the subjectivity involved in defining taxonomic categories. While our methodology attempted to minimize bias by incorporating multiple machine learning techniques, human intervention was necessary to refine and interpret the results.

6. Conclusions

This research successfully developed a structured taxonomy for classifying the impact areas of open government data. Our approach integrated empirical data-driven methodologies with conceptual reasoning to create a robust and scalable classification system. The initial research question guiding this study was: How can we systematically classify and assess the impact areas of OGD initiatives? In response, we utilized machine learning techniques such as Latent Dirichlet Allocation (LDA), Non-negative Matrix Factorization (NMF), and Hierarchical Dirichlet Process (HDP) to extract meaningful topics from OGD use cases. Through iterative refinements, we structured OGD impact areas into key dimensions, including Society, Health, Infrastructure, Governance, and Environment.

This taxonomy provides a comprehensive framework for understanding how OGD contributes to various sectors and facilitates more effective policymaking and data utilization. Ultimately, this research contributes to the broader effort of systematically classifying and evaluating OGD’s impact, offering a structured methodology that can be adapted and refined in future studies.

Future work should focus on developing a classification model that can process incoming documents and probabilistically classify them into the relevant taxons, thus enhancing the taxonomy’s applicability and impact assessment capabilities.