Making CTI Sharing Trustworthy: BlockCTI's Implementation and Performance Evaluation

1. Introduction

Sharing Cyber Threat Intelligence (CTI) has become an essential practice for strengthening the ability to detect, prevent, and respond to cybersecurity incidents [1]. Public and private organizations have begun exchanging Indicators of Compromise (IoCs), malware signatures, Tactics, Techniques, and Procedures (TTPs), and other intelligence artifacts via collaborative platforms, enabling threats identified by one institution to be rapidly recognized by others [3]. Standards such as STIX (Structured Threat Information eXpression) [4] and protocols such as TAXII (Trusted Automated eXchange of Intelligence Information) [5] have established themselves as widely used mechanisms for fostering interoperability among tools and organizations. However, as the volume of shared information grows, so does the need to ensure that such information is reliable, maintains its integrity, and originates from sufficiently high-quality sources to inform security decisions.

Although current platforms facilitate information exchange, several challenges remain. Key among these are the difficulty of verifying the provenance of shared evidence, the potential for record manipulation or tampering over time, the lack of standardized mechanisms to assess the quality of received information, and the limited traceability of the validation process [6]. In collaborative environments comprising multiple independent organizations, incorrect, outdated, or deliberately false information can compromise automated defense processes, increase the incidence of false positives, and undermine trust among network participants [7]. Consequently, there is a need to develop mechanisms capable of recording, validating, and sharing cyber intelligence in a manner that is transparent, auditable, and resistant to unauthorized modification.

In recent work, BlockCTI was proposed as a conceptual architecture to enhance the trustworthiness of Cyber Threat Intelligence (CTI) sharing platforms in governmental environments [2]. That work established the architectural design and principles underlying an integrated approach combining three key components: a permissioned blockchain for data immutability and traceability, a Proof of Quality (PoQ) consensus mechanism to replace computational validation with expert-driven evaluation [8], and the NATO Admiralty Scale [10] to standardize the assessment of source reliability and information credibility. This combination enables security incidents to be evaluated before being recorded on the blockchain, considering structural data aspects alongside attributes such as completeness, verifiability, interoperability, originality, and timeliness of shared evidence.

BlockCTI [2] integrates specialized components to generate, validate, coordinate, store, and disseminate security incidents within a decentralized environment. Unlike approaches that employ blockchain solely as an immutable storage mechanism, the proposed architecture incorporates a structured quality assessment process for threat intelligence before the network accepts it. This ensures that only information deemed trustworthy by qualified security experts is recorded and shared across participating organizations.

To validate the proposed system, the present paper develops a functional prototype using independent services deployed in Docker containers. These services are responsible for incident validation, decision coordination, blockchain persistence, and integration with intelligence-sharing platforms. The experimental evaluation comprises five complementary analyses: blockchain scalability, sensitivity to the number of validators, coordination resilience, behavior under varying workloads, and robustness when processing incidents of different quality levels.

The results demonstrate that the proposed architecture maintains operational performance across various infrastructure configurations, exhibiting consistent behavior as the blockchain scales, the number of validators increases, and the workload increases. Furthermore, the experiments show that the Proof of Quality mechanism effectively distinguishes high-quality incidents from those containing degraded or adversarial information, ensuring that only records deemed sufficiently reliable are persisted and disseminated. These results indicate that combining a permissioned blockchain, quality assessment, and criteria inspired by the NATO Admiralty Scale offers a viable approach to enhancing the reliability of collaborative Cyber Threat Intelligence sharing.

The remainder of this article is organized as follows. Section 2 introduces fundamental concepts related to Cyber Threat Intelligence sharing, blockchain, Proof of Quality, and the NATO Admiralty Scale. Section 3 discusses key related works found in the literature. Section 4 describes the BlockCTI architecture and its validation mechanism in detail. Section 5 presents the experimental methodology and discusses the results obtained across different evaluation scenarios. Finally, Section 6 presents the study’s conclusions, limitations, and prospects for future research.

2. Background

This section introduces the key concepts necessary to understand this paper.

2.1. Cyber Threat Intelligence

Effective collaborative cybersecurity in government networks requires moving beyond simply sharing raw data to sharing structured Cyber Threat Intelligence (CTI). CTI is the process of collecting and analyzing information to predict and respond to threats [3]. In environments where multiple organizations work together, CTI enables teams to make informed decisions about threats. However, this only works if the shared intelligence is reliable and usable across different systems and organizations.

2.2. STIX, TAXII, and MISP

To enable CTI sharing across systems, organizations use the Structured Threat Information eXpression (STIX) standard, which offers a machine-readable format for security data. STIX helps security teams consistently structure and communicate cyber threat information—including indicators of compromise, malware details, attack patterns, vulnerabilities, and threat actor data. This standardization allows different tools and organizations to process threat data automatically, without manual interpretation.

Intelligence formatted in STIX is distributed through the Trusted Automated eXchange of Intelligence Information (TAXII) [5], which securely shares Cyber Threat Intelligence (CTI) between organizations. TAXII facilitates both real-time and batch sharing of standardized threat data. This information is integrated with the Malware Information Sharing Platform (MISP), an open-source tool for collecting and collaborating on threat data [11]. Together, STIX, TAXII, and MISP create an ecosystem that enhances the way security teams exchange threat information, improving collective defense.

However, while these standards and platforms work well for communication between different systems, they do not provide a way to verify where information came from or to create a complete, auditable record of what happened to the data as it moves between organizations.

2.3. Blockchain

To provide a more secure infrastructure for these exchanges, organizations can use private blockchains—distributed databases that maintain an immutable record of all transactions. [12]. Unlike public blockchains like Bitcoin that anyone can join [13], private blockchains restrict access to authorized participants only. This controlled environment helps protect sensitive data and ensures data integrity, meaning that records cannot be changed or deleted once added. Private blockchains also provide non-repudiation, meaning participants cannot deny having sent or received information, thereby creating accountability throughout the sharing process. These properties are essential for sensitive governmental operations where data security and record-keeping are critical [1].

2.4. Proof of Quality (PoQ)

Traditional blockchains rely on consensus mechanisms that validate computational work or require voting among participants. However, these methods are inadequate for intelligence sharing, which prioritizes the quality and reliability of threat information over computational validation. Standard consensus can’t assess the accuracy or trustworthiness of shared intelligence; it only indicates agreement among participants or task completion.

The Proof of Quality (PoQ) model is an alternative consensus mechanism designed for threat intelligence sharing [8], focusing on the quality and merit of information rather than computational power or voting [12]. PoQ uses expert judgment to assess the reliability and accuracy of threat intelligence, potentially using frameworks such as the Admiralty Scale (NATO system) [10] to evaluate credibility.

2.5. NATO Admiralty Scale for intelligence assessment

The Admiralty Scale separates source reliability from information credibility, enabling analysts to evaluate each independently. This distinction is important as a reliable source might present unverified information, while an unreliable source could share accurate data. By allowing human security experts to assess the quality of threat intelligence using professional judgment, the Admiralty Scale offers a structured method for evaluating trust.

3. Related work

Recent scholarship has explored the integration of blockchain technology to address challenges in secure, decentralized Cyber Threat Intelligence (CTI) sharing. Cha et al. [14] proposed a blockchain-based CTI system architecture tailored to sustainable computing environments, demonstrating the viability of distributed ledgers for efficient, resource-conscious threat data exchange. This foundational work established blockchain’s potential for managing large-scale CTI operations while maintaining computational efficiency.

Security concerns in decentralized sharing environments have motivated additional research. Gong and Lee [15] introduced BLOCIS, a blockchain-based sharing framework designed to resist Sybil attacks, in which malicious actors create multiple forged identities to manipulate threat information pools. By addressing identity integrity and authentication challenges, their framework established critical security principles relevant to government intelligence-sharing networks.

The semantic interoperability of threat information across heterogeneous systems has also received attention. Homan et al. [6] proposed a network model that integrates the STIX standard for machine-readable exchange of incident data. By ensuring that threat information can be consistently understood across independent organizations, their work laid important groundwork for automated processing of shared intelligence.

Chatziamanetoglou et al. [8] developed a CTI sharing model incorporating a Proof of Quality (PoQ) consensus mechanism. Their contribution represents a significant conceptual shift away from computational validation toward qualitative assessment of intelligence merit. However, their work does not establish standardized methods for applying quality metrics consistently across expert domains and organizational contexts.

While existing research demonstrates that blockchain provides immutability and STIX enables interoperability, a critical gap remains: the lack of a standardized, expert-driven framework for trust evaluation that bridges these technical capabilities. Prior work formalized BlockCTI [2] as an architectural solution to this gap, integrating the NATO Admiralty Scale into the PoQ consensus mechanism. In that work, trustworthiness was defined as a composite property combining the technical integrity of blockchain infrastructure with the semantic credibility established through expert human validation. By formalizing the Admiralty Scale within the consensus process, it provides a rigorous evaluation framework that moves beyond generic peer review toward standardized reliability and credibility assessment, thereby enabling more informed, evidence-based decision-making in governmental cybersecurity operations.

4. BlockCTI: Architecture and Proof-of-Quality Validation Mechanism

This subsection provides an overview of the proposed solution.

4.1. Architecture Overview

BlockCTI was conceived as a distributed architecture to support the trustworthy sharing of Cyber Threat Intelligence within government environments, combining immutability, traceability, and qualitative validation of reported incidents [2].

The proposal was modeled on an institutional ecosystem in which various entities participate in the intelligence cycle, from the initial incident submission to the dissemination of validated information. In this design, the architecture integrates a permissioned blockchain with a Proof of Quality (PoQ) mechanism and evaluation criteria inspired by the NATO Admiralty Scale; consequently, the acceptance of an incident depends not only on syntactic compliance but also on the quality and credibility of the shared content.

Furthermore, the proposal maintains interoperability with established industry standards such as STIX, TAXII, and MISP, facilitating integration with ecosystems already utilized by cyber defense analysts and organizations.

Figure 1 presents an overview of the proposal. An incident is received, sanitized, and submitted for validation by multiple specialized nodes; the aggregated decision is then recorded on an immutable ledger and subsequently disseminated to authorized entities.

4.2. System Components

BlockCTI [2] consists of a set of functionally specialized components. The first is the Incident Generator, which represents the source of the incident report—either an external entity or a node within the collaborative network.

Next, the Validation Gateway serves as the architecture’s entry point, receiving the incident, sanitizing the data, and preparing the content for validation. This mediation role is crucial, as it reduces the risk of exposing sensitive information and ensures the content reaches the system in a format suitable for processing.

Validator Nodes form the core of the PoQ mechanism. Each validator examines the incident using a set of qualitative metrics and assigns a score reflecting their assessment of the report’s reliability.

The Coordinator Entity receives the individual evaluations, consolidates the votes, and applies the final decision rule based on the aggregated average and NATO scale thresholds.

The blockchain records the decision outcome immutably, preserving the history of the validated incident and enabling future audits. Finally, TAXII and MISP services disseminate the validated intelligence to authorized consumers, completing the collaborative sharing cycle.

4.3. Operational Workflow

The BlockCTI operational workflow comprises four phases: incident reporting, data validation, Proof-of-Quality consensus, and intelligence dissemination [2], as illustrated in Figure 2.

Initially, the reporting entity submits an incident describing the observed event in a structured format. This report is forwarded to the gateway, which performs validation and sanitization before passing it to the validation layer. In this phase, the system ensures the received data is suitable for processing while minimizing the risk of unauthorized disclosure of sensitive details and guaranteeing content standardization in STIX 2.1. [2]

Subsequently, the incident is submitted to validator nodes, which perform a qualitative assessment of the content and generate their respective scores. The coordinating entity aggregates these assessments, interprets the result using the NATO Admiralty Scale, and decides whether to accept or reject the incident. Upon approval, the result is committed to the private blockchain, where it remains as an immutable and auditable record. Finally, the validated intelligence is disseminated to authorized consumers via TAXII and MISP, enabling integration into operational monitoring and response workflows [2].

4.4. Proof-of-Quality Consensus

The consensus process in BlockCTI relies on a qualitative assessment of intelligence shared via PoQ [2]. To implement this process, the system employs seven complementary metrics (Table 1): breadth, verifiability, intelligence, interoperability, syntactic accuracy, originality, and timeliness. These metrics capture the structural and semantic dimensions of the incident, enabling evaluation of whether the report is complete, verifiable by independent sources, contextualized, compliant with the expected format, original, and timely.

Each validator assigns a discrete score between 0 and 5 based on these metrics, reflecting their interpretation of the incident’s reliability within the NATO Admiralty Scale. Rather than relying on a single judgment, BlockCTI aggregates individual assessments by calculating the arithmetic mean of the votes, yielding a consolidated score that represents the perceived quality of the incident [2]. The result is mapped to a NATO scale category: Confirmed, Probably True, Possibly True, Doubtful, Improbable, or Cannot Judge, as shown in Table 2.

This mechanism balances robustness and cost: more validators reduce bias and increase confidence, but they also raise coordination delays and processing costs, creating a trade-off between accuracy and speed.

4.5. Prototype Implementation

The proposal was implemented as a functional prototype composed of independent services, each implemented in Docker containers, enabling the system to run in a controlled environment and allowing for reproducible experimental evaluation. The implementation follows a modular design in which architectural roles are distributed among generation, validation, coordination, ledger, and dissemination services.

This modularity facilitates system maintenance and evolution while also enabling exploration of different architectural configurations without altering the solution’s core logic. Consequently, the prototype goes beyond a mere conceptual description of the architecture, serving instead as an operational implementation that supports controlled experiments across various scenarios.

5. Evaluations

This section presents the evaluations conducted on the proposed solution.

5.1. Experimental Methodology

The experimental evaluation was conducted in accordance with the methodology proposed by [16]. The objective was to assess BlockCTI’s behavior by modifying various architectural and operational properties to understand their impact on latency, throughput, approval rate, and quality score. The experiments were organized into five blocks, each focusing on blockchain scalability, validator sensitivity, coordination resilience, load variation, and incident quality.

Across all blocks, the observed variables included the numbers of blockchain nodes, validators, and coordinators, the load level, and the quality of reported incidents. In each scenario, measurements were repeated several times and aggregated using the mean, standard deviation, confidence interval, and distribution per replication; this approach minimized the impact of isolated noise and enhanced the analysis’s robustness.

The evaluation methodology was developed incrementally. First, the architecture’s functionality was validated; then, scenarios were adjusted to measure one factor at a time. Finally, results were compiled into comparable tables and graphs. This design avoids conflating the effects of scale, consensus, coordination, load, and quality, which helps ensure an objective interpretation of the results. The study aims to assess how the prototype responds to controlled changes in operating conditions.

5.2. Execution Environment and Configuration

BlockCTI was implemented in a fully containerized environment, which enabled service isolation, reproducibility, and greater control. The architecture used Docker containers managed by Docker Compose, featuring separate services for incident generation, gateway, validators, coordinators, ledger, TAXII, MISP, and observability. This modular setup enabled independent measurements and the emulation of various scenarios without altering the core logic.

From a technological standpoint, the implementation used Python for experimental logic, scenario generation, and graph generation. The environment incorporated Prometheus and Grafana for observability and STIX 2.1-based CTI representation, featuring conceptual integration with TAXII and MISP. The experiment was executed on a Linux Mint 22.3 host equipped with an Intel Core i7-11700K processor and 16 GB of RAM, maintaining an environment stable enough to support the required number of repetitions.

The experimental setup remained consistent across all blocks, except for parameters intentionally varied for each scenario. The study was conducted with 30 repetitions per scenario and 20 events per repetition, enabling calculation of averages, dispersion, and confidence intervals, with a sample size sufficient for reliable trend comparisons.

5.3. Results

5.3.1. Block A: Blockchain Scalability

The first block evaluated the effect of blockchain growth on system performance by varying the number of simulated logical nodes among 46, 92, 138, and 184. The objective was to determine how network expansion impacts coordination costs, ledger persistence, and system processing capacity, as shown in Figure 3 and Figure 4.

The results demonstrated the expected monotonic behavior: as the number of nodes increased, throughput decreased (Figure 3), and latencies for the coordination and ledger stages rose (Figure 4), while the average PoQ score remained stable. This pattern indicates that the system preserves its validation logic but incurs higher operational costs when sustaining a larger network.

5.3.2. Block B: Sensitivity to the Number of Validators

The second block investigated PoQ sensitivity to the number of validators, maintaining the blockchain with 184 nodes while varying the validator set size among 3, 5, 7, 9, and 11 nodes. This assessment was crucial for understanding the extent to which increased validation redundancy influences decision robustness and coordination costs, as illustrated in Figure 5 and Figure 6.

The experiments showed that the average PoQ remained relatively stable, with a slight downward trend as the number of validators increased. The results show that, as the number of validators increased, throughput decreased (Figure 5) and end-to-end latency increased more sharply (Figure 6). This demonstrates that adding validators increases the load on the decision-making process but does not drastically alter the quality class assigned to incidents.

5.3.3. Block C: Resilient Coordination

This set of experiments analyzed BlockCTI’s resilience with respect to the number of coordinators, comparing configurations with one and three coordinators while maintaining a blockchain with 184 nodes and a minimum set of three validators (Figure 7 and Figure 8).

Introducing multiple coordinators slightly raised coordination costs and latency but preserved the decision-making logic. However, it enhanced resilience, lowering the risk of depending on a single coordinator.

5.3.4. Block D: Load Variation

The fourth block examined load variation by simulating four demand profiles: low, moderate, high, and burst. The objective here was to observe how the system responds as the volume and intensity of requests increase over time; the results are presented in Figure 9 and Figure 10.

The results showed the behavior expected of a distributed system under load: end-to-end latency increased with load intensity (Figure 10), while throughput rose to the high-load scenario but dropped during the burst mode, a more aggressive and concentrated condition (Figure 9). However, the approval rate remained stable across all scenarios, indicating that the system’s decision-making logic holds as the load increases.

5.3.5. Block E: Robustness to Incident Quality

The fifth block evaluated the system’s robustness regarding incident quality by comparing four levels: high, medium, low, and adversarial. This was the most critical experiment for demonstrating BlockCTI’s unique value, as it directly tested whether the PoQ mechanism could distinguish reliable inputs from degraded or malicious reports (Figure 11 and Figure 12).

The results aligned with expectations: the high-, medium-, and low-quality scenarios were accepted, whereas the adversarial scenario was rejected, resulting in a significant drop in the PoQ score (Figure 12). Furthermore, a progressive decline in the score was observed as quality decreased (Figure 11), indicating that the system successfully captured the differences between trust profiles.

5.4. Discussion of Results

The results show that BlockCTI maintained its operational logic across varying conditions of scale, coordination, and load, while simultaneously introducing a decision-making mechanism sensitive to incident quality. Blocks A, B, C, and D demonstrate that the architecture responds predictably to structural environmental changes: increases in nodes, validators, coordinators, and load entail higher execution costs, particularly regarding latency and throughput, without disrupting the flow. This behavior aligns with expectations for a reliability-oriented distributed architecture, where gains in robustness and auditability come with certain operational costs.

However, the most significant result appears in Block E. In this scenario, the Proof of Quality mechanism demonstrated superior discriminatory capability, rejecting adversarial inputs while accepting incidents across varying levels of acceptable quality. This suggests that the architecture does more than simply store CTI immutably; it also implements a filtering layer that mitigates the impact of low-trust information within the collaborative ecosystem. In summary, the experiments indicate that BlockCTI achieves the desired balance between integrity, traceability, and qualitative assessment, making it a viable solution for contexts where the reliability of shared intelligence is as critical as its availability.

6. Conclusions and Future Work

This article implements BlockCTI, a distributed architecture for the trustworthy sharing of Cyber Threat Intelligence (CTI) in collaborative environments, supported by a permissioned blockchain, a Proof of Quality mechanism, and criteria inspired by the NATO Admiralty Scale. The work addressed the issue that CTI sharing, despite being widely adopted through standardized formats and protocols, still faces limitations in trust, traceability, and the ability to distinguish high-quality reports from incomplete, inconsistent, or potentially adversarial information. Consequently, the proposal sought not only to record information immutably but also to introduce a structured qualitative validation step before data is persisted on the blockchain.

6.1. Main Contributions

The main contribution of this paper is the functional prototype of BlockCTI, comprising specialized services for intelligence generation, validation, coordination, recording, and dissemination. The solution integrates the incident workflow with established industry standards—such as STIX 2.1, TAXII, and MISP—and organizes validation through multiple validators that assign scores based on seven quality metrics. Experiments demonstrated that the architecture responds predictably to variations in scale, consensus, coordination, load, and incident quality, highlighting its viability as a mechanism to support reliable CTI sharing.

6.2. Study Limitations

The study has limitations that must be considered when interpreting the findings. The main limitation is that the PoQ decision process relies on human analysts or validators, which can lead to operational costs and delays, especially during periods of high incident volume. While this design ensures quality and reliability, it makes the process less agile compared to fully automated methods.

Another limitation stems from the prototype’s experimental nature. Tests were conducted in a containerized environment using controlled scenarios, an approach suitable for comparative evaluation but not yet a substitute for large-scale deployment in a real-world production environment.

Furthermore, the incidents used in the experiments were modeled to represent various quality conditions, yet they remained within a finite and relatively controlled set of scenarios. Consequently, generalizing the results to more heterogeneous operational contexts requires caution.

Finally, the architecture was evaluated using a specific permissioned blockchain configuration and component integration, leaving open the question of how different infrastructure and consensus choices might affect the system’s ultimate behavior.

6.3. Future Work

For future work, one direction is to explore hybrid decision-making that combines human assessment with automated techniques for classifying incident quality. This could reduce reliance on validators, speed up decisions, and enhance scalability while maintaining qualitative criteria. Another possibility is to investigate adaptive threshold policies that adjust approval thresholds based on system load, validator history, or incident criticality.

Expanding the experimental evaluation with larger, more diverse incident datasets and testing BlockCTI in a distributed operational environment would better assess the architecture’s robustness under varied network conditions. Additionally, incorporating new CTI sources, reputation mechanisms, and threat prioritization would improve its effectiveness in collaborative cyber defense settings.