A Multi-Framework Validation Approach for Blockchain-Based Healthcare Devices: A Case Study

1. Introduction

The healthcare industry has long depended on centralized systems for patient data storage and management. Despite being practical for organizations, these solutions frequently lead to vulnerabilities in security and provide patients with limited access to their most private data [1].

Blockchain technology offers an alternative solution through its decentralized approach to data management. However, successfully integrating blockchain into practical healthcare devices isn't straightforward and comes with challenges that need careful consideration [2].

The recent work suggests that beyond technical considerations, we must carefully balance data privacy concerns, adoption barriers, and stakeholder and user needs throughout the development process [3,4,5,6,7,8]. Although significant research has explored blockchain's potential in healthcare [8,9,10,11], we have found that existing approaches often examine design, user acceptance, and technical implementation separately, potentially missing valuable insights that could emerge from their synthesis. By bringing these perspectives together, we can better understand how to create blockchain-enabled healthcare devices that are technically sound, truly serve user needs, and promote adoption.

Building upon our previous research on blockchain-based health wearables [12], this study presents a comprehensive validation approach examining these challenges through the continued development and evaluation of a Smart Fidget Toy prototype.

Our previous work established preliminary design guidelines for designing blockchain-powered devices and demonstrated the initial feasibility of integrating blockchain technology into a health monitoring device. However, questions remain about how different evaluation frameworks can be combined to better understand and improve user acceptance and blockchain integration in healthcare devices.

The Smart Fidget Toy represents a new approach to health data collection, combining traditional tactile interaction with blockchain-based secure data management. While our initial prototype validated basic technical feasibility, this study extends that work by synthesizing insights from three complementary approaches: prototype evaluation by experts using Web3 Design Guidelines [13] and User Acceptance Toolkit [14], and an extended user testing study.

By bringing these perspectives together, we aim to develop more effective solutions that address technical requirements and user needs in blockchain-enabled healthcare devices. This synthesis allows us to identify where different approaches reinforce each other and where they suggest novel solutions that might not be apparent from any single perspective.

This multi-framework validation approach addresses several key research questions:

• How do insights from different evaluation frameworks complement or contradict each other?
• How do users' trust and engagement with blockchain features evolve over device usage?
• How can multiple evaluation perspectives be effectively combined to improve blockchain integration in healthcare devices?

While prior studies typically evaluate blockchain applications from isolated perspectives, this paper introduces and demonstrates a novel multi-framework validation approach. This is the systematic synthesis of findings from distinct but complementary evaluation methods. In this case, technical guideline compliance, user acceptance modeling, and real-world user testing. The novelty of this research lies not in a technological breakthrough, but in the application and validation of this combined method. By bringing these viewpoints together, our approach provides a holistic understanding of the inherent tensions between technical feasibility and practical usability, uncovering critical insights that a single, siloed analysis would miss.

This paper's remaining sections are arranged as follows: Section 2 examines relevant research from design frameworks, user acceptance studies, and blockchain healthcare applications. Our methodology, which includes the improved Smart Fidget Toy prototype and our synthesis of three complementary approaches, is presented in Section 3. The findings are analyzed in Section 4, which looks at how knowledge from various methodologies comes together to offer better solutions for blockchain-based medical equipment. While Section 6 wraps up the paper with essential lessons for creating successful blockchain-based health monitoring solutions, Section 5 addresses the implications of our findings and makes recommendations for future research directions.

2. Related Work

2.1. Blockchain in Healthcare

The decentralized and immutable nature of blockchain technology makes it an ideal solution for improving the security and privacy of healthcare data [8]. Using cryptographic techniques, blockchain ensures that patient data is securely stored, reducing the risk of unauthorized access and data breaches. The technology also supports the secure sharing of electronic health records (EHRs) across healthcare providers [15].

Patients can grant and revoke access to their data using blockchain-powered apps, ensuring that they remain at the center of their healthcare journey [16]. This patient-centric approach enhances patient engagement and improves the accuracy and reliability of health records by allowing patients to contribute to and own their data [17,18]. This patient-centric control naturally extends to the management of electronic health records.

Blockchain technology is being increasingly applied to improve the management of EHRs. Traditional EHR systems often face centralization, security, and privacy issues. Blockchain offers a decentralized approach, enhancing the trustless and secure exchange of health information among stakeholders [10,11].

Several blockchain platforms and frameworks have been proposed and implemented in healthcare. Hyperledger Fabric and Ethereum are the most commonly used platforms, providing the necessary infrastructure for secure and efficient data transactions [19,20]. These platforms support the development of smart contracts, which automate processes such as insurance billing and prescription management [11,21].

2.2. User-Centered Design of Blockchain Applications

Compared to conventional digital products, designing for Web3 presents new concepts. Our previous work [13] presents a set of foundational Web3 design guidelines emphasizing transparency, user autonomy, and contextual onboarding, aiming to reduce cognitive load while navigating decentralized interfaces. These principles align with broader frameworks like the Ethereum Foundation's "7 Heuristics for Web3 Interface Design" [22].

Unlike Web2, blockchain-based applications challenge conventional assumptions around user trust models, data flow, and interface expectations. For example, concepts like immutability, gas fees, and self-custody require novel design approaches to educate and empower users without overwhelming them [23,24]. The lack of centralized authority also necessitates rethinking fallback mechanisms, error resolution, and support systems [25].

Researchers have observed that while Web3 offers prospects for data ownership and user empowerment, it also presents significant adoption challenges [26]. These barriers include complicated technical and financial concepts that must be communicated effectively through the interface [23]. According to studies, users commonly misinterpret important blockchain concepts, such as transaction fees, security measures, and the permanence of on-chain operations [27].

Blockchain's transparency poses a tension with users' expectations for privacy. Designers must balance openness with privacy by offering selective disclosure, pseudonymity, and off-chain data handling options when necessary [28]. User control is central to Web3 philosophy, making data sovereignty and consent management critical components of user-centered design. Interfaces should empower users to control how and when their data is shared and with whom [29,30].

2.3. User Acceptance of Blockchain Technology

User acceptance of new technologies has been extensively studied through various theoretical frameworks, with several models offering particular relevance to blockchain adoption in healthcare contexts. The Technology Acceptance Model (TAM) remains one of the most influential frameworks for understanding technology adoption, focusing on two primary factors: perceived usefulness and perceived ease of use [31]. These factors influence users' attitudes toward technology, affecting their behavioral intentions to use it [31].

TAM's straightforward structure has made it a foundation for technology acceptance research, though its simplicity sometimes fails to capture the complexity of emerging technologies like blockchain. Several researchers have extended TAM to address blockchain-specific adoption factors better. Albayati et al. [32] incorporated external variables into TAM, demonstrating that regulatory support and user experience significantly improve trust and acceptance of blockchain applications. Their work highlights how institutional factors interact with individual perceptions to shape blockchain adoption decisions.

Similarly, Chen et al. [33] expanded TAM by incorporating strategic management and social influence at the firm level, as well as individual innovation and self-efficacy at the personal level, revealing how multi-level factors significantly impact blockchain adoption intentions. The Unified Theory of Acceptance and Use of Technology (UTAUT) represents a more comprehensive approach, integrating elements from eight different models, including TAM [34]. UTAUT identifies four key determinants of user acceptance, while UTAUT2 extends this model by adding hedonic motivation, price value, and habit, making it particularly relevant for consumer contexts [35]. This extended framework provides valuable insights for understanding blockchain adoption in healthcare. Recent research has demonstrated the value of combining theoretical models to capture the complexities of blockchain adoption better. For example, Alazab et al. [36] integrated UTAUT with Task-Technology Fit and Information System Success models to analyze blockchain adoption in supply chains. Their integrated approach highlighted how trust and inter-organizational relationships critically influence the adoption process, particularly relevant to healthcare contexts where multiple stakeholders must coordinate around sensitive patient data.

2.4. Summary and Research Positioning

The literature reveals that blockchain offers promising solutions for healthcare data management through improved security and patient control. Its decentralized structure provides opportunities for improved data sovereignty and transparency while maintaining privacy and security. However, these benefits often face significant usability challenges that hinder adoption [25].

Designing for blockchain requires fundamentally different approaches than traditional systems, as concepts like immutability and self-custody require novel design patterns to empower users without overwhelming them [13,23]. A review of current research exposes a notable disconnect between technical implementations, design guidelines, and user acceptance studies, with these aspects typically examined in isolation rather than as interconnected elements. This siloed approach is problematic because it fails to capture the complex interplay where technical feasibility constrains design choices, which dictate user acceptance. Many studies remain focused on conceptual models or laboratory evaluations, with limited real-world testing of blockchain healthcare applications, leaving a gap in understanding the complexities of actual usage contexts.

Our research addresses this gap by proposing and applying a multi-framework validation approach. The novelty of our work lies not in assessing one of these facets alone, but in the synthesis and triangulation of three distinct perspectives: an expert-based technical evaluation using Web3 Design Guidelines [13], a predictive assessment with a User Acceptance Toolkit [14], and an experiential study through extended user testing.

This integrated methodology provides complementary perspectives that yield a more comprehensive understanding, which is crucial as technological innovations often contain dimensions that cannot be adequately captured by any single evaluation method [37].

By synthesizing findings, we can identify areas where the frameworks reinforce each other, thus strengthening validity through triangulation [38], while also uncovering tensions and contradictions that reveal deeper insights into the challenges of blockchain integration. This approach allows us to build a more nuanced understanding of how to design blockchain-enabled healthcare devices that are technically sound and truly meet user needs and promote adoption, directly responding to calls in the literature for more integrated, user-centered solutions.

3. Methods and Materials

This study employs a convergent mixed-methods design [38] to evaluate the usability and user acceptance of blockchain integration in healthcare devices. This design facilitates the parallel collection of quantitative and qualitative data, which is then systematically merged and compared to provide a comprehensive understanding. We utilize a multi-framework validation approach, examining an improved version of a blockchain-enabled Smart Fidget Toy, now named CipherPal, as a case study.

This approach combines three complementary frameworks to achieve a holistic evaluation. First, the Web3 Design Guidelines [13] were used for an expert-driven assessment focused on adherence to established blockchain-specific UX/UI principles. Second, the User Acceptance Toolkit [14], grounded in the UTAUT2 model, was applied to identify potential adoption barriers and socio-technical factors proactively. Finally, an Extended User Testing study was conducted to gather real-world data on practical usability and the experience of interacting with the device over time, building on our previous prototype work.

3.1. Study Design

The convergent design facilitates parallel quantitative and qualitative data collection through three complementary evaluation frameworks. Each framework provides distinct insights while contributing to a comprehensive understanding of blockchain integration in healthcare devices. The Web3 Design Guidelines evaluation offers an expert assessment of blockchain-specific interface requirements and interaction patterns. The User Acceptance Toolkit provides a structured analysis of adoption factors and behavioral influences. The extended user testing study enables longitudinal observation of real-world usage patterns and evolution of user engagement with blockchain features (Figure 1).

These frameworks operate concurrently, with findings from each stream informing the others throughout the study period. Quantitative data includes usage metrics, implementation scoring, and feature utilization patterns, while qualitative data encompasses expert evaluations, user interviews, and observational insights. This multi-dimensional approach thoroughly examines both technical implementation and experiential aspects of blockchain integration.

Data from each framework will be analyzed independently initially. Subsequently, the findings from all three strands will be systematically merged, compared, and synthesized during the interpretation phase. This triangulation allows for a richer understanding than any single method could provide alone. It enables us to identify areas where insights from the different frameworks complement or contradict each other, and to explore how user trust, engagement with blockchain features, and data-sharing behaviours evolve through sustained device interaction.

By integrating these diverse data sources, this study design directly addresses the core research questions regarding the intersection of technical feasibility, design effectiveness, and user acceptance in the specific context of blockchain-enabled health technologies like CipherPal.

To aid reproducibility, the complete instruments used for data collection, including the Web3 Design Guidelines evaluation form, the onboarding survey, and the daily reflection survey, are provided in Appendix A, Appendix B, and Appendix C. Furthermore, the User Acceptance Toolkit template and the completed canvases from the group evaluation sessions are publicly available in a FigJam file, which can be accessed at [39].

3.2. Smart Fidget Toy Implementation

This section details the implementation of the enhanced Smart Fidget Toy prototype used as the case study in this research. The prototype represents an evolution from the mid-fidelity version described in previous work [12], incorporating technical and physical refinements based on user feedback and design iterations.

3.2.1. Physical Design

The physical device is constructed from three main structural components: a base module housing a rotary mechanism, a custom double-sided Printed Circuit Board (PCB) which contains the core electronics, and an upper interaction surface featuring tactile elements (Figure 2).

Several key hardware features are integrated into this structure. A microcontroller serves as the main processing unit, supported by onboard flash memory. To monitor heart rate, photoplethysmography (PPG) sensors are integrated at key interaction points, such as under the main button and the rotating knob surface. A rotary encoder captures rotational interactions, and a tactile button is used for clicking interactions.

For user feedback, an LED provides visual status indications (e.g., yellow for waiting, blue for data collection, green for success, red for errors) based on the firmware logic. Furthermore, a standard data and charging port facilitates connectivity for charging the internal battery and enables serial communication with the companion web application for data transfer and initial device configuration.

The device's housing was fabricated using 3D printing. Rigid structural parts were printed using a clear photopolymer, while flexible, transparent elements such as those covering sensors or providing tactile feedback were printed using a mix of rubber-like and rigid photopolymers.

3.2.2. System Architecture and Data Flow

The prototype connects hardware sensors to a blockchain-based backend via a companion web application designed for secure and user-controlled data management. The complete system architecture, detailing this data flow from the physical device to the blockchain, is illustrated in Figure 3.

The system hosts a custom smart contract using the Polygon zkEVM Cardona testnet [40] to host a custom smart contract. This platform was selected for this proof-of-concept study because it is a zero-knowledge network, offering potential future scalability and privacy benefits while maintaining compatibility with standard Ethereum development tools. The smart contract is primarily responsible for managing references to user data. Users, identified by their wallet address, can authorize the storage of records on the blockchain. These records contain pointers to the actual session data, which is stored off-chain for efficiency, along with information necessary for authorized retrieval.

Users interact with the system through a web application. This application provides the interface for connecting the device, synchronizing and visualizing collected data, and managing blockchain interactions. From a user perspective, authentication and any necessary transaction confirmations for blockchain operations are handled using standard browser-based cryptocurrency wallets.

For security, the device firmware encrypts all collected session data on the device itself before it is transmitted. An encryption key is established during an initial pairing process between the device and the web application and stored locally on both. Data collection is session-based, typically starting with user interaction (like a button press or rotation) and concluding after a defined period of inactivity. The firmware records interaction types, relevant health sensor readings (such as heart rate metrics), and session timing information during active sessions. All sensitive session data is encrypted on the device prior to any transfer.

The web application manages secure data transfer and device configuration through a direct connection. This allows users to perform operations like registering their wallet address with the device, initiating data transfers to the backend system, and managing basic device settings.

Access control is anchored to the user's blockchain wallet throughout the system. The web application requires a wallet connection to interact with the smart contract, ensuring that functions related to data retrieval only grant access to information associated with the user's specific address. Similarly, accessing and decrypting the off-chain session data depends on information securely managed via the blockchain and accessible only through the user's wallet, reinforcing user control over their information.

The complete source code for the device firmware and the companion web application has been publicly available on GitHub [41,42].

3.2.3. Data Privacy and Control

The system employs a security architecture built on a two-layer encryption model and the principle of user-centric control. This design ensures the confidentiality of sensitive health data at rest and in transit by strategically separating raw data storage from on-chain records. All critical write operations are predicated on explicit user consent, which is enforced through cryptographic wallet interactions.

The first layer of security is implemented on the device's hardware, where each user session is encrypted using AES-256 before being written into internal flash memory. A deterministic AES key is derived for this purpose by computing the SHA-256 hash of a concatenated secret, which consists of a hardcoded firmware phrase and a unique symmetric key exchanged during the initial device pairing. A random Initialization Vector is generated for each session to ensure semantic security. The companion web application subsequently decrypts this data for user review by re-deriving the same AES key from its stored symmetric key.

The second layer of security governs the permanent storage of data. After user review, the compiled daily data object is uploaded to the InterPlanetary File System (IPFS) [43], keeping sensitive information off-chain. The IPFS content identifier (CID) is then encrypted on the client side using a newly generated, single-use AES key. The final step requires a user-initiated transaction, authorized via their cryptocurrency wallet. This explicit confirmation serves as an auditable, on-chain consent mechanism, tying data ownership to the user's wallet address. The transaction stores a secure pointer on the blockchain, consisting of the encrypted CID and the corresponding key required for decryption. This model ensures that while a record is publicly verifiable, only the user, possessing the necessary wallet credentials, can resolve the pointer and access the underlying health data.

3.3. Evaluation Studies

To evaluate the CipherPal prototype and the effectiveness of its integrated blockchain features, we employed three distinct evaluation methods, each providing a unique perspective on usability, acceptance, and real-world application.

3.3.1. Web3 Design Guidelines Evaluation

An expert evaluation was conducted involving three UX/UI design experts, including one of the researchers who was also involved in developing the Web3 Design Guidelines. The other two evaluators were independent UX/UI designers, each with at least four years of experience in interface design and prior work on Web3 projects.

The selection of three experts is consistent with practices for formative usability evaluation. The goal of this expert review was not to achieve statistical generalization, but to identify critical usability issues to inform design improvements. Research by Nielsen [44] demonstrates that a small group of 3–5 experts is highly effective for this purpose, capable of identifying most usability problems while balancing resource investment and problem discovery.

All evaluators applied the Web3 Design Guidelines framework [13], focusing specifically on the criteria outlined for the Essential implementation level, as CipherPal is currently a proof-of-concept. The evaluation procedure followed three phases. Initially, experts independently explored the CipherPal device and its companion web application, completing standard tasks like setup and data synchronization to familiarize themselves with the system from a new user's perspective.

The second phase involved a systematic assessment guided by the structured evaluation forms (Appendix A). The experts first used detailed checklist criteria derived from the Essential Level guidelines to assess the presence and implementation quality of specific functionalities and interface elements divided into 4 main sections (namely Onboarding, Transaction Flows, Settings & Configuration, and Other Features). Based on this detailed checklist assessment within each user flow, the experts then assigned an overall score for that section, using a scale from "0 - Not implemented" to "4 - Fully implemented with no issues", reflecting the quality of implementation for that specific flow or feature set.

The final phase consisted of semi-structured interviews where experts elaborated on their evaluations, discussed observed usability issues or strengths, and provided recommendations.

Data collection methods were employed to capture comprehensive insights. The completed structured evaluation forms documented the expert assessments against each criterion and the overall scores per section. Screen recordings captured the experts' navigation and interaction with the CipherPal system, while audio recordings preserved their think-aloud commentary during the evaluation. The concluding semi-structured interviews provided further qualitative context and clarification for the assessments documented on the forms.

This expert evaluation phase yielded foundational insights into CipherPal's adherence to established Web3 design principles and pinpointed specific areas for potential improvement.

3.3.2. User Acceptance Toolkit Assessment

To assess potential user acceptance and adoption factors specifically for the CipherPal prototype, the study utilized the User Acceptance Design Toolkit [14]. This toolkit, grounded in the UTAUT2 model [35], is designed to bridge the gap between established theoretical frameworks of user acceptance and practical design methodologies, particularly when dealing with complex emerging technologies like blockchain.

The assessment involved six professionals organized into three distinct pairs to ensure diverse perspectives and expertise. One pair comprised a software company CEO and an interaction designer (G1), combining business strategy and design expertise. The second pair included one of the researchers and a UX/UI designer (G2), providing both academic and practical viewpoints. The third pair comprised two UX/UI designers (G3), one with prior blockchain experience, contributing specialized interface design expertise.

The use of three distinct pairs represented a purposeful sampling strategy chosen to elicit rich insights from varied professional combinations. This approach aligns with qualitative methodologies where the focus is on the depth and range of perspectives captured from information-rich cases, rather than achieving statistical generalizability [38].

Each pair participated in a facilitated one-hour session conducted using the FigJam [45] digital whiteboard platform. These sessions followed a consistent structure: an introduction to the CipherPal prototype and its functionalities, an overview of the theoretical concepts underpinning the toolkit (primarily UTAUT2), and then collaborative application of the toolkit's various exercises and canvases to the CipherPal case. The toolkit exercises guided the pairs to analyze key areas relevant to acceptance, including user context, potential behavioral influences, identification of adoption barriers, and assessment of CipherPal's value propositions and perceived user benefits.

Data collection employed multiple methods. Completed toolkit exercise documents (FigJam canvases) captured the direct outputs of the collaborative work. Screen and audio recordings documented the entire session, including discussions and interactions with the toolkit. These recordings were subsequently transcribed.

The analysis process for this data was multi-stage. First, for each group (G1, G2, G3), relevant quotes concerning their inputs into the toolkit or their comments about the toolkit itself were extracted from the session recordings and transcripts. Simultaneously, the specific inputs provided by each group within the FigJam toolkit canvases were systematically listed.

Second, this prepared data (extracted quotes and listed inputs) was thematically analyzed within each group. This analysis focused on identifying patterns and insights related to ten predefined themes derived from the initial analysis of the raw data and the toolkit's structure: User Characterization, Adoption Barriers, Usage Context, Value Perception, Physical Design, Habit Formation, Blockchain-Specific Acceptance Factors, Social Perception and Stigma, and Design Recommendations.

Finally, the thematic findings from each of the three groups were compared and contrasted. This cross-group analysis aimed to identify areas of agreement and contradiction regarding predicted user acceptance, potential challenges, and design improvements for CipherPal.

This structured assessment phase provided deep insights into potential acceptance factors, benefiting from the varied expertise of the participants. The process allowed for both structured input via the toolkit and rich qualitative discussion, while the detailed analysis method enabled a systematic comparison of perspectives across the different professional pairings.

3.3.3. Extended User Testing

An extended user testing study was conducted to gain insights into the real-world usage patterns, practical usability, and evolving perceptions of CipherPal over time. This extended study involved five participants, aged between 24 and 36, who integrated the device into their daily routines for a period of 3 days.

The selection of five participants is grounded in established principles for formative usability research. Foundational work by Nielsen and Landauer [46] demonstrates that testing with approximately five users is sufficient to uncover about 85% of the usability problems in an interface, providing substantial insights for iterative improvement within the practical constraints of a prototype study. This sample size was also deemed sufficient for this specific extended study, as it followed earlier phases of iterative development that included user testing and feedback on previous prototype versions [12].

The methodology was designed to capture naturalistic interaction and gather feedback through multiple channels. The study began with an individual, researcher-led onboarding session. During this session, participants received the CipherPal device. The researcher explained the study's purpose, the device's functionality, the specific data collection and blockchain storage mechanisms (including data encryption, IPFS [43] storage, and on-chain record management), and the daily actions participants needed to complete.

Given the prototype stage and minimal functionality present within the companion web application itself for guiding onboarding, the researcher actively assisted participants with the initial setup. This included guidance on installing required software (such as MetaMask [47]), creating a new wallet if needed, and securely sharing their wallet address so the researcher could transfer a small amount of testnet cryptocurrency necessary for covering blockchain transaction ("gas") fees during the study. Participants then completed an initial Onboarding Survey (Appendix B), capturing baseline data regarding their comfort with technology, prior experience with related technologies, and current health tracking habits.

Throughout the three-day usage period, the researcher remained available to participants via preferred communication channels to answer questions and provide necessary troubleshooting assistance. Relevant informal feedback and researcher observations noted during these support interactions were also documented to complement the structured data collection.

For the study period, participants used the CipherPal device as they normally would. Daily device usage patterns, including frequency and duration of interaction sessions and specific interaction types (rotations, clicks), were logged directly by the device firmware. Each day in the evening, participants completed Daily Reflection forms (Appendix C), providing regular feedback on the device's usability, factors influencing acceptance and adoption (such as comfort using it in public or perceived importance of blockchain storage), and any barriers encountered.

To ensure exploration of key blockchain-related features, participants were prompted on the second day to explore the data access request functionality and on the third day to try annotating a fidget session, unless they had already done so independently.

User interactions with the companion web application (Figure 4) were also logged using the analytics tool Microsoft Clarity [48] to understand navigation and feature usage. Upon completion of the usage period, semi-structured interviews were conducted with each participant. These interviews provided an opportunity to delve deeper into their overall experience, exploring themes that emerged from their daily reflections and gathering rich qualitative feedback on the prototype's performance, perceived value, and the impact of its blockchain integration.

This multi-faceted approach yielded a comprehensive dataset comprising quantitative and qualitative information. The data collected included responses from the onboarding and daily surveys, detailed interaction logs generated by the device firmware, web application usage analytics, and transcripts from the final semi-structured interviews, allowing for a thorough analysis of the user experience over time.

3.4 Data Analysis

Consistent with the convergent mixed-methods design, the data analysis strategy involved parallel processing of the quantitative and qualitative data gathered from each of the three evaluation frameworks, followed by a comprehensive synthesis phase.

3.4.1. Individual Framework Analysis

The analysis was performed on a rich, mixed-methods dataset from the three distinct evaluation frameworks. Data from the Web3 Design Guidelines evaluation included quantitative structured evaluation forms, screen and audio recordings of the expert sessions, and transcripts from the concluding semi-structured interviews. The User Acceptance Toolkit assessment yielded qualitative data in the form of completed digital whiteboard (FigJam) canvases and full transcripts from the facilitated group discussions. Finally, the extended user testing produced a diverse dataset comprising: quantitative responses from onboarding and daily surveys, web analytics data capturing application usage, and qualitative data from open-ended survey responses, researcher support notes, and transcripts from the final semi-structured interviews.

As dictated by the convergent mixed-methods design, the data collected from each of the three evaluation frameworks were first analyzed independently to understand the unique insights offered by each perspective.

The analysis of the Web3 Guidelines evaluation data proceeded in stages to provide both quantitative and qualitative insights into CipherPal's alignment with the Essential Level guidelines.

First, the structured evaluation forms completed by the three experts were examined to assess the implementation status of specific guideline criteria within each defined user flow (e.g., Onboarding, Transaction Flow, Settings & Configuration). This involved tallying which checklist items corresponding to specific guideline recommendations were identified by the experts as being present or absent in the prototype's features.

Following this initial assessment, the overall quantitative scores (on a scale from 0 - "Not implemented" to 4 - "Fully implemented with no issues") assigned by each expert to these distinct user flow sections were analyzed. Average scores were calculated for each section to gauge the perceived level of implementation quality, and the level of agreement between the three experts on these scores was assessed to understand the consistency of the evaluation.

Finally, a qualitative thematic analysis was conducted on the rich data from the think-aloud commentary captured during the session recordings to understand the context and reasoning behind these assessments. This thematic analysis focused on identifying specific usability issues, expert interpretations of guideline adherence, perceived strengths of the implementation, and concrete suggestions for improvement mentioned by the experts.

For the User Acceptance Toolkit assessment, thematic analysis was the primary method applied to the data captured in the completed FigJam canvases and the transcripts of the group discussions. This analysis aimed to extract key themes related to predicted adoption factors, user concerns, and potential adoption barriers, interpreted through the lens of the UTAUT2 constructs embedded within the toolkit.

Finally, the rich dataset from the extended user testing underwent a mixed analysis approach. Quantitative data included usage metrics (interaction frequency) from device logs rendered within the web app and numerical survey responses (e.g., Likert scales), analyzed descriptively.

Web analytics data from Microsoft Clarity underwent specific processing: records from non-participants and participant sessions showing no user interaction (no clicks or mouse movement) were excluded. The remaining Clarity data and survey responses were used to analyze navigation patterns and specific user action flows.

This involved identifying sequences related to core tasks such as attempting data uploads, successfully uploading data, exploring shared data access features (requesting, approving, rejecting access), attempting to annotate sessions, exploring previous records, and instances indicating error recovery. A key limitation acknowledged is that interactions solely within the browser-based wallet extension (e.g., MetaMask confirmations) could not be directly tracked by Clarity or the web application logs.

Qualitative data from open-ended survey responses, researcher notes from support interactions, and final semi-structured interview transcripts underwent thematic analysis. This analysis focused on understanding evolving user experiences, usability perceptions, and challenges, specifically coded against predefined themes derived from the research questions and initial data review: physical device experience, data ownership and privacy perceptions, application-device connection usability, social context and usage patterns, overall value perception, and encountered technical issues.

3.4.2. Cross-Framework Synthesis

Following the individual analyses, the findings from the three frameworks were synthesized to develop a holistic understanding. This synthesis focused on identifying areas of convergence, where findings from different methods reinforced each other, thus strengthening the validity through triangulation.

Equally important was the analysis of contradictions between the frameworks, highlighting each perspective's unique insights and potential tensions between technical compliance, predicted acceptance, and real-world usability. This integrated analysis combined technical implementation insights (primarily from the guidelines evaluation) with experiential data to formulate comprehensive and well-grounded recommendations.

The overall effectiveness of the CipherPal prototype and the multi-framework evaluation approach was assessed against several key validation criteria derived from the research goals. These included evaluating the technical efficacy of the prototype's implementation, assessing the quality of the user experience, analyzing the specific impact of blockchain integration on aspects like trust and data control, identifying patterns in feature utilization, and looking for indicators pointing towards potential user adoption or rejection. These criteria guided the final interpretation of the synthesized results.

4. Results

4.1. Web3 Design Guidelines Evaluation Analysis

The expert evaluation of the CipherPal prototype against the Essential Level of the Web3 Design Guidelines yielded quantitative scores and identified specific areas of adherence and deviation. Analysis of the implementation status and expert scoring revealed varying levels of compliance across different user interaction flows.

4.1.1. Onboarding Flow

Expert evaluation of the prototype revealed that its various features and potential issues were identified by differing numbers of experts, with qualitative feedback providing deeper context. Table 1 summarizes how many experts identified each key aspect.

In User Registration, core elements such as support for primary wallet types, a clear connection status display, and consistent interaction models were identified by all three experts (3/3). This successful identification of foundational ease-of-use was echoed by Expert 2, who noted, "I just connect it and it works, and my wallet instantly worked.” However, fewer experts identified issues related to handling connection errors (2/3) and adapting to varying user expertise or knowledge gaps (1/3). Notably, no experts identified clear error recovery paths (0/3), indicating a significant oversight in the prototype.

For Wallet Connection and Management, all three experts identified clear affordances and status displays (3/3). In contrast, two of the three experts pointed out the need for network compatibility indication and improved error handling (2/3). A practical challenge in this area was highlighted by Expert 2 (E2), who reported that "Phantom Wallet did not support the testnet," underscoring a specific compatibility issue identified.

The evaluation indicated that aspects of Trust Building require significant attention. The presence (or importance) of transparent data handling policies was identified by two of the three experts (2/3), with Expert 2 voicing related uncertainty: "Not sure about providing transparent data policies." Furthermore, clear recovery mechanisms were identified by only one of the three experts (1/3).

Similarly, within Value Building, the prototype's effective use of familiar Web2 patterns was recognized by all three experts (3/3). However, the explicit communication of Web3 benefits and specific value propositions was less consistently identified as being present or effective, with two of three experts (2/3) noting these aspects. Expert 3 (E3) suggested an improvement: "The value building can be more visual, there is a lot of text on it."

Progressive Feature Introduction presented a varied picture in terms of expert identification. While the management of cognitive load during onboarding was identified as being addressed by all experts (3/3), the provision of contextual learning was not identified by any expert (0/3). This absence was keenly felt, as Expert 3 observed, "People that use Web3 for the first time might get confused, I don’t remember if there was any explanation on that." Other desirable aspects, including point-of-need explanations, progressive disclosure for complex features, success verification mechanisms, and human-readable data formats, were each identified as areas for consideration or improvement by two experts (2/3). Reinforcing this, Expert 3 suggested, "Maybe when you are uploading the data, it can explain what this data is for,” highlighting a missed opportunity for point-of-need information.

Finally, elements of Security Awareness Building were partially identified. The explanation of data usage was noted by all three experts (3/3). However, two of the three experts identified the need to address specific Web3 concerns, show data permanence, or present privacy implications (2/3).

4.1.2. Transaction Flow

CipherPal strongly adhered to Transaction Initiation and Confirmation guidelines in the Transaction Flow, achieving solid expert agreement (3/3) on presenting clear signing requests, explaining transaction details, showing fee estimates, and providing confirmation steps.

Status Visualization and Updates during transactions were also generally well-implemented, with an agreement (3/3) on showing network status, transaction states, process status, and network confirmations. Areas identified for potential improvement within this flow included the clarity of blockchain state indication, support for in-app feedback related to wallet requests, and the presentation of human-readable transaction data, each identified by two out of three experts (2/3).

While the core transaction steps were clear, experts pointed out potential confusion points and missing information. For instance, the use of technical jargon like blockchain explorers needed refinement: "What is Polygonscan [refers to the explorer link in the dashboard with the copy ‘See on Polygonscan’]? Copywriting can be a bit different" (E3).

4.1.3. Setting and Configuration

Referring to Settings and Configuration, the evaluation revealed lower overall implementation levels. Network Interaction Management features, such as supporting network switching or handling network errors, were minimally identified (1/3 agreement). While showing the current network clearly and indicating network requirements was noted by two experts (2/3), robust functionality in this area appeared underdeveloped.

Privacy Controls and Preferences also showed limited implementation. Privacy setting enablement was not identified (0/3), data visibility options received partial agreement (2/3), and preference customization was minimally recognized (1/3). Within Network and Wallet Management features in the settings context, support for primary wallet types was consistently noted (3/3). Still, capabilities related to showing connection status or handling errors were less evident (2/3), and support for network switching was very limited (1/3).

Data Visibility and Control showed partial implementation, indicating where data will be written, and achieved a unified agreement (3/3). In contrast, features like indicating data origin (1/3), clearly explaining data usage (1/3), and permanence (2/3), or enabling basic permission management (2/3) were less consistently recognized.

Feedback on settings highlighted missing functionalities and information. Experts suggested future enhancements like multi-chain or alternative wallet support. A lack of wallet details within the settings was also pointed out: "Maybe this app is missing information about my Wallet" (E3). The need for better wallet integration, which was mentioned during onboarding feedback, was echoed here.

4.1.4. Other Features

Regarding Other Features, the evaluation unanimously confirmed (3/3 experts agreeing "None" were implemented) that the current CipherPal prototype does not yet incorporate any Basic Governance Features. While confirming the absence of governance features, experts offered differing opinions on their necessity and potential implementation. One stated, "I think you don’t need governance and community", while the other saw potential: "Some governance and community features can be implemented" (E2) or implementing "a common canvas with anonymised activity of app use” (E1).

4.1.5. Overall Assessment Scores

Quantitatively, the prototype achieved an overall average implementation score of 2.83 (on a 0-4 scale) across all evaluated Essential Level guidelines (Figure 5). This score suggests a high level of partial implementation, indicating the system is generally functional relative to essential guidelines but possesses areas needing refinement or minor issue resolution.

The evaluation areas demonstrating the highest scores, reflecting stronger alignment with the guidelines, were User Registration (average: 3.67), Transaction Initiation and Confirmation (3.50), and Wallet Connection and Management (3.33). On the contrary, the areas identified as requiring the most significant development were Progressive Feature Introduction (2.00) and Network Interaction Management (2.00) (Figure 6).

Inter-rater agreement on these scores was generally high across most evaluated sections. Score variations between the three experts typically remained within +/- 1 point, often with two experts assigning identical scores. However, greater divergence in expert opinion, indicating less consensus on the implementation quality, was noted specifically for Progressive Feature Introduction, Security Awareness Building, and Privacy Controls and Preferences, where one evaluator consistently provided lower scores compared to the other two. This quantitative assessment underscores the prototype's strengths in core transaction and registration flows while highlighting opportunities for enhancing user guidance, network management, and privacy controls.

Overall, the experts found the prototype promising and easy to interact with once understood. Its innovative approach and potential real-world applications were praised: "It is very innovative because it is combining a toy with data collection" (E3), "It is a very safe way of sharing your data with doctors" (E3). Potential use cases like pharmacy verification were mentioned: "Pharmacies can use it and see if I have the diagnosis related to the medicine and it was inserted by the hospital" (E2).

Experts expressed interest in its future and suggested further exploration. The potential for features like data tracking for motivation and the broader implications for data verification in an era of AI were also touched upon "It makes me think how it can be important with verifying that the data is real and the source of data, and now it is a trouble that LLMs can replicate some documents and data" (E2).

4.2. User Acceptance Toolkit Assessment Results

The User Acceptance Toolkit assessment, involving three professional groups (G1, G2, G3), yielded significant insights regarding potential adoption factors for the CipherPal prototype. Analysis of the toolkit outputs and discussions highlighted key perspectives on target users, anticipated usage patterns, and potential barriers grounded in the UTAUT2 framework [35].

4.2.1. User Characterization and Adoption Barriers

A consensus emerged across groups characterizing the target user as technologically proficient or an "early adopter". Group 1 (G1) narrowed the focus to young females with specific medical monitoring needs, while Group 2 (G2) proposed a broader 18-30 age range across genders, and Group 3 (G3) suggested a wider 16-45 age range. This convergence implies that CipherPal's blockchain features might pose an initial adoption challenge for users less familiar with advanced technologies.

The toolkit facilitated the identification of potential adoption barriers rooted in UTAUT2 constructs, with variations observed between groups. G1 initially flagged Facilitating Conditions, Price Value, and Habit, later refining their focus to Hedonic Motivation and Habit. G2's initial concerns centered on Performance Expectancy, Effort Expectancy, Facilitating Conditions, and Habit, later revised to Effort Expectancy, Social Influence, and Habit. G3 identified the widest array of initial barriers (Social Influence, Facilitating Conditions, Hedonic Motivation, Habit), subsequently expanding to also include Performance Expectancy, Effort Expectancy, and Price Value.

4.2.2. Usage Context

Dual usage scenarios — health tracking and stress management/fidgeting — were consistently identified across all groups, pointing towards a hybrid value proposition. This duality also implies targeting users comfortable with novel tech, as noted by G1: "You need to be a very early adopter, like someone that really does advanced stuff to use a device like that."

The anticipated context of use surfaced potential challenges, particularly regarding social perception in public environments, as highlighted by G2 and G3. G3 articulated this concern vividly: "In some context, it could have a negative impact: school > bullying, work > bad image with superiors and colleagues." This finding suggests that Social Influence could be a more critical factor than perhaps anticipated during initial design.

4.2.3. Physical Design and Habit Formation

The assessment underscored the critical role of CipherPal's physical design in driving acceptance and enabling habit formation. G1 emphasized that enjoyment is key, stating users might "stop using it if it is not fun," recommending "more haptic feedback and tactile experience." G2 acknowledged the "very stimulating tactile feedback" but cautioned it might be "quite distracting [for others]." G3 explicitly linked physical appeal to user persistence, concluding "the physical sensation of the product is important" because a lack of appeal could deter users from overcoming the "learning curve of uploading and learning all the features."

Consequently, the physical form factor emerged as crucial for sustained use. Consistent recommendations arose across groups for alternative designs to improve integration into daily life and promote habit formation. Suggestions included "key chain" or "phone cover" formats (G1), functioning as a "token or accessory" (G2), or adopting a "pebble-type form [that] could be better for hand feeling" (G3).

4.2.4. Blockchain-Specific Acceptance Factors

Despite blockchain's centrality to CipherPal's concept, the toolkit assessment indicated it was not a primary focus for participants when evaluating user acceptance. Discussions around blockchain's technical demands were often implicitly covered under Effort Expectancy or Facilitating Conditions. G1 briefly mentioned data security concerns like "ethical issues related to the protection of sensitive data." At the same time, G2 noted the potential for users "becoming self-aware about their data," hinting at data ownership benefits. However, explicit, detailed discussions on blockchain's unique value propositions as acceptance drivers were largely absent, suggesting that these evaluators might not perceive its current implementation as a core user benefit.

4.2.5. Social Perception and Stigma

A significant finding from the assessment was the potential social stigma associated with using CipherPal. G2 raised concerns that the device "makes it obvious for others that the user has mental health issues" or could lead to perceptions of the user being "childish." G3 echoed this, noting "others might start thinking weirdly about the user, and the probability is considerable, 50/50."

This social dimension was strongly emphasized by G3, who reflected: "The social aspect of the object is huge... how people perceive you when you're using the product," directly linking this to aesthetics: "the aesthetic dimension of the project is huge because of the social implications." These insights suggest the current prototype's design may not adequately address social acceptance, potentially hindering adoption, particularly in shared or public spaces.

4.2.6. Design Recommendations

The insights gathered through the toolkit assessment culminated in several concrete design recommendations, primarily focusing on improving the physical attributes, user engagement, and social integration of CipherPal, rather than modifications to its core blockchain functionality. A strong, consistent theme across all groups (G1, G2, G3) was the need to reimagine the device's physical form.

Participants suggested exploring alternative, potentially less conspicuous designs such as phone cases, key chains, or smoother 'pebble' shapes to facilitate use better and improve discretion in various social contexts. Relatedly, specific advice emerged, notably from G2, to make the device "as low key as possible," potentially even "switching off the LEDs," further emphasizing the desire for subtlety. Complementing these form factor suggestions, G2 and G3 proposed incorporating personalization options, which could foster a stronger user connection and help mitigate potential social stigma.

Beyond the overall shape and visibility, the sensory experience was highlighted. G1 specifically recommended enriching the interaction with "more haptic and tactile experience," underscoring the importance of the device's hedonic qualities for sustained engagement. Recommendations also addressed engagement strategies to support user motivation further and address potential Effort Expectancy barriers identified earlier.

G2 raised the possibility of exploring data monetization through "regular payments to the users," while G3 focused on making the collected data itself more compelling through improved "data visualization." Crucially, G3 also stressed the necessity of "proper onboarding and user support" to ensure users can navigate the initial learning curve smoothly, linking back to the importance of Facilitating Conditions for adoption.

4.3. Extended User Testing Findings: Usability, Acceptance, and Adoption

The extended user testing study provided rich insights into how participants interacted with the CipherPal device and companion application over three days in their daily lives. Analysis of onboarding and daily surveys, web analytics (Clarity), conversations from participants, and semi-structured interviews revealed key findings related to usability, acceptance, and potential adoption barriers.

4.3.1. Physical Device Experience and Interaction

Participants generally responded positively to the physical interaction with CipherPal. The tactile feedback derived from the rotation mechanism was frequently described as "satisfying" (P5), "nice" (P3), and ultimately "comforting" (P1) after an initial adaptation period.

The device's ergonomics were also deemed suitable by most users, with P3 commenting that "the movement of the hand and the size of the device. You got it very right." Similarly, P4 appreciated the specific interactions, stating, “I like the double physical interaction—the soft button and spinning.”

The initial setup was perceived positively, as reflected in the Onboarding Survey results, which showed high ratings for the Setup Experience (60% Very Good, 40% Excellent). Despite these positive initial impressions, several usability challenges related to the physical design emerged during the extended use period.

The design incorporating two counter-rotating parts, while providing the core tactile experience, was noted by P1 as making the device "easy to drop" and occasionally led to parts detaching, although P3 found reassembly straightforward.

A more consistent complaint across multiple participants (P1, P3, P5) concerned the LED's brightness and remaining constant illumination. The lights were frequently described as "too bright" and "annoying," which impacted comfort when using the device in public spaces or at night, leading P5 to turn it away. Reinforcing this desire for discretion, P4 noted, “I'd rather it didn't have lights, so it's more discreet and people don't get too close to use it.”

Issues related to size and portability were also raised, while the dimensions suited some users' hands, P5 found it "a bit too big", which suggests a potential need for a smaller form factor for better pocketability.

Finally, participants encountered practical difficulties with charging and battery management (P5), noting the general inconvenience of charging another device (P3) and specific issues like the inability to charge when the connected computer was off (P5).

4.3.2. Application Interaction, Connection, and Data Management

Interaction with the companion web application and the processes for connecting the device and managing data proved to be a source of usability friction. While participants generally appreciated the ability to view their collected data, the visualization features were often found lacking. Many expressed a desire for more granular detail and improved visualizations.

For instance, P3 specifically requested visibility of precise interaction counts ("I want to know how many times I rotated the device... Each thing I do, I want to know how many I was doing"), while P5 suggested integrated views combining fidgeting data with other health metrics ("I would like to see everything in the same graph, like my glucose, my heart rate, and the spins").

Compounding this, some participants found the presented data confusing, citing instances of illogical heart rate readings (e.g., 0 BPM or impossibly high values reported in a Daily Survey) or unclear session durations (P3). Daily survey results reflected this mixed experience regarding data access, with 40% of responses strongly agreeing they could access data easily, but 20% disagreeing and others remaining neutral or only moderately agreeing.

The device-application connection process emerged as a primary pain point. The need to connect the device via the web app to upload data or to enable data recording was considered counterintuitive. "It was weird that I had to sign in to the app for it to start recording data" (P1).

The automatic disconnection after data upload was particularly perplexing, leading users like P1 and P2 to make repeated, confused attempts to reconnect (P1, P2). Device connection problems were frequently encountered, manifesting with wrong readings of the wallet stored on the device through Serial, noted particularly on Windows devices (P4, P5), which often required a browser restart to resolve.

Other connection barriers included issues arising from Metamask being inadvertently signed out (P3). These frustrations were occasionally visible in usage analytics, such as "rage clicks" observed when P3 struggled with a non-responsive wallet connection button.

Further complications arose during the data upload process. Participants like P2 found the manual, multi-step procedure burdensome, involving explicit user action and blockchain transaction confirmations, and expressed a desire for automated uploads. The process was also prone to errors and ambiguity.

Specific failures included uploads being blocked by insufficient testnet funds (P2), bugs leading to data being overwritten (P1), or saved to the incorrect date (P5). The lack of feedback during the upload, such as a loading indicator, added to the uncertainty. "There was no loading spinner to indicate if the data was being uploaded" (P4). Web analytics also captured instances of "dead clicks" during the upload flow for multiple users (P1, P3, P4, P5), suggesting confusion or issues with UI element responsiveness in this critical step.

4.3.3. Usage Patterns

Analysis of usage patterns revealed that CipherPal found its niche primarily within work or home environments, often integrated into desk-based activities. Participants frequently reported using it while working on their computers (P3, P5), browsing the internet, or watching videos (P1), highlighting its role as an accessible object for fidgeting during periods of low physical activity.

Its constant availability was noted, with P2 mentioning it was "always there" and P3 keeping it nearby on their desk. Some participants engaged in extended or continuous use, with P5 reporting using it for "hours" while working, and both P1 and P5 mentioned holding it in one hand while performing other tasks like eating dinner or using two hands for work. P4 echoed this, finding it 'a fun toy to use when I was taking a break' and specifically envisioned its utility in stressful situations: “I feel like I'd use it a lot more at work, especially during meetings, which are the most stressful for me.”

Daily Survey data provided quantitative insights into these patterns: usage frequency typically ranged from 1-3 times per day (33% of responses) to a more common 4-6 times (53%), with only 13% reporting higher frequencies. Most interaction sessions were relatively short, typically lasting 1-5 minutes.

The rotating interaction was dominant, cited in two-thirds (10/15) of daily reflections, with the remainder reporting a mix of interactions. Usage trends over the three-day study varied between participants, with some showing increased engagement and others a decrease, possibly reflecting a novelty effect or evolving integration into routines.

4.3.4. Social Context

The social context of CipherPal usage presented a more complex picture. While Daily Surveys indicated a reasonable level of comfort using the device in public (73.3% agreed or strongly agreed), qualitative data revealed significant caveats.

The primary obstacle was the device's evidence due to the bright LEDs. P1 explicitly stated, "I feel like people will notice me more with the lighting," and consequently stopped attempting to use it during their commute. This sentiment was echoed by others (P3, P5) who found the lights problematic.

Despite these concerns about drawing attention, the device could sometimes facilitate social interaction. P1 described it as a potential "icebreaker," noting that its unusual appearance prompted questions. P5, however, observed that while co-workers were interested when shown the device, they didn't inquire unprompted during normal use.

Participants like P1 focused on the tangible fidgeting aspect and basic data collection, often simplifying or omitting the underlying blockchain technology when explaining the device.

4.3.5. User-Perceived Value and Factors Influencing Adoption

The primary perceived value of CipherPal, acting as a potential driver for adoption, was consistently identified by participants as its positive impact on mental well-being and focus. Daily reflections frequently highlighted benefits such as stress and anxiety relief ("keeping my fingers occupied while working, lowering a bit of stress," "anxiety relief"), focus enhancement ("helped me stay in focus"), and providing a general calming effect.

P5 underscored this therapeutic potential, finding it helpful for managing stress and compulsive behaviors ("skin picking issue"). Beyond these core benefits, factors like inherent enjoyment (P3 finding it "like a game"), the satisfaction derived from the tactile interaction, and increased self-awareness of fidgeting habits (P1) also contributed to the device's perceived worth.

Several factors emerged that could facilitate adoption. According to Daily Surveys, users expressed confidence in the device's potential to fit into their daily lives, with 73.3% anticipating easy routine integration. Furthermore, despite documented usability hurdles, a significant majority (80%) felt the device was "worth the effort to use," suggesting the perceived benefits currently outweighed the operational friction for many. The perceived ease of the core interaction (66.7% strongly agreeing it was easy to use) also lowers a potential barrier, even if technical complexities exist elsewhere in the system. P4 confirmed this: "The whole experience was easy, especially when using the device.”

However, significant barriers and considerations for long-term adoption were also evident. The substantial usability challenges related to the application, device connection, data management, and physical design limitations like LED brightness and charging inconveniences threaten sustained use. Participant uncertainty about long-term benefits (53.3% were Neutral in surveys) further tempers adoption prospects.

Suggestions for improvement, such as P3's desire for a more "developed product" or P5's interest in deeper data integration, highlight features potentially necessary to convert initial interest into lasting engagement. Finally, the possibility that the observed positive engagement was influenced by a novelty effect (P1) represents a risk to predicting genuine long-term adoption without addressing the core usability and design issues. For instance, P4 noted the challenge of integrating it into daily routines outside the home: “The hardest part was remembering to bring it with me when I left the house. I think it would be a habit to learn.”

4.3.6. User Perception of Blockchain Interactions

While participants generally demonstrated a conceptual understanding that CipherPal utilized blockchain for data storage (P1, P3, P5), their practical interaction with blockchain components primarily translated into usability friction rather than a perceived improvement of the user experience.

The necessary steps involving blockchain technology frequently manifested as obstacles during routine use. The mechanics of on-chain data storage introduced complexities unfamiliar to typical consumer device interactions. The need to acquire and manage testnet cryptocurrency to pay for transaction fees, and the consequence of uploads failing due to insufficient funds (P2), added a layer of technical overhead.

The multi-step confirmation process required via the wallet extension for each data upload was also noted as heavy (P2), interrupting the user flow and adding perceived effort without an immediate, tangible benefit for the user interacting with the fidget device. Participants like P1 tended to simplify explanations of the device to others, often omitting the blockchain details, suggesting the technical implementation wasn't seen as a core selling point or easily communicable feature.

Interestingly, despite these interaction challenges, user acceptance of the concept of blockchain storage was generally positive. Daily Surveys revealed high comfort levels, with 60% strongly agreeing and 20% agreeing that their data is being stored on a blockchain. However, opinions were more divided regarding the perceived importance of blockchain for this specific application, with 40% remaining Neutral and only 46.6% agreeing that it was important.

This suggests that while users were not inherently opposed to blockchain technology, its current implementation within CipherPal added complexity and friction that overshadowed any perceived advantages for the end-user experience during the study period. The primary user engagement remained focused on the physical interaction and basic data viewing, with the blockchain layer operating more as a background hurdle than a value-adding feature in its current form.

4.3.7. Synthesis of Extended User Testing Insights

The three-day extended user testing painted a clear picture of CipherPal: a device with a compelling core value proposition centered on tangible interaction and perceived well-being benefits, currently hindered by significant, yet addressable, usability and design challenges.

Participants consistently responded positively to the physical, tactile experience of the device, describing the rotational feedback as satisfying and comforting. This positive interaction translated into perceived benefits like stress reduction, with some users finding genuine therapeutic value.

The device integrated reasonably well into desk-based work routines due to its constant availability, leading most participants to feel the experience was "worth the effort" despite difficulties, and all expressed interest in continuing use post-study, underscoring the strength of its fundamental appeal.

However, this positive core experience was significantly undermined by critical barriers primarily related to the digital ecosystem and specific physical design choices. The most pervasive frustrations stemmed from the software and connectivity aspects: the connection process was confusing, wallet interactions were prone to errors, and the manual, multi-step data upload process felt cumbersome and unreliable.

Furthermore, the excessively bright LEDs emerged as a major obstacle, impacting user comfort and discretion, particularly in public or low-light conditions. Inconveniences related to charging, and to a lesser extent, concerns about handling and portability, added further friction. These cumulative issues represent substantial obstacles to seamless integration into daily life and sustained adoption.

These findings provide clear, actionable insights for future development. To realize CipherPal's potential, addressing the identified usability friction and design flaws is paramount. Key priorities should include drastically simplifying the device connection and data synchronization processes, potentially automating uploads, and eliminating complex blockchain transaction confirmations from the direct user flow.

Refining the physical design to incorporate adjustable or dimmer LEDs, improve the charging experience, and possibly explore alternative form factors is also crucial. Improving the companion application to offer more insightful and customizable data visualizations would further increase user engagement.

5. Discussion

This study employed a multi-framework validation approach to evaluate CipherPal, a blockchain-enabled smart fidget toy, integrating perspectives from Web3 design guidelines, user acceptance modeling, and extended user testing. The aim was to gain comprehensive insights into the challenges and opportunities of designing blockchain-based health devices, focusing on the interplay between technical implementation, usability, and user acceptance. This section discusses the synthesized findings, their implications, the study's limitations, and directions for future research.

5.1. Synthesis of Findings Across Evaluation Frameworks: Complementary and Contrasting Insights

Integrating three distinct evaluation frameworks provided a richer, more nuanced understanding of CipherPal than any single method could offer, directly addressing our first research question regarding how insights from different frameworks complement or contradict each other.

Several key themes emerged consistently across these frameworks. For instance, the importance of the physical design and tactile experience was highlighted both in the User Acceptance Toolkit assessment, where it was linked to habit formation and overcoming learning curves, and confirmed by user testing participants, who found the interaction satisfying and comforting despite ergonomic issues.

Furthermore, the need for improved onboarding and user guidance, particularly regarding blockchain concepts, was identified by experts evaluating against Web3 Guidelines, echoed in the Toolkit assessment's recommendations for support, and reflected in user testing difficulties with setup and connection. Additionally, usability issues related to device-application connection and data synchronization were flagged as potential barriers in the Toolkit assessment (implying high Effort Expectancy/low Facilitating Conditions) and strongly manifested as major friction points during user testing.

The value of the multi-framework approach is evident in two key patterns that emerged from the analysis: the validation of findings through convergence, and the identification of critical issues through divergence.

Convergence occurred when a core theme was identified independently across all three frameworks, strongly validating its importance. For instance, the need for improved onboarding and the challenges with device-application connection were first flagged as guideline violations by experts, then predicted as a significant adoption barrier by the User Acceptance Toolkit, and finally confirmed as a primary source of real-world frustration during user testing.

Conversely, divergence was crucial for uncovering issues that a single methodology would have missed. Some of the most significant barriers were found by only one framework, yet were raised by multiple participants within it, proving their severity. For example, while the expert review focused more on the digital interface, the extended user testing explicitly designed to capture initial usability, first impressions, and immediate barriers to adoption rather than long-term habit formation was the only method to reveal that the device's bright LEDs were a major physical and social deterrent, a critical finding for the device's real-world use that would have otherwise been overlooked.

Naturally, the frameworks emphasized different aspects due to their distinct goals. Experts focused on guideline compliance and technical implementation quality. Toolkit participants concentrated on predicting user behaviour, motivations, and barriers. Users, understandably, centered on their immediate experience: the satisfying fidgeting, the frustrating connection issues, and the perceived personal benefits. This synthesis demonstrates that a multi-framework approach effectively triangulates findings, providing a holistic view that balances technical requirements, adoption predictions, and lived user experience.

5.2. User Experience, Acceptance, and the Perceived Role of Blockchain

The synthesis of these frameworks revealed a central narrative in CipherPal's evaluation, which is best understood through the lens of the target user identified during the study: the technologically proficient 'early adopter'. From this perspective, the evaluation showed a compelling core user experience rooted in physical interaction, which was consistently coupled with significant usability challenges linked to its digital and blockchain components. Participants consistently valued the device's tactile nature and perceived benefits for focus and stress relief, aligning with the intended "fidget toy" aspect and suggesting strong potential related to Hedonic Motivation. This core appeal was strong enough that most users felt the device was "worth the effort" and expressed interest in continued use despite frustrations.

However, the integration of blockchain technology, while conceptually accepted by users, manifested primarily as a source of friction in practice. Addressing our second research question on user trust and engagement, we found that while users were comfortable with the idea of blockchain storage, their practical engagement with blockchain features (wallet connections, transaction confirmations, fee management) was overwhelmingly negative, characterized by confusion, errors, and perceived effort.

This suggests that the current implementation significantly increased Effort Expectancy and suffered from poor Facilitating Conditions, hindering rather than enhancing the user experience. The potential benefits of blockchain, such as improved data control and security emphasized in the literature [8,16], were overshadowed by these immediate interaction barriers. Users did not strongly perceive the Performance Expectancy benefit of blockchain in this context, often omitting it when explaining the device.

Trust in the concept did not translate into positive engagement with the implementation. This highlights a critical challenge identified by prior work [23,25], bridging the gap between blockchain's potential and its usable application for non-expert users.

5.3. Effectiveness of the Multi-Framework Validation Approach for Blockchain Integration

Our findings strongly support the value of using multiple evaluation perspectives to improve blockchain integration in healthcare devices (RQ3). The complexity of merging physical interaction, health data, and decentralized technology necessitates insights beyond any single methodology.

The Web3 Design Guidelines ensured foundational blockchain interaction patterns were considered, identifying technical and UX debt early. The User Acceptance Toolkit prompted proactive thinking about adoption drivers and barriers, crucially highlighting social factors and habit formation. The Extended User Testing provided essential grounding, revealing how theoretical benefits and guideline adherence translated (or failed to translate) into practical usability and user value. For instance, while the guidelines evaluation noted partial implementation of transaction flows, user testing revealed the experiential burden of these flows.

This triangulation [38] allowed for cross-validation (e.g., physical design importance) and revealed tensions (e.g., conceptual acceptance vs. practical friction of blockchain). By combining these perspectives, we gain a more robust understanding necessary for developing technically sound, user-friendly, and contextually appropriate solutions, addressing the research gaps identified.

5.4. Implications for Design and Development

To translate our findings into a clear roadmap, it is crucial to disentangle the identified barriers into two distinct categories. The first includes implementation-specific issues related to the prototype's fidelity. These encompass problems noted during user testing, such as device-application connectivity errors, data synchronization failures, and charging inconveniences. Such issues, while significant, can be addressed through further engineering and refinement.

The second category involves more fundamental conceptual challenges inherent to the design and technology choice. This includes the cognitive load and interaction friction introduced by the blockchain components, manifesting as a key source of user frustration. It also contains the socio-cultural challenges related to the device's physical design, such as the potential for social stigma, which was flagged in the toolkit assessment and confirmed by users' reactions to the device's conspicuous LEDs.

This distinction is critical because it separates addressable engineering tasks from deeper design challenges that require more fundamental rethinking. The following implications offer a path forward for tackling both.

For the CipherPal prototype, the most urgent priority is radically simplifying the digital user experience. Our user testing revealed that the tiresome process of connecting the device, synchronizing data, and confirming transactions created significant frustration that undermined the device’s core appeal. Therefore, development should focus on streamlining or automating these processes and abstracting complex blockchain transaction confirmations from the primary user flow where possible. This abstraction must also include the management of transaction ("gas") fees.

As this was a proof-of-concept study, a testnet environment was used where users had to manage testnet currency, a process that proved to be a significant barrier. This requirement is untenable for non-expert users for any real-world application operating on a mainnet. Future iterations must therefore explore solutions such as meta-transactions or service-sponsored transactions to abstract this financial complexity from the user experience completely.

This simplification must be coupled with a redesign of the user feedback mechanisms, particularly by addressing the disruptive brightness of the LEDs and providing clearer status indicators during data transfer. Improving the onboarding experience with more precise, contextual explanations is also essential for lowering the initial barrier to entry.

Our multi-framework evaluation highlighted a critical principle for health-related wearables: social perception and the need for discretion can be primary drivers of adoption, potentially outweighing novel technological features. This insight emerged clearly from the convergence of two of our frameworks. The User Acceptance Toolkit proactively identified that the device could be stigmatizing in public settings like work or school. This predictive concern was then strongly validated during user testing, where participants explicitly noted that the device’s bright LEDs and form factor made them feel conspicuous and uncomfortable in public. Therefore, a key implication is that the physical form must be refined for ergonomics and discretion. The user suggestions to explore alternative form factors, such as keychains or pebbles, reflect this fundamental need. This demonstrates that for such devices to succeed, designers must holistically integrate physical, digital, and social factors from the outset, treating potential social stigma as a core design challenge.

Finally, users must perceive a clear benefit from the technology to tolerate any remaining complexity. Our findings show that the potential of blockchain was overshadowed by its friction, partly because its value was not communicated. Therefore, a key implication for CipherPal is the need to enrich the data visualization within the companion app, offering more granular and insightful views that provide tangible value back to the user. More broadly, this points to the critical need to ensure users understand the specific benefits of blockchain integration in a way that outweighs any added cognitive load or effort.

Beyond the CipherPal case study, the principles and challenges identified are relevant to a wider range of blockchain-based health applications. As suggested by experts during our evaluation, the value of verifiable, user-controlled data extends to numerous potential use cases. These include systems for verifiable prescription adherence, where a pharmacy could confirm a diagnosis before dispensing medication, secure remote patient monitoring systems that provide auditable data to doctors, and other applications where verifying the authenticity and source of health data is critical in an era of artificial intelligence.

Ultimately, these specific and general implications demonstrate that developing successful blockchain health solutions is a multifaceted challenge. The necessity of employing iterative, multi-perspective evaluation combining expert reviews with user acceptance modeling and real-world testing is paramount for comprehensively identifying and addressing these intermixed issues and for creating products that are not only technologically innovative but also genuinely usable and valuable.

5.5. Limitations

When interpreting the findings, it is important to consider certain aspects of this study's design and context. Firstly, the sample sizes in each evaluation phase (3 experts, 6 toolkit professionals, 5 end-users) were relatively small, limiting the generalizability of the findings. While N=5 for user testing aligns with formative usability principles [46], it is insufficient for definitive conclusions about long-term adoption or diverse user population responses, especially considering the participants were relatively young (24-36) and likely tech-savvy.

Secondly, while providing valuable initial insights, the three-day user testing duration may not be sufficient to thoroughly understand long-term usage patterns, habit formation, or the potential waning of novelty effects. Evolving trust and understanding of blockchain features likely require longer observation.

Thirdly, participants interacted with a functional prototype possessing known bugs and limitations (e.g., connection issues, reliance on testnet, specific hardware choices), meaning the prototype's fidelity inevitably influenced the user experience and may not reflect a market-ready product.

Additionally, the researcher's active involvement in onboarding and providing support, necessitated by the prototype stage, could have influenced participant behavior or feedback. The specificity of the device itself, being a unique combination of fidgeting and mood data collection, means findings related to its specific form factor, interaction style, and perceived value might not directly apply to other types of blockchain-based health applications.

This study also evaluates the CipherPal prototype as a standalone system and does not include a direct comparative analysis against traditional, centralized healthcare data management solutions. Such a comparison would be a valuable future study to quantify the specific trade-offs between decentralized and centralized models in this context.

Furthermore, the study primarily focused on usability, user experience, and acceptance, not technical performance benchmarking. Therefore, we did not measure quantitative performance metrics such as transaction latency, throughput, or device energy consumption. Concurrently, technical logging limitations prevented direct tracking of user interactions within the browser wallet extension, leaving gaps in the detailed analysis of the transaction confirmation process. A complete technical evaluation, including performance analysis, is critical for assessing scalability and remains a key direction for future work.

Finally, while our study followed strict ethical protocols approved by an institutional review board, a future market-ready version would require a formal audit for regulatory compliance, including adherence to medical device software standards and data protection regulations.

5.6. Future Research

Several avenues for future research emerge based on this study's findings and limitations. Future work should prioritize iterative refinement, developing an improved CipherPal prototype that addresses the key usability issues identified. This refined prototype should then be evaluated through longitudinal studies conducted over weeks or months to assess sustained engagement, habit formation, and evolving perceptions of blockchain value beyond the novelty effect.

Concurrently, research should focus on optimizing the blockchain interaction by experimenting with different technical approaches, such as exploring mobile application potential, introducing different data uploading mechanisms, and improving transaction interaction.

Developing and testing different UX/UI strategies for effectively communicating the specific benefits of blockchain clearly and compellingly represents another important direction.

Furthermore, exploring possibilities for integration and interoperability with other health platforms and improving granular data access controls could further realize the potential of patient-centric data management.

There is also scope for refining the multi-framework evaluation approach, investigating how best to structure and synthesize data from combined methods for blockchain-based systems.

By addressing these areas, future research can build upon this work to advance the development of blockchain-enabled health devices that are technically innovative, genuinely usable, valuable, and adoptable by end-users.

6. Conclusion

This study addressed the challenge of integrating blockchain technology into healthcare devices to improve data control, moving beyond siloed design or user acceptance evaluations. While promising, this integration faces significant usability, user understanding, and adoption challenges. By employing a multi-framework validation approach and synthesizing insights from an expert evaluation using Web3 Design Guidelines, a User Acceptance Toolkit assessment, and extended user testing, we comprehensively examined the CipherPal smart fidget toy, a proof-of-concept prototype. We aimed to understand the complex interplay between technical implementation, design principles, and real-world user experience, addressing specific research questions about how different evaluation frameworks interact and can be combined for better blockchain integration.

The findings demonstrate the effectiveness of this multi-framework approach in revealing a critical tension at the heart of blockchain healthcare adoption: a strong conceptual acceptance of blockchain's benefits is insufficient to overcome the practical usability friction introduced by a flawed implementation. Our method revealed precisely how this tension manifests. The User Acceptance Toolkit assessment predicted that Hedonic Motivation would be a key driver for user engagement. This was strongly validated in user testing, where the device’s core appeal was its enjoyable physical interaction. Simultaneously, the Web3 Design Guidelines evaluation identified technical weaknesses in the transaction and onboarding flows. The user testing then confirmed the real-world impact of these gaps, revealing that the high Effort Expectancy of connecting the device and confirming transactions created significant frustration that overshadowed the technology's theoretical benefits.

The primary contribution of this research is, therefore, methodological. Through this triangulation connecting predictive models of acceptance with real-world user frustration and expert technical assessment, we demonstrate the necessity of a holistic, user-centered approach for developing viable blockchain-enabled health devices. Our findings empirically highlight the significant gap that can exist between blockchain's theoretical benefits and the practical realities of user interaction. This work strongly implies that blockchain complexity must be abstracted away from the end-user whenever possible for successful adoption. The value proposition of using blockchain needs clear communication, and the design process must meticulously integrate physical, digital, and social factors from the outset.

While acknowledging the study's limitations regarding sample size, prototype fidelity, and testing duration, the insights gained pave the way for future research focused on iterative refinement, longitudinal evaluation, and optimizing the blockchain user experience. Ultimately, realizing the potential of blockchain in healthcare hinges not just on technological innovation but on designing systems that seamlessly integrate into users' lives, providing tangible value without undue burden. Achieving truly user-centric blockchain health solutions requires a persistent commitment to balancing cutting-edge technology with fundamental usability principles and thoughtful design.