Towards Non-destructive Quality Testing of Complex Biomedical Devices – a Generalized Closed Loop System Approach Utilizing Real-Time in Line Process Analytical Technology

1. Introduction

Cardiovascular diseases (CVD) are the number one cause for deaths globally affecting not just high-income but also the middle and low-income countries [1], [2], [3]. According to some statistics published by World Health Organization (WHO), an estimated 17.9 million people deaths were caused by CVDs in 2019, representing 32% of global deaths and of these, 85% were caused due to either ischaemic heart disease or stroke. Clinically, a heart attack or myocardial infarction, is a term used to denote the condition when there is evidence of myocardial necrosis in a clinical setting consistent with myocardial ischaemia [4]. Put simply, it is a serious medical condition in which the supply of blood to the heart is suddenly obstructed, usually caused by a blood clot. This blood clot, referred clinically as Atherosclerosis, is essentially a plaque comprising of fat, cholesterol, fibrin and calcium that develops on the inner walls of the arteries [5]. Atherosclerosis is deemed to be the principal cause of myocardial infarction [6].

Acute myocardial infarction is divided into ST-segment elevation myocardial infarction (STEMI) and Non ST-segment elevation myocardial infarction (NSTEMI) [7]. Depending on the type of myocardial infarction, common treatment options include Thrombolysis for NSTEMI, which is basically dissolving the blood clots and restoring blood flow through a combination of medication. For STEMI, which is the most serious type of heart attack, recommended treatment option is Percutaneous Coronary intervention (PCI), commonly known as Coronary Angioplasty with stenting [8]. This procedure incorporates a combinational medical device such as a radial metal stent deployed using an inflating balloon via minimal invasive procedures. PCI with bare metal stents has its own set of complications and risks. The biggest post intervention risks associated with PCI using metal stents are risk of stent thrombosis (blood clotting) and restenosis (narrowing of arteries) [9]. Stent technology since then has evolved rapidly and currently, the most prominent stents used are Drug Eluting Stent (DES). DES have considerably reduced the post procedure complications related stent thrombosis and restenosis [10], [11].

With the reduction in post intervention complications for the procedure, demand for immediate intervention for these conditions has led to surge in manufacturing volumes for these combinational devices. As of 2016, the global market for vascular stents was estimated at $7.22 billion, with coronary artery stents accounting for 67.3% of the vascular stent market [12]. The global stent market is expected to grow at a Compounded Annual Growth Rate (CAGR) of 6.80% by 2028 [13]. Currently these life saving devices are pre-dominantly manufactured either using a semi-automated production process or a manual-assembly process in conjunction with classical quality control approaches [14]. This poses a significant challenge for the manufacturers of these life saving devices. The current semi-automated manufacturing process and classical approach to quality with acceptance sampling and batch release can no longer support the increasing demands for production. This underscores the urgent need for manufacturers to look for alternatives to classical quality control approaches towards more advanced Industry 4.0 based solutions.

In a guidance document released by US Food and Drug Administration (FDA) in 2004, the regulators have encouraged the industry to develop and implement innovative Process Analytical Technologies and Tools (PAT) for improving pharmaceutical development, manufacturing, and quality assurance through innovation in product and process development, process analysis, and process control [15]. FDA suggests the manufacturers to have extensive understanding of their processes, critical product and process parameters along with the ability to control processes through quality systems and strive for continuous improvement [16]. It continues to emphasize the need for industries to move away from classical batch release and control strategies towards real-time release testing (RTRT) through utilization of PAT. This paper aims to highlight the financial and quality implications of using current destructive end-of-batch sampling in stent delivery catheter manufacturing. Additionally, it will present a pathway forward for the development and adoption of a non-destructive (NDT) PAT for this use case and a generalized framework that would be readily deployable within the regulated medical device manufacturing industry in general.

2. End of Batch Sampling

The current manufacturing process for these catheters relies on batch sampling technique based on acceptance criteria derived from the product design specifications for accepting or rejecting manufactured lots using destructive tensile testing as shown in Figure 1. Acceptance sampling in pharmaceutical/medical device industry is based on ANSI/ASQ Z1.4 tables for inspection by attributes and ANSI/ASQ Z1.9 for inspection by variables.

Current lot size for these catheters is 120, therefore, 3 random parts in every lot are sampled and inspected as per the Table 1. Dodge & Romig [17], first to introduce operating characteristics (OC) curve, defines them as a representation of the performance of a sampling plan to that of various defect levels. In this case as shown in Figure 2, the lot proportion defective p accepted by the sampling plan (probability of acceptance $P_a$) is given by the formula $P_a=1-p$. The OC curve for this sampling plan for acceptance number a=0, meaning no rejects in the 3 random samples inspected for a lot size of 120, provides an 88% probability of acceptance with Lot proportion defective of 4%.

With the current manufacturing volume going up and anticipated to grow over the 5 years minimum, continuing to do acceptance sampling will become more time consuming, more costly and more labour intensive. For this particular use case, the manufacturer intends an annual planned production volume of 4 million catheters. With the lot size of 120, that is approximately 33.3 thousand batches. Current plan for acceptance sampling needs three samples per test per lot for three different destructive tensile tests which brings the annual volume of testing samples to almost 300 thousand. Average selling price of a DES catheter is approx. $200 and manufacturing cost approx. $55 per catheter. This brings the cost of sampling to almost $60 million in lost revenue and $16.5 million in testing costs, excluding cost of labour, missed opportunities for higher capacity and cost of additional samples in case the initial sampling fails, and more samples needs to be pulled for releasing the batch.

Acceptance Sampling by Attributes

Measurement type: Go/no go

Lot quality in proportion defective

Lot size: 120

The utilization of acceptance sampling methods also entails certain fallacies that warrant consideration. As highlighted by Wheelers and Chambers [18], these limitations pertain to the acceptance sampling approach. The approach inherently operates under the assumption that lot quality exhibits significant variability across different batches while remaining homogeneous within a given batch. Decisions regarding the acceptance or rejection of each batch hinge on the outcomes of both measured and unmeasured samples within that batch, under the presumption of homogeneity. In cases where product quality displays substantial variation not only between batches but also within individual batches, the foundational assumptions of acceptance sampling are no longer valid. This introduces a significant level of uncertainty concerning the quality of batches delivered to customers, which is deemed unacceptable, particularly in the context of Class III medical devices like stent delivery catheters. This itself presents a strong proposition for the manufacturer to look for smarter solutions that are cost effective than the classical statistics-based approach and that allows for growth and higher capacity expansion without adding additional testing burden.

3. Materials and Methods

3.1. Research Methodology

This research work is an extension of the work carried out by the authors in relation to develop a closed loop cyber physical production system that intends to answer three key focus areas for catheter manufacturers, and by extension, the highly regulated medical device industry. The three research objectives for the overall work being carried out are: quick product changeovers, real-time quality and self-adaptive closed-loop control system [19], [20]. This paper in particular focuses on the real-time quality control objective by proposing a generalized version of a closed-loop system that focuses on moving away from destructive testing of catheters at the end of batch towards a non-destructive in line real-time inspection system.

The overall study employs a mixed-method approach to the experimental design, combining qualitative and quantitative research paradigms in an exploratory sequential manner as shown in Figure 3. This approach allows the researcher to gain a comprehensive understanding of the research problem by utilizing both qualitative and quantitative methods in different phases of the study ([21] via [22]). The whole process design lifecycle is subjected to Action Design Research (ADR) science for artefact generation through continuous improvement loops as proposed initially by Sein et al. [23] and further improved by Mullarkey and Hevner [24].

In Phase I of the research, a qualitative approach is employed to address two main objectives. Firstly, an extensive review of academic literature is conducted to gain a comprehensive understanding of manufacturing systems, including their types, advantages, and disadvantages. This thorough exploration allows the researcher to form a well-informed opinion on the most suitable manufacturing system for the specific research case. Additionally, recognizing the importance of real-world applications, the researcher investigates successful solutions implemented in other industries, international standards, and innovative approaches developed by organizations, R&D departments, and research laboratories. By drawing inspiration from these diverse sources, the aim is to adapt and apply effective solutions to the unique context of medical device manufacturing. Throughout Phase I, a conceptual framework for a closed-loop Cyber Physical Production System (CPPS) is developed, integrating insights from academic literature and existing solutions/standards. This integrated framework serves as the foundational basis for further development and implementation of the closed-loop CPPS.

In Phase II, a quantitative approach is implemented to guide the creation, data generation, collection, analysis, and testing of the CPPS artefact. Building upon the conceptual CPPS model, the focus shifts towards designing a practical machine or artefact that directly addresses the research objectives and primary research question as shown in Figure 4. This involves translating the conceptual model into a tangible prototype that can be tested and refined. Throughout this phase, the artefact prototype undergoes a series of continuous improvement cycles, following the principles of the ADR science. By iteratively refining the artefact based on feedback and insights gained from each cycle, the aim is to enhance its functionality, efficiency, and overall performance.

3.2. Artefact Development

In this research, Solidworks served as the primary tool for hardware design and development, providing a comprehensive platform for creating detailed three-dimensional models and prototypes. Leveraging its robust features and intuitive interface, the researcher and the extended mechanical design team utilized Solidworks to design and refine the physical components of the artefact, ensuring precise dimensions and optimal functionality. Subsequently, Microsoft VB.NET played a pivotal role in the software and system design architecture development process, facilitating the creation of a cohesive and efficient software framework to complement the hardware components.

Figure 4. Tools used for artefact development and testing.

Figure 4. Tools used for artefact development and testing.

Following this, the Cognex Insight software was employed for vision system development, enabling the implementation of visual inspection capabilities. Lastly, Minitab served as a crucial tool for validation testing, enabling rigorous statistical analyses to validate the accuracy and precision of the developed artefact's capabilities. Through the integration of these software tools, the researcher effectively designed, developed, and validated the prototype artefact, thereby advancing the objectives of this research.

The conceptual CPPS framework, hardware development, software design, artefact build, and validation testing are covered in detail in author’s previous work in [19] and [20] and as shown in Figure 5 and 6. This research will proceed onto the exploring the impact of development of this NDT on the overall production process of these catheters and if the research objective has been met or not. It is also to foresee, if the custom developed conceptual framework for catheter manufacturing can be generalized for the overall medtech industry.

4. Results

4.1. System Performance

The closed-loop system along with the inline vision system is implemented successfully as part of the experimental test rig (see Figure 7).

The system is also validated successfully as per the stringent regulation of the regulatory bodies as documented in [20]. The performance of the vision system is found to be statistically equivalent to the manual inspection carried out by a trained operator (see Figure 8).

4.2. Effectiveness of Closed-Loop Control

Figure 8 illustrates the influence of the closed-loop control framework along with the PAT system on the production process. It shows the pareto analysis based on the count of missed units of the production process. The analysis clearly demonstrates the significant impact of the models, as the number of missed units in the process has reduced by more than 50% in terms of volume following the activation of the system. This notable reduction underscores the effectiveness of the decision models in improving the overall performance and efficiency of the production system.

A significant impact of the framework is also seen in the cycle time of the machine and the number of moves the machine makes within the production cycle of each unit. Since the prototype is connected to a live database that captures vision system’s run data in real time, a trending software trends these parameters in real time for the shop floor engineering staff.

Figure 9 shows a snapshot of the machine’s average cycle time pre the models are activated for machine bonding workstep. The cycle time isn’t consistent over time and the average cycle time for the machine is approximately around 25.3 seconds.

Figure 10 illustrates a time segment displaying the average cycle time of the machine after the activation of the models for the machine bonding workstep. Evidently, the average cycle time exhibits a significantly higher level of stability during the specific time. Furthermore, there is a noteworthy reduction of at least 2 seconds in the overall cycle time of the machine. Upon activating the models, the machine achieves an average cycle time of approximately 23.5 seconds per unit.

Similarly, the number of moves performed by the part holding collet to align the part is examined. The target number of moves for the system is set at three moves. However, Figure 11 demonstrates that the system lacks stability, frequently requiring additional moves before initiating the bonding process. This instability contributes to the overall cycle time. Upon model activation with the vision system, as depicted in Figure 12, the number of moves becomes considerably more stable.

Figure 10. Machine average cycle time post model activation.

Figure 10. Machine average cycle time post model activation.

Figure 11. Move count pre model activation.

Figure 11. Move count pre model activation.

Figure 12. Move count post model activation.

Figure 12. Move count post model activation.

4.3. Tangible and Intangible Benefits

Section 2 highlighted the current cost burden due to end of batch Acceptable Quality Level (AQL) sampling for the current catheter manufacturing and batch release process. The cost for scaling the current prototype to a high-volume manufacturing line is roughly approximately $4.5-5 million. There is a direct cost saving of approximately $11.5 million per year once the scaled-up solution with real time inspection fully replaces the end of batch AQL acceptance sampling. There are associated intangible benefits with the solution as well in terms of saved intervention efforts for filling up of paper records for AQL sampling, eliminating the instances of non-compliances due to tester error and a major boost to the product quality. The implementation of two data-based feedback control loops has significantly enhanced resource optimization and contributed to a notable reduction in scrap within the manufacturing process. By incorporating closed-loop control systems, the operational efficiency of the production line has been substantially improved across various key performance indicators. The count of product misses for machine related issues have significantly reduced from 1890 units to 868 units, representing a substantial 54% decrease within the similar timeframe (see Figure 8). The instances of product miss resulting from quality investigations have plummeted from 1357 to 125 units, demonstrating an impressive 90% reduction over a similar period. These improvements underscore the effectiveness of closed loop system in enhancing operational efficiency, minimizing waste, and optimizing resource utilization within the regulated manufacturing environment of these catheters.

5. Discussion

The successful implementation of the conceptual closed loop CPPS underscores the transformative impact of leveraging advanced control strategies and standardized methodologies in modern manufacturing environments. However, while these solutions have demonstrated significant efficacy in the context of specific manufacturing scenarios, there remains a compelling need to generalize the CPPS framework to accommodate diverse manufacturing settings and regulatory requirements. By developing a generalized CPPS framework, tailored to address the unique needs and challenges of regulated manufacturing environments, organizations can unlock greater scalability, adaptability, and interoperability across their manufacturing operations. This holistic approach not only ensures continued process optimization and efficiency gains but also fosters innovation and resilience in the face of evolving market dynamics and regulatory landscapes.

The CPPS model, as initially conceived in [19], is tailored explicitly to the case study concerning catheter manufacturing. Data-driven CPPS models, which are predominantly referenced in academic literature, tend to share a similar characteristic in that they are custom-designed to address specific use cases. The availability of generalized CPPS models applicable to diverse manufacturing environments remains limited in evidence. Suvarna et al. [25] did introduce a CPPS framework model, designed for broader applicability within manufacturing environment. This framework primarily emphasizes data collection from diverse sources, consolidation within a data repository, and utilization of this data for Machine Learning (ML) model training. Subsequently, these trained models are redeployed within the production environment to enact essential control measures. While this approach holds practical merit for deployment in numerous standard manufacturing scenarios, it doesn’t necessarily aligns with the stringent requirements of regulated manufacturing, particularly in fields such as medical devices. The reason for this disparity lies in the fact that regulatory authorities are still in the process of adapting to the integration of standalone AI/ML models within their regulatory frameworks. Complete acceptance of these technologies, along with the establishment of comprehensive guidelines for their use and deployment, is yet to be achieved. Notably, the model proposed by Suvarna et al. [25] does not delineate how it ensures the maintenance of product quality or the handling of erroneous situations, should they arise in contexts for which the model was not specifically trained.

To address this, this research attempts to close the gap by proposing a Guha model that is generalized enough to be deployable across regulated manufacturing environment, as shown in Figure 13.

Figure 13. Proposed Guha model for CPPS deployment within regulated manufacturing.

Figure 13. Proposed Guha model for CPPS deployment within regulated manufacturing.

A model of this nature possesses justifiable attributes for acceptance and deployment by regulatory bodies. This justification stems from its capacity to guarantee that, under no circumstances, can a manufacturing scenario transpire wherein the product is manufactured beyond the normal processing window. Essentially, this model serves as a mechanism to uphold the system's operation within predefined optimal processing conditions and limits.

5. Conclusions

This research study successfully addressed the three main research objectives within the context of highly regulated complex medical device manufacturing. Through a comprehensive investigation and extensive literature review, it became evident that existing solutions were inadequate for the unique challenges faced by medical device manufacturers. To bridge this gap, a holistic theoretical framework was developed, leading to the creation of a novel data-driven closed-loop CPPS model. This model effectively integrated the research objectives, providing a cohesive and comprehensive solution to the broader research theme. The implementation of this model along with the custom solutions developed for the research objectives at hand, demonstrated significant improvements in the ability of the system to manage quick product changeovers, process control, real-time quality monitoring, and closed-loop decision-making. This resulted in reduced scrap, enhanced process cycle times, and improved overall operational efficiency. The successful achievement of the research objectives highlights the potential of innovative approaches and technologies to address the complexities and regulatory requirements of the medical device manufacturing industry, ultimately leading to improved product quality, patient safety, and business performance.