Visual Biofeedback Tool for Quantitative Pressure Monitoring in Lower-Limb Prosthetic Sockets

: Obtaining a good socket fit is an iterative process dependent on the skill and experience of the prosthetist creating it, and requiring individualisation based on the size and shape. There is no standard measurement system used to aid prosthetic socket creation, despite the severe impacts on physical health and quality of life if one is ill-fitting. Pressure sensors embedded in a prosthetic socket were used to collect data at the socket-residuum interface. To choose an interpolation method, a 2D grid was used, with previously collected walking test pressure data, to simplify the sensor array with a border for extrapolation. Four multivariable interpolation methods were evaluated to create a colour map of the pressure data. Radial Basis Function interpolation was chosen as it produced a clear image with a graduated interpolation between data points and was used to create a colour map across the surface of a 3D prosthetic socket model. For the model to be accessible to clinical audiences, a desktop application was created using PyQt to view the model. The created application allowed for connection to the sensors via Bluetooth, with the pressure data updating the colour map on the 3D model in real-time. The created application shows the potential for a clinical product, however further development informed by feedback from rehabilitation clinicians and prosthesis users is required


Introduction
The prosthetic socket is the point of load transfer between the body and the prosthesis and is a complex interface [1,2]. Obtaining a good socket fit is an iterative process dependent on the skill and experience of the prosthetist creating it, requiring individual adaptation based on the size and shape of each user's residual limb [3,4]. The socket is not an ideal environment for the residual limb due to the load distribution over anatomy and tissues that have not evolved for this purpose. The bony prominences and skin on the residual limb are subjected to a hot, moist environment with greater loading conditions compared to pre-amputation [5]. Therefore, the residual limb is highly susceptible to complications -including skin tissue breakdown -that can lead to deep tissue injury and the formation of pressure ulcers [6][7][8][9]. A critical factor in the development of pressure sores is the presence of shear in the soft tissues, often present due to movement of the residual limb.10 Pressure sores negatively affect the quality of life of 62% of prosthesis users, frequently prohibiting prosthetic use [6,8,11,12].
There is no standard measurement system used to aid prosthetic socket creation, despite the severe impacts on physical health and quality of life if one is ill-fitting. The knowledge to solve patient-reported problems or adapt the socket for different needs is gained only with years of experience [3,4]. The process of fitting a socket is made more difficult for those who have lost sensation in their residual limbs due to nerve damage, burns or skin grafting. These individuals cannot provide feedback to prosthetists if the socket is uncomfortable due to ill-fit or high pressure.
Published literature shows that effective biofeedback methods have been used with people with lower-limb amputations alongside conventional techniques for monitoring and assessing loading, motion symmetry and improving gait [13,14]. Biofeedback in these settings is most commonly presented visually, with auditory and haptic methods (delivering feedback via a method of touch e.g. using sub-threshold electrical stimulation on the skin to signal change in environment) also popular techniques [13]. Although biofeedback is effective in both clinical and non-clinical settings, it is particularly beneficial during the early stages of rehabilitation [13]. Using biofeedback technologies is generally received well by trial participants and increases motivation to complete sustained rehabilitation [15,16], though there have been usability issues reported within the elderly population [13].
Methods to measure the environment inside the prosthetic socket have been developed and tested for several decades. Due to their associated medical implications, pressure and shear are of particular interest [17][18][19][20][21]. Few commercial technologies exist and they have limitations similar to devices used in research studies, generally including high costs [22][23][24] Research studies either use commercial technologies [17,18] or focus on the development of new sensors [19][20][21]. It is more common for commercial products to also focus on the user interface for display and interpretation of the collected data. However, the user interfaces only work with their associated sensors, can be platform and device specific, and often require an internet connection. The research technologies frequently require specialist technical knowledge and licensed software (e.g. MatLab) [19][20][21], making them less suitable for a clinical environment.
The research team has created an embedded network of up to 144 pressure sensors, arranged in 12 strips of 12 sensors, connected to a control board with an inertial measurement unit (tri-axis accelerometer, gyroscope and magnetometer) to detect and measure movement [25]. The hardware transmits the data via Bluetooth to a computer or mobile device for analysis and post-processing. The data is saved to a file, in comma separated variable (CSV) format, at a sampling frequency of 200Hz.
The aim of this study was to create a proof-of-concept real-time biofeedback tool to display the pressure measurements from the sensors created by Hopkins et al. for clinicians [25]. The quantitative information is intended to guide the prosthetists with their fitting by providing additional information that previously they would not have had, to support them as they fit sockets.

Results
Multiple multivariate interpolation methods for unstructured data were run on the 2D model of the socket [ Figure 1]. Radial Basis Function displayed the smoothest graduations between data points and was chosen as the optimal interpolation method. Nearest Neighbour (NN) produced a pixelated image that would not be clinically useful and was excluded from further testing. Radial Basis Function (RBF), Clough Toucher (CT) and Linear (N Dimension) (LND) all produced images with smooth gradations between data points, however Clough Tocher and Linear (N Dimension) were unable to extrapolate past the boundaries of the sensor data. Extrapolation is necessary to allow clinicians to visualise the pressure distribution across the whole socket; not only the area enclosed by sensors as they will not necessarily be located at the socket boundaries. The pressure values are represented using a smooth colour gradient from the lowest in green, through blue, and the highest in red. The colour gradient can be changed if desired. Whilst the pressure distribution changes at the different time points, the methods of each interpolation did not. All methods created images with similar characteristics, showing the timepoint in the gait cycle did not impact the appropriateness of the interpolation methods.
A static image of the 3D model with a colour map using RBF interpolation was created [ Figure 2] to show an estimated pressure distribution across the entire prosthetic socket. The desktop application was created with a simple user-flow [ Figure 3] to demonstrate the possibility of using 3D visualisation in real-time as a visual biofeedback tool for clinicians. A Graphical User Interface was created with two screens and two buttons to navigate between the screens. Navigating to the second screen initiates the connection to the Bluetooth sensors via a dialog box with an "OK" action button, which triggers the host device to scan for the sensors. The main window is disabled whilst the application is scanning for the sensors, and another dialogue box is shown to confirm that the sensors are connected to the host device. Once sensors are connected and the test session started using a "Start/Stop" button in the main window, a stream of data is received by the application and the pressure values are extracted to feed the function that creates the pressure map on the 3D model. The window can be resized, and the model can be reorientated by clicking and dragging. Clicking the 'Start/Stop" button breaks the Bluetooth connection with the sensors.

Discussion
A proof-of-concept real-time biofeedback tool to display the pressure measurements from the sensors created by Hopkins et al. [25] for clinicians was created. An RBF interpolation method was used to create a colour map of pressure across the surface of a 3D prosthetic socket model using measured values from 144 sensors embedded in a real socket.
For the sensor hardware to be useful in a clinical environment, the information relevant to clinicians and people with amputation(s) must be displayed in an easy-to-understand manner, requiring little interpretation or training and no additional technical knowledge. Clinicians are busy and have a short timeframe in which to see and treat patients; the technology must not be a burden for them to use to ensure adoption into clinical practice.
Data visualisation was kept in the simplest form, relaying pressure information: no analysis or post-processing was done on the data. Further analysis adds complexity and time, which is not always available or possible in the clinical environment when treating patients. After initial consultation with the target users regarding the clinical application, further functionality for data analysis, such as reporting numerical metrics and comparisons can be implemented.
Choices for programming language were limited due to the complexity of the mathematics required in a short space of time to create a colour map that updates in real-time. Python is a versatile language, particularly useful for complex mathematics, however, desktop applications are not typically written in Python due to the way the code is distributed and run. It is possible to re-write the application and model in C++ using the VTK and PyQT frameworks, which may yield an application which is more efficient to run, particularly with added functionality in later versions. For the proof-of-concept, however, Python is sufficient to demonstrate the possibility of creating such a biofeedback tool.
The interpolation methods evaluated were purely the built-in methods to the functions used and chosen purely based on subjective observation. From the four interpolation methods assessed, only one was suitable as it provided smooth colour gradients between values and extrapolated data. It would be prudent to evaluate further methods of interpolation which may be less computationally expensive.
Commercial technologies are available that provide live visual biofeedback of pressure within the socket. Novel.de has created Pliance sensors that can measure the pressure and load distribution of any object on a 2D grid, similar to the initial testing of interpolation methods in the current study [22] The sensors, however, do not cover the entire surface of the socket, only a target area. The sensor network must be wired to a Windows computer via a control unit for data transfer to enable visualisation.
F-Socket, created by Tekscan [24], is well used in research studies [26][27][28][29][30][31][32]. The sensors are connected via USB to a computer with a range of 100m [24]. The associated software provides information about the peak pressures in the socket, centre of force and forcetime graphs [24]. The software displays the data in both 2D and 3D data and allows for side-by-side comparisons of different test sessions, either giving information about the socket as a whole or focussing on regions of interest [24]. Both graphical data and movie files can be imported and exported into the software [24]. The AdaptTech Insight system [23] is the most similar to the tool created in this paper, however, it only shows a circular interpolation around each sensor rather than a colour map across the entire socket. Generally only 4 strips of 8 sensors are used for testing [33], therefore little of the socket area is covered. The AdaptTech software is only available for iOS devices (typically iPad), which receives the data and displays it in real time as well as saving it for future access (acting as a patient record system) and requires wireless internet connection for use.
The existing commercial technologies are associated with high costs and have limitations for their usage, e.g. requiring internet connection or a device with a specific operating system. The application created in this study can be used on any laptop or desktop computer with a Bluetooth connection. The platform-agnostic and internet-independent nature of the application allows this tool to be accessible to a broader audience, especially in low-cost and remote settings as additional costs to meet the application requirements are not incurred. If Bluetooth connectivity is not built in, a dongle can be used instead.
In addition, an advantage of this new tool is that it can be used with different pressure sensor technologies, with only minor changes to the code. The proof-of-concept tool has been created to complement the sensors created by Hopkins et al. [25], however, it can be adapted to allow easy connection to different hardware. The Python script required to receive Bluetooth data from the hardware needs to be individual for different sensors. Whilst data communication is not standard due to the differences in the way developers choose to structure the data transmitted, the Bluetooth protocols remain the same. To facilitate easy change, only one script in the code needs to be changed.
A proof-of-concept application has been created which demonstrates the technological feasibility of creating an application to display real-time pressure information across the surface of a prosthetic socket. However, the tool must be shared with prosthetists, physiotherapists and prosthesis users to understand the clinical utility of the tool. Such consultation is vital before progressing with development of the application, to ensure it meets the needs of the target users and provides maximum value.
Before the application can be used to receive personal data via Bluetooth, it is essential to build in the required data security characteristics of the application to protect the data. It would also be beneficial to add in functionality to allow users to select the sensors they would like to connect to after scanning the surrounding area. Both the functionality to choose the type of sensors being used (e.g. brand) and also, specifically which sensors to connect to (instead of defining one address in the code).

Materials and Methods
Given the limitations of existing products, it was decided the biofeedback tool must function cross-platform. The software needed to be compatible with different pressure sensor technologies, with only minor technical adjustments. The proof-of-concept tool has been created to complement the sensors created by Hopkins et al. [25] The tool has been created to be easy to use, requiring no additional training, technical knowledge, or use of the internet. The pressure data from the sensors should be updated in real-time as the application receives them to allow for the data to be used immediately for clinical decision making. Immediate use of the data means the tool can be used in consultations, giving instant feedback to inform clinical decision making.
Previously collected pressure data of a walking test in a CSV file was used to evaluate different interpolation methods. Due to the application being created in a sandbox environment (isolated from any external network connection e.g. Bluetooth, internet) when using personal data, there was no need to focus on data security aspects for the proof-ofconcept application.
To choose an appropriate interpolation method, a 2D 156 x 156 grid was used to include the 12 x 12 sensor array (evenly distributed) and a border for extrapolation. The grid was created using built-in functions from the numpy [34] library in Python and visualised using matplotlib [35]. Sensor data collected through participant testing and saved into CSV files were imported into Python for use. The grid was populated with data from five evenly spaced time points spanning the length of the participant's gait cycle, each 250 milliseconds apart. The time points were chosen to determine whether the interpolation methods were appropriate for different pressure distributions across the gait cycle.
Four multivariable interpolation methods were evaluated, each compatible with unstructured data and therefore appropriate for both 2D and 3D data: Clough Tocher (CT),36 Radial Basis Function (RBF) [37], Nearest Neighbour [38] (NN) and Linear N Dimension (LND) [39] The most appropriate interpolation method was identified based on observation of smooth gradations between pressure values and the ability to extrapolate, as well as interpolate, data. The chosen interpolation method from the 2D testing was taken forward for testing on a 3D socket model.
A computational model of a prosthetic socket was obtained by laser scanning a casting mould of a transtibial prosthetic socket. The model was cleaned up, including fixing of any holes and artifacts produced during the scanning process, using Geomagic Wrap [40], 3-Matic was used to add thickness to the model, simulating that of a socket. The 144 vertices representing the sensors were picked out manually using Paraview (Various Locations, USA). For the proof-of-concept, the real-life sensor locations were approximated on the model: evenly distributed in twelve lines of twelve sensors to represent the sensor strips.
The 3D model was imported into Python and the sensor locations and values assigned to the relevant vertices. The Visualisation Toolkit library was used to create a colour map with the chosen Radial Basis Function interpolation from the 2D testing between the vertices assigned sensor values.
For the model to be accessible to a clinical audience, it was decided that a desktop application was required for them to view the model. The creation of the application would also allow connection to the Bluetooth sensors, with the data being used to update the colour map on the 3D model as data is received in real-time.
PyQt was chosen to create the desktop application's Graphical User Interface (GUI) and additional functionality as it allows for the application to run on all major operating systems. For the application to work, the host device requires a Bluetooth connection. PyQt's Bluetooth module was utilised to implement the relevant Bluetooth functionality. The sensors' Bluetooth address was specified in the code for simplicity, so the host device looks for and connects to only the specified sensors in this proof-of-concept application.

Conclusions
Visual biofeedback has been shown to have a positive effect on rehabilitation when it complements traditional methods. The proof-of-concept tool created provides live information about the pressure distribution inside a prosthetic socket when linked to an appropriate sensor network. A desktop application was created that can run on any of the standard operating systems; has no requirement for internet connection; can be connected to any sensor network with minor changes to the code; and has been designed to be easy to use without the requirement for technical knowledge or training.
Further improvements and testing should be completed; however, the current application shows the potential of a clinical product. End-user feedback is vital for the development of both the design and functionality of this application. Feedback from rehabilitation clinicians in the clinical environment, the end-users in this case, should be used to guide further improvements.