Neuromorphic Sensor Based on FSR

1. Introduction

The spiking neural networks represents the third generation of neural networks which introduces time in information processing and learning. The SNN are based on spiking neurons that model rigorously the physiology of the neural cells in the brain. The hardware implementation of the SNN benefits from low power consumption, high reliability and very good signal to noise ratio, which makes SNN a good candidate for the implementation of control units of robotic systems. In robotic systems, the motion is of high importance because it provides the robots with the ability to interact mechanically with the environment. Typically, the motion is performed by electric, pneumatic or hydraulic actuators , while the control of motion is based on the feedback from neuromorphic sensors. This type of sensors includes a sensing component and the additional electronic circuits that converts the measured parameter into trains of impulses with parameter dependent frequency [1]. There are several methods to implement the sensing components that are based on ferromagnetic powder in a resin [2] or on Euler-Bernoulli bending rods. The later method is used to implement multi-point fine sensing using the principle of whiskers in living organisms [3]. An advanced method to implement the sensing components use piezo-electric field effect transistors FET that converts the physical measure into electrical current [4]. In neuromorphic sensors, this current is used by additional analogue hardware to generate spikes with corresponding frequency [5]. Another type of tactile sensor with spiking output includes one organic transistor for sensing and another transistor for pulse generation [6].

However, these types of sensing components are expensive or has limited availability. In contrast, force sensing resistor (FSR) is cheap and easy to use making this type of component suitable for a wide range of applications starting from implementation of wearable sensors in gaming [7], to prostheses for human in biomedical engineering [8] and for monitoring cardiorespiratory parameters during sleep [9].

Force Sensing Resistors

FSR uses polymer thick film technology to convert the force into resistance which varies in the opposite direction with the applied force. The FSR is built of two layers of flexible polyester film substrate, each with a layer of conductive silver ink. Between the layers there is a compressible polymer which responds to the applied force by increasing the conductance [8]. With no pressure the sensor behaves as an open switch due to infinite resistance, and when the sensor is pressed the drop of resistance is significant for equivalent mass below 100 grams. The variation per unit mass reduces following the power law while the compression mass increases to several kilograms. With proper design of the sensing component, the repeatability of the measurements for one sensor is below ±2% showing good reliability of the FSR [10].

Compression Load Cells

A CLC converts the applied force into voltage being based on strain gauge (SG) which represents the measuring component of the load cell. The strain gauge is a small grid consisting of wires made of an alloy of copper and nickel which are sensitive to the deformation of the load cell surface. This changes the resistance of the SG and with proper electronic circuit the force is converted into voltage. The circuit is based on Wheatstone bridge which connects multiple SGs and detects of deformation as a change in electrical signal [11]. The CLC benefits from high linearity and repeatability of the response even at extremely high load capacities but comes at higher cost than the FSR.

SMA Actuators

Among the electric actuators the shape memory alloy (SMA) represents an elegant alternative to the widely used motors because of their small size, low weight and significant force-weight ratio [12]. The SMA is a metallic material which changes its state when heated from martensite phase when cold to austenite phase which represents the memorized shape. During the austenite phase which is determined by higher temperature the actuator shortens its length with a significant force. For example, an actuator made of nitinol with 0.006 inch diameter can pull up a mass of 320 grams according to the technical data. Also, the lifetime of SMA is of order of 10⁶ actuation cycles implying that these actuators have significant reliability. However there are some disadvantages of these actuators such as the nonlinearity due to the difference between the contraction and respectively relaxation temperatures. The contraction speed and the initial length are influenced by the environmental temperature, that affects also the relaxation time when no auxiliary cooling method is used. In addition, the change in length during contraction is about 4% which increases the length of necessary actuators for long displacements.

Despite these disadvantages, the SMA benefits from the same actuation principle and are silent in operation as the natural muscles being suitable for actuation of the bioinspired systems. In addition, the spiking neurons are able to control directly the contraction of SMA actuators in a biomimetic manner by the firing rate [13,14].

The Proposed Sensor

In this work we proposed a neuromorphic sensor based on FSR (NS_FSR) that includes a spiking neuron implemented in analogue hardware which integrates the FSR in the SOMA. Compared with the existing neuromorphic pressure sensors [5,13] the proposed NS_FSR benefits from simplified circuit over CLC based sensor because the sensing device is integrated in the neuron and has significantly lower cost than the FET based ones. The NS_FSR is validated in a robotic application where SNN controls the force of an anthropomorphic finger actuated by SMA. The behavior of the finger is evaluated when the FSR based sensor and the CLC are used independently.

The remainder of the paper is organized as follows: Section 2 details the schematics of the SOMA with FSR and the structure of the SNN for evaluation of the NS_FSR and CLC in robotic finger control together with the experimental setup. The results are presented in the Section 3 followed by the conclusions that focuses on the advantages of the proposed sensor and the future work.

2. Materials and Methods

The proposed sensor is based on an electronic spiking neuron implemented in analogue hardware which obtained good results for the control of SMA actuated anthropomorphic fingers [13,14].

Electronic Neuron

This neuron model includes one input module denoted SOMA for integration of the incoming pulses and activates the synapses (SYN) when the activation threshold is detected. The SYNs stores the synaptic weights and generates excitatory or inhibitory spikes which intensity depends on the weights. In addition, SYNs model the Hebbian learning mechanisms that makes the neuron to respond to temporal coincidence of incoming stimuli.

The Neuromorphic Sensor

The schematic of the SOMA is presented in Figure 1. The capacitor $C_M$ integrates the incoming spikes while the transistor $T_{TH}$ detects the neuron activation threshold $T_{TH}$ and activates the synapses using another transistor $T_{ACT}$. When $V_{TH}$ is reached the SOMA activates for a fixed period of time $t_{ACT}$ while the synapses to generate a spike. The SOMA can be activated by incoming pulses that are integrated by C_M of by a continuous potential $V_E$. In this case, the activation rate of the SOMA, and consequently of the neuron, depends on $V_E$ and $R_E$. Thus, the firing rate of the neuron $f_N$ varies with ${\mathrm{V}}_E$ when ${\mathrm{R}}_E$ is fixed, or with ${\mathrm{R}}_E$ when the input potential is constant.

Starting from this characteristic of the electronic neuron, the neuromorphic sensor uses the FSR instead of ${\mathrm{R}}_E$ and a fixed potential ${\mathrm{V}}_{FSR}$ to activate the neuron. The value of ${\mathrm{V}}_{FSR}$ is chosen empirically to increase the range of the spiking rates generated by the SOMA while keeping its sensor’s sensitivity high. In order to avoid the use of an additional power supply, ${\mathrm{V}}_{FSR}$ is adapted to the VDD through the auxiliary circuit AUX which is highlighted in blue in Figure 1. The synapses driven by this modified SOMA generate spikes which frequency depends on the applied force on the FSR.

The Structure of the SNN

The neuromorphic sensor NS_FSR is tested in a control application where the force applied on the tip of an anthropomorphic finger is regulated by a SNN which structure is shown in Figure 2a. This neural network drives the SMA wire that actuates the finger, and receives information from the sensor. The input layer of the SNN includes several excitatory neurons $E_{\mathrm{1,4}}$ that activates the motor neurons $M_{\mathrm{1,2}}$ that builds the output layer. The motor neurons are inhibited by the NS_FSR through the inhibitory synapses IS₁ and IS₂ as in Figure 2a. For evaluation of the SNN behavior with the load cell the NS_FSR is replaced by the CLC with an inhibitory neuron and the circuit POT as in Figure 2b.

The motor neurons stimulates through SMA driver the actuators which contraction force is determined directly by the neurons’ spiking rate. The frequency of the motor neurons depends on the resultant activity of NS_FSR and $E_{\mathrm{1,4}}$ which frequency $f_E$ can be varied by adjusting the potential $V_E$. Therefore, considering that firing rate of motor neuron increases with the frequency of excitatory neurons, the force of the SMA actuator can be set using $V_E.$

Experimental Setup

We validated the proposed sensor using an anthropomorphic robotic finger that is flexed by a SMA actuator and has two force sensors on the finger’s tip, see Figure 3. The actuator is implemented 0.006” SMA wires type Flexinol LT (produced by Dynalloy, inc) that can reach up to 4% stroke and 321g force when a maximum of 410 mA heats the wire. For the reported experiments, the length of the actuator is 115 cm that is powered at $V_{CC}=24V.$ We compared the performances of a compression load cell type FS2050 (produced by TE Connectivity) and the force sensing resistor type FSR03CE. In order to reduce the influence of the contact area on the FSR response and other surface related errors [15] we glued a metal disk with diameter of 20 mm in front of the sensor. The system is evaluated by measuring the output of the CLC and the neuron’s frequency when during steady state when the finger pushes on an elastic tweezers. Note that the CLC is below the FSR in order to evaluate the behaviors of the finger with the two sensors in the same conditions. During the experiments the FSR based sensor and CLC are not connected to the input of the SNN in the same time implying that the use of two sensors for feedback is exclusive. However, the information about the finger’s force is obtained by reading the analogue output of the CLC even if the NS_FSR feeds the SNN. In order to deepen the comparison between the two sensors we determined the energy consumptions for the NS_FSR and respectively, of the CLC by measuring the electrical current that is used by the two circuits.

Figure 4 presents the diagram of the experimental setup that includes SNN, the SMA driver and the two sensors that are fixed on the finger’s tip.

The CLC is connected through a potentiometer POT to one inhibitory neuron included in the inhibitory area of the SNN that is shown in green. The FSR uses a SOMA, shown in dark blue, to drive the corresponding inhibitory synapses that is also integrated in the inhibitory area.

3. Results

The experimental work for this research is split in two phases that are focused on i) the evaluation of the NS_FSR and of CLC response off system, and ii) the validation of NS_FSR in robotic finger control.

i) Sensors response with the load mass

During the first experimental phase we determined the variation of the firing rate $f_E$ of the SOMA with the input voltage $V_E$ followed by evaluation of the response of CLC and NS_FSR with the load mass $m_L$. As presented in Figure 5a,b the activation rate ${\mathrm{f}}_E$ of the SOMA varies linearly with ${\mathrm{V}}_E$ when ${\mathrm{R}}_E$ is fixed, and similarly, the CLC out increases linearly with $m_L$. This behavior implies that the spiking rate of the inhibitory neuron is proportional with the load mass when the load cells is used as a force sensor.

For evaluation of the CLC’s response, we used 14 disks of 3.5 grams that are stacked sequentially above the sensor. For triggering the activity of the inhibitory neuron with the first disk we added an offset mass of 7.5 grams implying that the minimum mass that is used in this experimental phase is 11 grams. For one set of measurements we read the CLC out when the mass varied between 11 to 56 grams with the resolution of 3.5 grams, and this experiment is repeated 5 times. The obtained results are given in Figure 5b which shows that the output of the CLC is proportional with $m_L$.

Similarly, for evaluation of the FSR based neuromorphic sensor NS_FSR we performed 5 sets of measurements when the load mass ranges between 11 and 56 grams, see Figure 6. For this sensor we measured the output frequency $f_{NS}$ of the neuron that is directly related to the FSR value and consequently to the applied force.

Note that the variation of the frequency $\Delta f_{NS}$ decreases with ${\Delta m}_L$ showing that the spiking rate generated by the neuromorphic sensor increases following a logarithmic-like function with $m_L$ mainly above 25 grams. This behavior is a consequence of the power law that describes the resistance-force function of the FSR [11].

ii) Control of the robotic finger

For the second experimental phase, we implemented a robotic finger that is actuated by SMA and includes both force sensors on the distal phalange. In this setup, we evaluated the ability of this robotic finger to push and hold one tip of an elastic tweezers while the other tip is blocked. The contraction force of the actuator is adjusted by changing the input voltage $V_E$ that activates the excitatory neurons (EN) at constant frequency. For having a clear view of the SNN activity, Figure 7 shows an example of the pulses generated by the SOMAs during activation of the neurons. On these diagrams that are recorded using the oscilloscope, the blue signal illustrates the activity of motor neurons (MN) that integrates the activity of EN (green signal) and of the NS_FSR (magenta signal).

In Figure 8 the force of the SMA actuator varies due to the intermittent activity of the motor neurons implying that for lower frequencies of the EN the regulatory performance of the SNN decreases. In contrast, for higher frequencies of the EN the variation of the finger’s force during regulatory regime is insignificant.

Figure 9a presents the variation of the CLC out with the frequency of the excitatory neurons when this CLC drives the inhibitory neurons as in Figure 2b.

The frequency $f_E$ of EN is set to several values between 350 and 550 Hz with resolution of 25 Hz. The limits of this variation interval are chosen empirically to reduce the oscillation of the finger force for lower $f_E$ and to reach maximum contraction force for higher $f_E$. Similarly, the variation of $V_F$ with $f_E$ is given in Figure 9b when the proposed FSR based neuromorphic sensor contributes to the regulation of the motor neurons’ activity. Taking into account the linearity of the load cell’s response, in this case the CLC is also used to measure the force of the finger on the tweezer’s tip.

Taking into account that power consumption represents an important parameter of the neuromorphic systems, besides analysis of the SNN activity we measured the maximum electrical current used by both sensors for the maximum force. The obtained values are ~4 mA at 5V for the CLC and about 80 µA at 1.6 V for the NS_FSR, showing that the power consumption of the neuromorphic sensor is more than 150 times lower than the power needed for CLC to operate. In addition, lower pressure increases the resistance of the FSR and the power consumption of the NS_FSR implying that higher sensitivity reduces the consumed energy even further.

5. Conclusions

In this work we proposed a neuromorphic sensor based on an electronic spiking neuron which includes a cheap force sensing resistor. The frequency of the output spikes that are generated by the sensor varies with the FSR value and consequently with the applied force. The validation of the proposed sensor was performed in two phases when we measured directly the spiking frequency when load masses with known values are placed on the sensor, and, for the second phase, we integrate the sensor in a SMA actuated robotic finger and measured the force during steady state when the finger pushes on an elastic tweezers.

The evaluation of the sensor response shows that the output of the CLC is proportional with the load mass and consequently with the applied force implying that the output of the neuron driven by the CLC is also linear. On the other hand, the variation of the output frequency of NS_FSR with the load mass follows a logarithmic-like function that is determined by the nonlinearity of the force-resistance function of the FSR. The results of the second experimental phase shows that despite the fact that the NS_FSR has lower linearity than the CLC, this FSR based sensor can be used to discriminate a wide range of applied forces. In addition, the power consumption of the NS_FSR is significantly lower than that of the CLC. This advantage makes the proposed sensor useful not only for implementation of bioinspired control systems in robotics, but also for low power electronics based on neuromorphic hardware.

For future work we intend to use the proposed sensor for a robotic hand with five fingers and to reduce the power consumption of the electrical circuit that implements the spiking neuron. In addition, we will design structures of spiking neurons that compensates for the errors of the FSR.