Artificial Intelligence-Based Algorithms in Medical Image Scan Segmentation and Intelligent Visual-Content Generation

1. Introduction

The human brain, a paramount example of evolutionary biological sophistication, transcends its anatomical categorization. Constituted by an estimated 86 billion neurons linked through an intricate web of synapses (ranging in the trillions), it is the epicenter of our cognitive, emotional, and consciousness-related functions [1]. This masterful structure of the central nervous system represents a nexus of myriad neurobiological processes, intricately overseeing sensory input conversion, motoric responses, and advanced cognitive functionalities. As a product of relentless evolutionary adaptations spanning millions of years, the brain epitomizes the apex of neurobiological optimization, synergizing complex neural circuitry with higher-order cognitive undertakings such as cognitive reasoning, emotional homeostasis, and the intricate processes of memory encoding, storage, and retrieval [1,2,3]. Thus, the human brain is a super-complex system whose functioning and intelligence depend rather on the type of neurons (depending on their role in the brain), their connections, and the way of supplying energy to neurons than the number of neurons [2]. It is an ideal reference model for the foundations of Artificial Intelligence (AI) [3,4].

Thus, processing and analysis of biomedical data for diagnostic purposes is a multidisciplinary field that combines AI, Machine Learning (ML), biostatistics, time series analysis as well as statistical physics and algebra (e.g. graph theory) [3]. Variables derived from biomedical phenomena can be described in several ways and in different domains (time, frequency, spectral values, spaces of states describing the biological system), depending on the characteristics and type of signal. Effective diagnosis of the early stages of the disease, as well as the determination of disease development trends, is a very difficult issue that requires taking into account many factors and parameters. Therefore, the state spaces of biomedical signals are huge and impossible to fully search, analyze, and classify even with the use of powerful computational resources. Therefore, it is necessary to use Artificial Intelligence, in particular, bio-inspired AI methods to limit research to a smaller but significant part of the state space.

Recently, computer-generated three-dimensional (3D) images have become increasingly important in medical diagnostics [5,6]. In particular, Extended Reality (XR) so-called Metaverse is increasingly used in health care and medical education, while it enables the deeper experience of the virtual world, especially through the development of depth perception, including the rendering of several modalities like vision, touch, and hearing [7]. In fact, medical images have different modalities and their accurate classification at the pixel level enables the accurate identification of disorders and abnormalities [7,8]. However, creating a 3D model of organs and/or their abnormalities is time-consuming and is often done manually or semi-automatically [10]. AI can automate this process and also contribute to increasing the quality of the resulting 3D objects [11,12] as well as visual content in the Metaverse [4,13]. To give the users a real sense of visual immersion, the developers should virtual objects of high quality [14]. In the context of medicine, it is combined with good quality medical data and their classification/segmentation algorithms with high accuracy, to faithfully reproduce the content in virtual three dimensions.

For that reason, this paper focuses on the overview of Artificial Intelligence-based algorithms in medical image scan segmentation and intelligent visual content generation in Extended Reality, including different types of neural networks used and learning rules, taking into account mathematical/theoretical foundations, algorithm accuracy, and performance, as well as open data availability.

2. Materials and Methods

The methodology of review methodology was based on the PRISMA Statement [15] and its extensions: PRISMA-S [16]. We considered recent publications, reports, protocols, and review papers from Scopus and Web of Science databases. The keywords: Artificial Intelligence, Machine Learning, Extended Reality, Mixed Reality (MR), Virtual Reality (VR), Metaverse, learning algorithms, learning rules, signal classification, signal segmentation, medical image scan segmentation, segmentation algorithms, classification algorithms, and their variations. The selected sources were analyzed in terms of compliance with the analyzed topic, and then their contribution to medical image scan segmentation. First, the obtained title and abstract were independently evaluated by the authors. The duplicated records have been removed. Moreover, we have considered the inclusion of criteria-like publication in the form of journal papers, books, and proceedings as well as technical reports. The search was limited to full-text articles in English, including electronic publications before printing. Also, the exclusion criteria like Ph. D. thesis and materials not related to medical image scan segmentation and Artificial Intelligence-based algorithms have been adopted. Subsequently, articles meeting the criteria were retrieved and analyzed. The documents used in this study were selected based on the procedure presented in Figure 1. Finally, 162 documents were taken into account.

3. Neural communication

Neurons, which are basic brain building blocks, function as the core computational units of the brain, underpinning the vast expanse of conscious and subconscious processes, and defining our neural identity with each electrochemical interaction [11]. The quintessence of neural communication is synaptic transmission. At these specialized junctions, the presynaptic neuron releases neurotransmitters, a diverse group of chemicals, into the synaptic cleft. Following the release, these compounds traverse the synaptic gap, interacting with receptors on the postsynaptic membrane, eliciting a series of intracellular events, potentially leading to the generation of an action potential, a transient depolarizing event propagated along the neuronal membrane. The multifunctionality of neurons is evident across physiological domains. While some mediate rudimentary autonomic functions, such as cardiac rhythm regulation, others participate in higher-order cognitive tasks, encompassing analytical reasoning and conceptual abstraction. Based on their anatomical localization and associated circuits, neurons can modulate affective states, dictate cognitive strategies, and contribute to individual behavioral phenotypes. Moreover, the synaptic connections between neurons exhibit plasticity, an inherent ability to modify their strength or form novel connections, representing experiential and learning-based adaptations. Such neural plasticity underscores the capacity for cognitive and behavioral adaptability, ensuring the brain's functional flexibility across an individual's lifespan.

Since the famous experiments of Adrian [17], it is assumed that in the nervous systems (including the brain), information is transmitted through weak electric currents (on the order of 100 (mV)), in particular employing action potentials (spikes) that are a transient, sudden (1-2 millisecond) change in the membrane potential of the cell/neuron associated with the transmission of information [18]. The stimulus for the creation of an action potential is a change in the electric potential in the cell's external environment. A wandering action potential is called a nerve impulse. In literature [19,20] it is assumed that the sequences of such action potentials, called spike-trains, play a key role in the transmission of information, and the times of appearance of these action potentials play a significant role. Mathematically, such time sequences can be and are modeled in particular after digitalization as trajectories (or their various variants) of certain stochastic processes (Bernoulli, Markov, Poisson, ...) [19,21,22,23,24,25,26,27].

4. Taxonomy of neural network applied in the medical image segmentation process

The Artificial Neural Networks (ANNs) are constructed with the perceptron neuron model [28] that is based on the binary decision rule. If the linearly weights $w_i$the sum of the input signals (input vector $x_i$) exceeds the threshold $t_{hr}$ neuron fires (i.e. the output is equal to 1) or if not output is equal to 0.

The basic input function is described as follows

$$f\left(x\right)=\left\{\begin{array}{c}1, \mathrm{i}\mathrm{f} w_1{x}_1+w_2{x}_2+\dots +w_n{x}_n\ge t_{hr}\\ 0, \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(1)

The output vector of all neurons in $l$-th layer can be expressed as well as the combination of the linear transformation and non-linear mapping (i.e. ANN activation values) [29].

$$a^l=h\left({\mathit{W}}^l{a}^{l-1}\right), i=1,\dots ,M$$

(2)

where $W^l$ is the weight matrix between layer $l$ and $l-1$, and $h(\bullet )$ denotes the activation function, in this case, Rectified Linear Unit (ReLU) $f\left(x\right)=x^{+}=\max \left(0,x\right)$ and the vector $a^l$ denotes the output of all neurons in $l$-th layer. The formula (2) has been quoted following the designations in the publication [29]. Neuron models from the Integrate-and-Fire family are among the simplest, however also the most frequently used. They are classified as spiking models. From a biophysical point of view, action potentials are the result of currents flowing through ion channels in the membrane of nerve cells. The Integrate-and-Fire neuron model [30,31] focuses on the dynamics of these currents and the resulting changes in membrane potential. Therefore, despite numerous simplifications, these models can capture the essence of neuronal behavior in terms of dynamic systems.

The concept of Integrate-and-Fire neurons is the following: the input ion stream depolarizes the neuron's cell membrane, increasing its electrical potential. An increase in potential above a certain threshold value$U_{thr}$ produces an action potential (i.e. an impulse in the form of Dirac's delta) and then the membrane potential is reset to the resting level. The leaky Integrate-and-Fire (LIF) neuron model [30,31] is an extended model of the Integrate-and-Fire neuron, in which the issue of time-independent memory is solved by equipping the cell membrane with a so-called leak. This mechanism causes ions to diffuse in the direction of lowering the potential to the resting level or another level $U_0\to U_{leak}<U_{thr}$. Thus, the third generation of neural networks, i.e. the Spiking Neural Networks (SNN) [32] are mostly based on the LIF, where the membrane potential $U\left(t\right)$ is determined by the equation

$${\tau }_m\frac{dU}{dt}=-[U\left(t\right)-U_{rest}]+R_{m}I\left(t\right),$$

(3)

where τ_m is the membrane time constant of the neuron, R_m is total membrane resistance, and I(t) is the electric current passing through the electrode. The spiking events are not explicitly modeled in the LIF model. Instead, when the membrane potential U(t) reaches a certain threshold U_th (spiking threshold), it is instantaneously reset to a lower value U_rest (reset potential) and the leaky integration process starts a new one with the initial value U_r. To mention just a little bit of realism to the dynamics of the LIF model, it is possible to add an absolute refractory period Δ_abs immediately after U(t) hits U_th. During the absolute refractory period, U(t) might be clamped to U_r, and the leaky integration process is re-initiated following a delay of Δ_abs after the spike. More generally, the membrane potential (3) can be presented as

$$U\left(t\right)={\sum }_{i=1}^N{\omega }_i{\sum }_{t_i<t}u(t-t_i)$$

(4)

where $u\left(t\right)$ is a fixed casual temporal kernel that is an operation that allows scale covariance and scale invariance in a causal-temporal and recursive system over time [33] and ${\omega }_i, i=1,.., N$ denotes the strength of neuron synapses. Following Equation (2), the neuron's output $m^l\left(t\right)$ (membrane potential after the neuron firing) can be described as follows [29]

$$m^l\left(t\right)=v^l(t-1)+W^l{x}^{l-1}\left(t\right) l=1,\dots ,N$$

(5)

where $v^l$ denotes the membrane potential before the neuron fires, $W^l$ is the weight in $l$-th layer ($l$ denoted layer index), and $x^{l-1}\left(t\right)$ is the input from the last layer. Thus, to avoid the loss of information the reset-by-subtraction” mechanism was introduced [34]

$$v^l\left(t\right)-v^l\left(t-1\right)=W^l{x}^{l-1}\left(t\right)-\left(H\right(m^l\left(t\right)-{\mathit{\theta }}^l\left){\theta }^l\right)$$

(6)

where $v^l\left(t\right)$is membrane potential after firing, $m^l\left(t\right)-$membrane potential before firing, $H(m^l\left(t\right)-{\theta }^l)$ refers to the output spikes of all neurons, and ${\mathit{\theta }}^l$ is a vector of the firing threshold ${\theta }^l$. There are also some applications of the concepts of the meta-neuron model in SNNs [35]. The main differences between the LIF neuron and meta neurons stay in the integration process, where meta neurons use a 2nd-order ordinary differential equation and an additional hidden variable. The basic differences between ANN and SNN (taking into account the type of neuron models) are presented in Figure 1.

Figure 1. The scheme of the basic differences between ANN and SNN takes into account the type of neuron models.

Figure 1. The scheme of the basic differences between ANN and SNN takes into account the type of neuron models.

4.1. Convolutional Neural Network

The most commonly used deep neural network (DNN) in medical image classification is the two-dimensional (2D) Convolutional Neural Network (CNN) [36,37]. In the Figure 2. The basic scheme of the SNN is presented. Its principle of operation is based on linear algebra, in particular matrix multiplication. CNNs consist of three types of layers: a convolutional layer, a pooling layer, and a fully connected layer. In fact, most computations are performed in the convolutional layer or layers. The image (pixels) is converted into binary values ​​and patterns are searched. Every convolutional layer operates a dot product between two matrices, namely one matrix is ​​a set of learnable parameters (kernel), and the second matrix is ​​a limited part of the receptive field. Each subsequent layer contains a filter/kernel that allows you to classify features with greater efficiency. A pooling layer reduces the number of parameters in the input, which causes the loss of part of the information calculated in the common layer/layers, however, it allows for improvement in the efficiency of the CNN network. This operation is performed by sliding windows [38]. Next, the output of these two layers is transformed into a one-dimensional vector, i.e. input to the fully connected layer. In this last type of layer, image classification based on the features extracted in the previous layers is performed, i.e. the object in the image is recognized. The output $y_{i,j}^{\left(k\right)}$ from CNN can be described as follows

$$y_{i,j}^{\left(k\right)}=\sigma ({\sum }_{l=1}^L{\sum }_{m=1}^M{x}_{i+l-1,j+m-1}^{\left(l\right)}{w}_{l.m}^{\left(k\right)}+b^{\left(k\right)})$$

(7)

where $x_{i,j}^{\left(l\right)}$ denotes input to the network at the spatial location $(i,j)$, $\sigma$ is the activation function, $w_{l.m}^{\left(k\right)}$ is the weight of the $m$th kernel at the $l$th channel producing the $k$th feature map, and $b^{\left(k\right)}$) is the bias for $k$th feature map.

In the case of large datasets, CNN achieves high efficiency and is resistant to noise [39]. The crucial disadvantages of CNNs in image processing are high computational requirements and difficulties in achieving high efficiency in the case of small datasets (i.e. if the dataset is too small the network may overfit to training data, and poorly recognize new data).

Figure 2. The basic scheme of the simple Convolutional Neural Network.

Figure 2. The basic scheme of the simple Convolutional Neural Network.

4.2. Recurrent Neural Network

Another neural network commonly applied in medical data analysis is the Recurrent Neural Network (RNN) [40]. In the Figure 3. The basic scheme of the RNN is presented. This type of network contains at least one feedback connection. The output of RNN can be expressed as [41]

$$y_i=W_{hy }\mathcal{H}\left(W_{hh}{h}_{i-1}+W_{xh}{x}_i+b_h\right)h_i+b_y$$

(8)

where $x_i$, $i-1,\dots ,N$, $W_{xh }$, $W_{hy }$,$W_{hh }$denotes weight matrices,$b_h$, $b_y$ are bias vectors, and $\mathcal{H}$ is the non-linear activation function, for example, ReLU, Sigmoid $f\left(x\right)=\frac{1}{1+e^{-x}}$, Tanh Function (Hyperbolic Tangent)$f\left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$ . The network operation is recursive since the hidden layer state depends on the current input and the previous state of the network. Thus, the hidden state $h_{i-1}$ is the memory of past inputs.

Thus, the RNN can operate on the sequential dataset and has an internal memory. It may have many inputs. However, RNNs exhibit learning-related problems, namely vanishing gradients (i.e. in the case of small gradients the updates of parameters are irrelevant) or exploding gradients (i.e. superposition of large error gradients leading to large parameter updates). These contribute to the long training process, low level of accuracy, and low network performance.

Figure 3. The basic scheme of the simple Recurrent Neural Network.

Figure 3. The basic scheme of the simple Recurrent Neural Network.

4.3. Spiking Neural Networks

Besides the Artificial Neural Networks, i.e. CNNs, and RNNs, one can also be applied to the medical signals bio-inspired neural networks like Spiking Neural Networks [41,42]. In the Figure 4. The basic scheme of the SNN is presented. SNNs encode information taking into account spike signals, and shells are promising in effectuating more complicated tasks, while the more spatiotemporal information is encoded with spike patterns [43]. They are mostly based on the LIF neuron model. SNNs were formulated to map organic neurons, i.e. the appearance of the presynaptic spike at synapse triggers the input signal $i\left(t\right)$ (the value of the current) that in the simplified cases can be written as follows

$$i\left(t\right)={\int }_0^{\infty }{S}_j\left(s-t\right)\exp (\frac{-s}{{\tau }_s})\mathrm{d}s$$

(9)

where ${\tau }_s$ denotes synaptic time constant, $S_j$ is a presynaptic spike train, $t$ is time [44]. In contrast, the majority of DNNs do not take into account temporal dynamics [45]. In fact, SNNs show promising capability in playing a similar performance as living brains. Moreover, the binary activation in SNNs enables the development of dedicated hardware for neuromorphic computing [46]. The potential benefits are low energy usage and greater parallelizability due to the local interactions.

Figure 4. The basic scheme of the simple Spiking Neural Network.

Figure 4. The basic scheme of the simple Spiking Neural Network.

5. Learning algorithms

The heart of Artificial Intelligence is its learning algorithms. At their core, strive to automate the learning process, enabling machines to recognize patterns, make decisions, and predict outcomes based on data. Their design is often a balance between theoretical rigor and practical applicability. While mathematics and statistics provide the foundation, translating these into algorithms that can operate on vast and diverse datasets requires creative programming skills [22]. One can distinguish many types of network training algorithms [47]. Below we briefly discuss the most important of them.

5.1. Back Propagation Algorithm

The most commonly used learning algorithm is the back propagation (BP) algorithm. Ititers overweight optimizations via error propagation in the neural networks. BP plays a pivotal role in enabling neural networks to recognize complex and non-linear patterns from large datasets [23,48,49]. From the mathematical point of view, it is a calculation of the cost function, which minimizes the calculated error of the output using gradient descent or delta rule [50]. It can be split into three stages: forward calculation, backward calculation, and computing the updated biases and weights. The input to the hidden layer ${\mathit{H}}_{\mathit{j}}$ is the weighted sum of the outputs of the input neurons and can be described as [51]

$${\mathit{H}}_{\mathit{j}}={\mathit{b}}_{\mathit{i}\mathit{n}}+{\sum }_{\mathit{i}=1}^{\mathit{n}}{\mathit{x}}_{\mathit{i}}{\mathit{w}}_{\mathit{i}\mathit{j}}$$

(10)

where ${\mathit{x}}_{\mathit{i}}$ is the input to the network (input layer), $\mathit{n}$ is the number of neurons in the input layer, ${\mathit{b}}_{\mathit{i}\mathit{n}}$ is the bias input layer, and ${\mathit{w}}_{\mathit{i}\mathit{j}}$ denotes the weight associated with the $\mathit{i}$-th input neuron and the $\mathit{j}$-th hidden neuron. The output ${\mathit{y}}_{\mathit{k}}$is as follows

$${\mathit{y}}_{\mathit{k}}={\mathit{b}}_{\mathit{h}}+{\sum }_{\mathit{j}=1}^{\mathit{m}}{\mathit{w}}_{\mathit{j}\mathit{k}}\mathit{F}\left(\mathit{H}\right(\mathit{j}\left)\right)$$

(11)

where $\mathit{F}\left(\mathit{H}\right(\mathit{j}\left)\right)$ is a transfer function, $\mathit{k}$ is the number of neurons in the hidden layer, and ${\mathit{b}}_{\mathit{h}}$ is the bias of the hidden layer. The most commonly used transfer function is the sigmoid transfer function $\mathit{F}\left(\mathit{H}\left(\mathit{j}\right)\right)=\frac{1}{1+{\mathit{e}}^{-\left(\mathit{H}\left(\mathit{j}\right)\right)}}$. The back propagation algorithm is especially effective when used in multi-layered neural architectures such as feed-forward neural networks, convolutional neural networks, and recurrent neural networks [26]. In image recognition, CNNs, energized by BP, can independently identify hierarchical features, from basic edges to detailed structures. Similarly, RNNs, amplified by BP, are adept at sequence-driven tasks like machine translation or speech recognition, as they incorporate previous data to influence present outputs. It is one of the most effective deep learning methods. However, BP requires large amounts of data and enormous computational efforts.

5.2. ANN-SNN Conversion

Artificial Neural Networks and Spiking Neural Networks are both computational models inspired by biological neural networks. While ANNs have been the mainstream for most deep learning applications due to their simplicity and effectiveness, SNNs are gaining traction because they mimic the behavior of real neurons more closely by using spikes or binary events for communication. To obtain a similar accuracy of the SNN-based algorithm as the algorithm using ANN, for example, the BP-type training rule consumes a lot of hardware resources. And the already existing platforms have limited optimization possibilities. Thus, the conversion of ANNs to SNNs seeks to harness the energy efficiency and bio-realism of SNNs without reinventing the training methodologies [28], while it is based on the ReLU activation function and LIF neuron model [52]. The basic principle of the conversion of ANNs to SNNs is mapping the activation value of the ANN neuron to the average postsynaptic potential (in fact, firing Rate) of SNN neurons, and the change of the membrane potential (i.e. the basic function of spiking neurons) can be expressed by the combination of the Equation (2) and Equation (6)[29]

$${\mathit{v}}^{\mathit{l}}\left(\mathit{t}\right)-{\mathit{v}}^{\mathit{l}}\left(\mathit{t}-1\right)={\mathit{W}}^{\mathit{l}}{\mathit{x}}^{\mathit{l}-1}\left(\mathit{t}\right)-{\mathit{s}}^{\mathit{l}}\left(\mathit{t}\right){\mathit{\theta }}^{\mathit{l}}$$

(12)

Here ${\mathit{s}}^{\mathit{l}}\left(\mathit{t}\right)$ refers to the output spikes of all neurons in layer $\mathit{l}$ at time $\mathit{t}$.

Tuning the right thresholds is paramount for the SNN to effectively and accurately represent information. Incorrectly set thresholds could lead to either too frequent or too rare spiking, potentially affecting the accuracy of the SNN post-conversion [35]. On the other hand, the neuromorphic hardware platforms that support SNNs natively can primarily offer energy efficiency benefits by converting ANNs to SNNs. Due to their event-driven nature, SNNs can be more computationally efficient [36]. However, the challenge lies in maintaining accuracy post-conversion. Some information might be lost during the transition, and not all ANN architectures and layers neatly convert to their SNN equivalents. The conversion from ANNs to SNNs is a promising direction, merging the advanced training methodologies of ANNs with the energy efficiency of SNNs. As we delve deeper into the realm of neuromorphic computing, this conversion process will play a pivotal role in bridging traditional deep learning with biologically-inspired neural models [37,38].

5.3. Supervised Hebbian Learning (SHL)

Taking into account Artificial Intelligence, Supervised Hebbian Learning (SHL)can be described as a general methodology for weight changes [53]. Thus, this weight increases when two neurons fire at the same time, while it decreases when two neurons fire independently. According to this rule, the change in weight can be written

$$∆\mathit{w}=\mathit{\eta }({\mathit{t}}^{\mathit{o}\mathit{u}\mathit{t}}-{\mathit{t}}^{\mathit{d}})$$

(13)

where $\mathit{\eta }$ is the learning rate (in fact, the small scalar that may vary with time, $\mathit{\eta }>0$), ${\mathit{t}}^{\mathit{o}\mathit{u}\mathit{t}}$ the actual time of the postsynaptic spike, while ${\mathit{t}}^{\mathit{d}}$ is the time of firing of the second presynaptic spike [54,55]. The crucial disadvantage of Hebbian learning is the fact that when the number of hidden layers increases the efficiency decreases, while in the case of 4 layers is still competitive [56].

5.4. Reinforcement Learning with Supervised Models

According to the additional constraints in the SHL rule, Reinforcement Learning with Supervised Models (ReSuMe) was proposed [54]. ReSuMe, is a dynamic hybrid learning paradigm. It effectively combines the resilience of Reinforcement Learning (RL) with the precision of Supervised Learning (SL). This fusion empowers ReSuMe to leverage feedback-driven mechanisms inherent in RL and take advantage of labeled guidance typical for SL [37,38,39]. The difference between SHL is that the learning signal is expected not to have or have a marginal direct effect on the value of the postsynaptic somatic membrane potential [57], thus the synaptic weights are modified as follows

$$\frac{\mathrm{d}}{\mathrm{d}\mathit{t}}{\mathit{w}}_{\mathit{j}\mathit{i}}\left(\mathit{t}\right)=\mathit{a}[{\mathit{S}}_{\mathit{d}}\left(\mathit{t}\right)-{\mathit{S}}_{\mathit{j}}\left(\mathit{t}\right)]{\overline{\mathit{S}}}_{\mathit{i}}\left(\mathit{t}\right)$$

(14)

where $\mathit{a}$ denoted learning rate, ${\mathit{S}}_{\mathit{d}}$ is desired/targeted spike train, ${\mathit{S}}_{\mathit{j}}\left(\mathit{t}\right)$ is the output of the network (spike train), and ${\overline{\mathit{S}}}_{\mathit{i}}\left(\mathit{t}\right)$ expresses the low-pass filtered input spike train. ReSuMe guided one of its most salient features exploration. By leveraging labeled data via SL, ReSuMe can effectively steer RL exploration, ensuring agents avoid falling into the trap of suboptimal policies. The hybrid nature of ReSuMe also grants it a unique resilience, especially in the face of noisy data or in reward-scarce environments. Moreover, its adaptability is noteworthy, making it an ideal choice for tasks that combine immediate feedback (through SL) with long-term strategic maneuvers (through RL). However, like all things, ReSuMe is not without challenges. A potential bottleneck in ReSuMe is computational complexity, as managing both RL and SL can sometimes strain computational resources. Another challenge is the precise tuning of the λ coefficient. The key is to find a balance where neither RL nor SL overly dominates the learning process. By melding immediate feedback from supervised learning with a deep reinforcement learning strategy, ReSuMe establishes itself as a formidable tool in Machine Learning [49,50,52].

5.5. Chronotron

The Chronotron, by its essence, challenges and reshapes our understanding of how information can be encoded and processed in neural structures [50,55]. Traditional neural models have predominantly focused on the spatial domain, emphasizing the architecture and interconnections between neurons. While this spatial component is undeniably critical, it offers only a part of the full informational symphony that the brain plays. Just as the rhythm and cadence of a song contribute as much to its essence as its melody, in the vast theater of the brain, timing is not just a factor; it is a storyteller in its own right. The brilliance of the Chronotron lies in its ability to discern and respond to this temporal narrative. Unlike its counterparts, which often treat time as a secondary parameter, the Chronotron places it center stage. As a consequence, it acknowledges and leverages the intricate interplay of spatial and temporal dynamics in neural computation. This means that it doesn't just consider which neurons are firing, but also pays meticulous attention to when they fire concerning one another. Thus, the membrane potential is

$$\mathit{u}\left(\mathit{t}\right)=\mathit{\eta }\left(\mathit{t}\right)+{\sum }_{\mathit{j}}{\mathit{w}}_{\mathit{j}}{\sum }_{{\mathit{t}}_{\mathit{j}}^{\mathit{f}}\le \mathit{t}}{\mathit{\epsilon }}_{\mathit{j}}(\mathit{t},{\mathit{t}}_{\mathit{j}}^{\mathit{f}})$$

(15)

Where the models the$\mathit{\eta }$ model's refractoriness is caused by the past presynaptic spikes, ${\mathit{w}}_{\mathit{j}}$is the synaptic efficacy, ${\mathit{t}}_{\mathit{j}}^{\mathit{f}}$is the time of appearance of the $\mathit{f}$-th presynaptic spike on the $\mathit{j}$ synapse, ${\mathit{\epsilon }}_{\mathit{j}}\left(\mathit{t},{\mathit{t}}_{\mathit{j}}^{\mathit{f}}\right)$denotes normalized kernel [58]. When $\mathit{u}\left(\mathit{t}\right)$ reaches the threshold level, a spike is fired. And $\mathit{u}\left(\mathit{t}\right)$ is reset to the value of reset potential. In this approach, it is crucial to find the appropriate error functions, i.e. such an error function that enables the minimization with a gradient descent method [59]. The advantage of this learning rule is the fact that it uses the same coding for inputs and outputs. Chronotron's hallmark, its granularity, can sometimes surge computational demands, especially during intense training. And like many cutting-edge neural frameworks, harnessing Chronotron's full potential can be intricate, necessitatin' fine-tuned parameters and rich, well-timed data.

5.6. Bio-inspired Learning Algorithms

Brain-inspired Artificial Intelligence approaches, in particular spiking neural networks, are becoming a promising energy-efficient alternative to traditional artificial neural networks [60]. However, the performance gap between SNNs and ANNs has been a significant obstacle to the wild SNNs application (applicable SNNs). To fully use the potential of SNNs, including the detection of the non-regularities in biomedical signals, and designing more specific networks, the mechanisms of their training should be improved, one of the possible directions of development is the bio-inspiring learning algorithms. Below we briefly discuss the most important of them.

5.6.1. Spike Timing Dependent Plasticity

Spike Timing Dependent Plasticity (STDP) is rooted in the idea that the precise timing of neural spikes critically affects changes in synaptic strength [61]. This principle highlights the intricate dance between time and neural activity, showcasing the dynamics of our neural circuits. This biologically plausible learning rule is a timing-dependent specialization of Hebbian learning (13) [62]. STDP shed light on the intricate interplay between timing and synaptic modification. It is based on the change in synaptic weight function

$$∆\mathit{W}=\mathit{\eta }\left(1+\mathit{\zeta }\right)\mathit{H}(\mathit{W};{\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}})$$

(16)

where $\mathit{\eta }$ denotes the learning speed, $\mathit{\zeta }$ is Gaussian white noise with zero mean, while $\mathit{H}(\mathit{W};{\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}})$ is the function, that determines the long-term potentiation (LTP, ie. presynaptic and postsynaptic neurons emit a high rate) and depression (LTD, i.e. presynaptic neurons emit a high rate) in the time window ${\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}}$ [63]

$$\mathit{H}(\mathit{W};{\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}})\left\{\begin{array}{c}{\mathit{a}}_{+}\left(\mathit{W}\right)exp(-\frac{|{\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}}|}{{\mathit{\tau }}_{+}}) \mathrm{f}\mathrm{o}\mathrm{r} {\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}}<0\\ {-\mathit{a}}_{-}\left(\mathit{W}\right)exp(-\frac{|{\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}}|}{{\mathit{\tau }}_{-}}) \mathrm{f}\mathrm{o}\mathrm{r} {\mathit{t}}_{\mathit{p}\mathit{r}\mathit{e}}-{\mathit{t}}_{\mathit{p}\mathit{o}\mathit{s}\mathit{t}}>0\end{array}\right.$$

(17)

where $\mathit{a}\left(\mathit{W}\right)$ is a scaling function that determines the weight dependence, while $\mathit{\tau }$ denotes the time constant for depression [61,62,63]. STDP's significance is underpinned by its numerous advantages. Chiefly, it offers a biologically authentic model by 'mimicking the temporal dynamics observed in real neural 'systems. Furthermore, its event-centric nature promotes unsupervised learning, enabling networks to autonomously adjust based on the temporal patterns present in input data. This time-based sensitivity equips STDP to adeptly process data with spatiotemporal attributes and detect intricate temporal relationships within neuronal signals [64,65]. However, STDP is not without its complexities. A prominent challenge is the fine-tuning of parameters. The exact values assigned to constants like $\mathit{a}\left(\mathit{w}\right)$and $\mathit{\tau }$ can substantially dictate the behavior and efficacy of STDP-informed networks. Balancing these values requires a meticulous approach. Moreover, the precision demanded by STDP's time-centric nature often calls for higher computational rigor, especially within simulation contexts. STDP stands as a testament to the elegance and intricacy of neural systems. By emphasizing the role of spike timing, STDP offers a vivid depiction of how synaptic interactions evolve [66,67].

5.6.2. Spike-Driven Synaptic Plasticity

Spike-Driven Synaptic Plasticity (SDSP) offers the ability to elucidate the causality in neural communication. It operates on a fundamental principle: the sequence and timing of spikes determine whether a synapse strengthens or weakens. If a neuron consistently fires just before its downstream counterpart, it's a strong indication of its influential role in the latter's activity. This "pre-before-post" firing often leads to synaptic strengthening, cementing the relationship between the two neurons. Conversely, if the sequence is reversed, with the downstream neuron firing before its predecessor, the connection may weaken, reflecting a lack of causal influence [68,69]. This causative aspect of SDSP provides valuable insights into the learning mechanisms of the brain. It suggests that our neural circuits are continually evolving, adjusting their connections based on the flow of spike-based information. Such adaptability ensures that our brains remain receptive to new information, enabling us to learn and adjust to ever-changing environments. Moreover, SDSP emphasizes the significance of precise spike timing. In the realm of neural computation, milliseconds matter. Small shifts in spike timing can change a synapse's fate, showcasing the brain's precision and sensitivity. This meticulousness in spike-driven modifications underscores the importance of timing in neural computations, hinting at the brain's capacity to encode and process temporal patterns with remarkable accuracy [70]. In this learning rule the changes in synaptic weights can be expressed as [64]

$$∆\mathit{w}=\left\{\begin{array}{c}{\mathit{\eta }}^{+}+{\mathit{e}}^{\raisebox{1ex}{$-|∆\mathit{t}|$}\!\left/ \!\raisebox{-1ex}{${\mathit{\tau }}^{+}$}\right.}\mathrm{i}\mathrm{f} ∆t>0\\ {\mathit{}\mathit{}\mathit{}\mathit{}\mathit{\eta }}^{-}+{\mathit{e}}^{\raisebox{1ex}{$-|∆\mathit{t}|$}\!\left/ \!\raisebox{-1ex}{${\mathit{\tau }}^{-}$}\right.}, \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(18)

where ${\mathit{\eta }}_{+}>0$ and ${\mathit{\eta }}_{-}<0$ denotes the learning parameters, ${\mathit{\tau }}_{+}$ and ${\mathit{\tau }}_{-}$ are time constraints, and $∆\mathit{t}$ is the difference between post- and pre-synaptic spikes. This representation, while streamlined, encapsulates the principle that the mere presence of a spike can induce modifications in the synaptic weight, either strengthening or weakening the connection based on the specific neural context and the directionality of the spike's influence [71,72,73].

The appeal of Spkie-Driven Synaptic Plasticity is manifold its primary virtue is its biological relevance. Focusing on individual spike occurrences mirrors the granular events that take place in real neural systems. Such an approach facilitates the modeling of neural networks in scenarios where individual spike occurrences are of paramount importance. Furthermore, by anchoring plasticity on singular events, this model is inherently suitable for real-time learning and rapid adaptability in dynamic environments [74].

A crucial challenge lies in the accurate capture and interpretation of individual spikes, especially in densely firing neural environments. Moreover, the plasticity model's sensitivity to' single events 'means that it can' be susceptible to noise, requiring sophisticated filtering mechanisms to discern genuine learning events from spurious spikes. SDS elucidates the profound influence of singular neuronal events on the grand tapestry of neural learning and adaptation [73].

5.6.3. Tempotron Learning Rule

One of the most interesting biological-inspired learning algorithms is the tempotron principle [65,76,77] It is designed to adapt synaptic weights based on the temporal precise patterns of incoming spikes, rather than only the frequency of such spikes. While traditional neural models might emphasize synaptic weights or connection topologies, tempotron underscores that the 'when' of a neural event can be as informative, if not more so, than the 'where' or 'how often' [78,79,80]. The tempotron learning rule is based on the LIF neuron model. It fires when (4) exceeds the threshold (binary decision). Thus, one can define the potential of the neuron’s membrane as a weighted sum of postsynaptic potentials (PSPs) from all appearance spikes [77]

$$\mathit{v}\left(\mathit{t}\right)=\sum _{\mathit{i}}{\mathit{\omega }}_{\mathit{i}}\sum _{{\mathit{t}}_{\mathit{i}}}\mathit{K}\left(\mathit{t}-{\mathit{t}}_{\mathit{i}}\right)+{\mathit{V}}_{\mathit{r}\mathit{e}\mathit{s}\mathit{t}}$$

(19)

where ${\mathit{\omega }}_{\mathit{i}}$ denotes synaptic efficacy, ${\mathit{t}}_{\mathit{i}}$ is the firing time of the $\mathit{i}$th afferents, ${\mathit{V}}_{\mathit{r}\mathit{e}\mathit{s}\mathit{t}}$ is resting potential, and $\mathit{K}$ is the normalized PSP kernel

$$\mathit{K}\left(\mathit{t}-{\mathit{t}}_{\mathit{i}}\right)={\mathit{V}}_0(\mathbf{exp}\left(\frac{-\left(\mathit{t}-{\mathit{t}}_{\mathit{i}}\right)}{{\mathit{\tau }}_{\mathit{m}}}\right)-\mathbf{e}\mathbf{x}\mathbf{p}(\frac{-\left(\mathit{t}-{\mathit{t}}_{\mathit{i}}\right)}{{\mathit{\tau }}_{\mathit{s}}}\left)\right)$$

(20)

where ${\mathit{\tau }}_{\mathit{m}}$ is the decay time constant of membrane integration, while ${\mathit{\tau }}_{\mathit{s}}$ denotes the decay time constant of synaptic currents.While the ${\mathit{V}}_0$ normalized the PSP that the maximum kernel value is equal to 1. The neuron is fired when the value of the potential of the neuron’s membrane (19) is greater than the value of the firing threshold. Next, the potential of the neuron’s membrane (19) smoothly decreases to the value of ${\mathit{V}}_{\mathit{r}\mathit{e}\mathit{s}\mathit{t}}$. In the case of the segmentation/classification task, the input to the neuron may belong to one of two classes, namely ${\mathit{P}}^{+}$when a stimulus occurs (i.e. pattern is presented) the neuron should fire), and ${\mathit{P}}^{-}$ when the pattern is presented neuron should not fired. Each input consists of $\mathit{N}$ spike trains. In turn, the tempotron learning rules are as follows

$$∆{\mathit{\omega }}_{\mathit{i}}=\mathit{\lambda }{\sum }_{{\mathit{t}}_{\mathit{i}}<{\mathit{t}}_{\mathit{m}\mathit{a}\mathit{x}}}\mathit{K}({\mathit{t}}_{\mathit{m}\mathit{a}\mathit{x}}-{\mathit{t}}_{\mathit{i}})$$

(21)

where ${\mathit{t}}_{\mathit{m}\mathit{a}\mathit{x}}$ is the time when the potential of the neuron’s membrane (19) reaches a maximum value. While $\mathit{\lambda }$ is the constant that is greater than zero in the case of ${\mathit{P}}^{+}$, and smaller than zero in the case ${\mathit{P}}^{-}$. In this operation, tempotron introduces gradient-decent dynamics, i.e. minimizing the cost function for each input pattern measures the maximum voltage that is generated by the erroneous patterns. In comparison to the STDP learning rule, tempotron can make the appropriate decision under a supervisory signal, by tuning fewer parameters than STDP. Thus, tempotron uses LTP and LTD mechanisms like STDP. The advantage of the tempotron learning rule is the speed of learning.

6. Neural networks and learning algorithms in the medical image segmentation process

Image segmentation has a crucial role in creating both, medical diagnosing supported by image analysis and virtual object creation like the medical digital twin (DT) of organs [66,67], holograms of the human organs [81,82], and virtual medical simulators [68,83]. One can split the image segmentation process into semantic segmentation (i.e. assigning a label or category to each pixel), instance segmentation (i.e. identifying and separating individual objects in an image and assigning a label to it), and panoptic segmentation (i.e. more complex tasks, which involves the two segmentations above) [77,78]. The application of AI enables to increase in the efficiency and speed of these processes [84]. In Table 1. the comparison of the AI-based algorithms applied in medical image scan segmentation taking into account the neuron model, the type of neural network, learning rule, and biological plausibility is shown. It turned out that the most commonly used in image segmentation are CNNs, in particular, Unet architecture and its variations [71,72,74,75,85]. In [73] the authors modified this neural network structure by adding dense and nested skip connections (UNet++), while [Yao et al., 2020] added the residual blocks and attention modules to enable the network to learn deeper features and increase the effectiveness of segmentation. To connect the efficiency of segmentation with access to global semantic information, often CNNs are combined with transformer blocks [85,86,87]. Another CNNs-based algorithm commonly used in medical image segmentation is You Only Look Once (YOLO), which is open-source software used under the GNU General Public License v3.0 license [88]. It uses one fully connected layer, the number (depending on the version) of convolution layers that are pre-trained with the CNN (YOLO v1 ImageNet, YOLO v2 Darknet-19, YOLO v3 Darknet-53, YOLO v4 CSPNet, YOLO v5 EfficientNet, YOLO v6 EfficientNet-L2, YOLO v7 ResNET, YOLO v8 RestNet), and pooling layer. The algorithm divides the input in the form of a photo into specific segmentations and then uses CNN to generate bounding boxes and class predictions. Recently, in image classification, SNN has become more popular [78,79] due to its low power consumption. However, SNN training rules require refinement to achieve ANN accuracy. Another interesting algorithm for natural image segmentation with was recently developed (April 2023) by Meta is the Segmentation Anything Model (SAM) [89,90]. This AI-based algorithm enables cutting out any object from the image with a single click. It uses CNNs and transformer-based architectures for image processing, in particular, transformers-based architectures are applied to extract the features, compute the embedding, and pomp the encoder. The first attempt has been made to apply it in the field of medical imaging, however, in medical segmentation, it is still not so accurate in comparison to other application fields [91,92]. The imperfections of the SAM algorithm in the field of medical image segmentation are mainly connected to insufficient numbers of training data. In [93], the authors proposed to apply the Med SAM Adapter to overcome the above limitations. The pre-training method like Masked Autoencoder (MAE), Contrastive Embedding-Mixup (e-Mix), and Shuffled Embedding Prediction (ShED) was applied. There is a lot of work in the area of ​​medical image segmentation using machine learning, but relatively little addresses the issue related to the network learning process itself (along with data, a key element in achieving high accuracy of the process) [94], see Table 1. Thus, the most commonly used learning algorithms in medical image segmentation are still on the low level of biological plausibility. On the other hand, in other image segmentation, in particular, biologically plausible learning algorithms are applied, for example, in the field of the images of handwritten digits [77].

The segmented structures (in this case organs and their disorders) may be next applied to the development of the 3D virtual environment [105]. These 3D objects may be implemented through for example, holograms displayed in the head-mounted display (HDMs) like Mixed Reality glasses in medical diagnostics [113], pre-operative imaging [114], surgical assistance [115,116], robotics surgery [117], and medical education [81,82]. However, the crucial issue is connected with the quality of obtained segmented structures, and this process can be significantly accelerated and improved by the use of Artificial Intelligence.

7. Data availability

One of the key issues in the development of AI algorithms in the field of medicine is the availability and quality of data, i.e. access to electronic health records (EHRs) [118,119]. Thus, the medical data should be anonymized. In Table 2 a summary of publicly available retrospective image scan medical databases is presented. Some authors also provide anonymized data upon request. It is worth stressing that data, including medical image scans, are subjected to various types of biases [120].

8. Discussion and conclusions

The effectiveness of learning algorithms is compared among others in terms of the number of learning cycles, number of objective function calculations, number of floating-point multiplications, computation time, and sensitivity to local minima. In addition to the selection of appropriate parameters and network structure, the selection of an appropriate (effective) network learning algorithm is of key importance. The most commonly applied learning algorithm in ANNs is backpropagation, however, it has a rather slow convergence rate and as a consequence, ANN has more redundancy [146]. On the other hand, the training of the SNNs due to quite complicated dynamics and the non-differentiable nature of the spike activity remains a challenge [147]. The three types of ANN and SNN learning rules can be distinguished: unsupervised learning, indirect, supervised learning, and direct supervised learning. Thus, a commonly used learning algorithm in SNNs is the arithmetic rule SpikePropo, which is similar in concept to the backpropagation (BP) algorithm, in which network parameters are iteratively updated in a direction to minimize the difference between the final outputs of the network and target labels [148,149]. The main difference between SNNs and ANNs is output dynamics. However, arithmetic-based learning rules are not a good choice for building biologically efficient networks. Other learning methods have been proposed for this purpose, including bio-inspired algorithms like spike-timing-dependent plasticity [150], spike-driven synaptic plasticity [151], and the tempotron learning rule [65,76,77]. STDP is unsupervised learning, which characterizes synaptic changes solely in terms of the temporal contiguity of presynaptic spikes and postsynaptic potentials or spikes [152], while spike-driven synaptic plasticity is supervised learning and uses rate coding. However, still, ANN with BP learning achieves a better classification performance than SNNs trained with STDP. To obtain better performance the combination of layer-wise STDP-based unsupervised and supervised spike-based BP was proposed [153,154]. Other commonly used learning algorithms are ReSuMe [57], and Chronotron [58]. The tempotron learning rule implements gradient-descent dynamics, which minimizes a cost function that measures the amount by which the maximum voltage generated by erroneous patterns deviates from the firing threshold. Tempotron learning is efficient in learning spiking patterns where information is embedded in precise timing spikes (temporal coding). Instead, [155] proposed a neuron normalization technique and an explicitly iterative neuron model, which resulted in a significant increase in the SNNs' learning rate. However, training the network still requires a lot of labeled samples (input data). Another learning algorithm is indirect. It firstly trains ANN (created with perceptron’s) and thereupon transforms it into its SNN version with the same network structure (i.e., ANN-SNN conversion) [156]. The disadvantage of such learning is the fact, that reliably estimating frequencies requires a nontrivial passage of time, and this learning rule fails to capture the temporal dynamics of a spiking system. The most popular direct supervised learning is gradient descent, which uses the first-spike time to encode input [157]. It uses the first-spike time to encode input signals and minimizes the difference between the network output and desired signals, the whole process of which is similar to the traditional BP. Thus, the application of the temporal coding-based learning rule, which could potentially carry the same information efficiently using less number of spikes than the rate coding, can help to increase the speed of calculations. On the other hand, active learning methods, including bio-inspired active learning (BAL), bio-inspired active learning on Firing Rate (BAL-FR), and bio-inspired active learning on membrane potential (BAL-M) have been proposed to reduce the size of the input data [158]. During the learning procedure, the labeled data sets are used to train the empirical behaviors of patterns, while the generalization behavior of patterns is extracted from unlabeled data sets. It leverages the difference between empirical and generalization behavior patterns to select the samples unmatched by the known patterns. This approach is based on the behavioral pattern differences of neurons in SNNs for active sample selection, and can effectively reduce the sample size required for SNNs training.

The integration of AI and Metaverse is a fact and suggests that AI may become the dominant approach for image scan segmentation and intelligent visual-content generation in the whole virtual world, not just medical applications [6,159]. Recently, the Segment Anything Model (SAM) based on AI was introduced for natural images [89], in [160] SAM was proposed to be applied to medical images with a high level of accuracy. Better image segmentation contributes the higher-quality virtual objects. AI application in the context of the Metaverse is connected with the identification and categorization of meta-verse virtual items [161]. Moreover, AI may lead to more efficient cybersecurity solutions in the virtual world [162]. However, this is closely related to the accuracy of AI-based algorithms and, consequently, the accuracy of their training.