Causal Fuzzy Deep Learning Algorithm for the Differentiation of Gliosarcoma From Glioblastoma

1. Introduction

Glioblastoma (GBM) is one of the most common brain cancers [1]. GBM generally happens in the glial cells [2] and has several variants. Gliosarcoma (GSM) is a cancer classified by the World Health Organization (WHO) as a variant of GBM [1,3]. It is particularly important to accurately distinguish GSM from GBM [3]. Radiomics is an emerging field for imaging data analysis. It has been successfully employed for the differentiation of the central nervous system tumors [4]. In a recent paper, Qian et al. [4] conducted a study to identify an optimal machine learning algorithm for the differentiation of GSM from GBM based on radiomics data analysis. Baldé and Ghosh [20] studied the causal effect of Edema on the differentiation of GSM from GBM by using machine learning algorithms in radiomics. To do so, the authors developed a two steps procedure: In the first step, they reduced the dataset dimension using the sure independence screening procedure [5,6]. In the second step, they employed the outcome adaptive lasso [7] or the generalized outcome adaptive lasso [8].

To find the causes of Gliosarcoma, most of the previous studies use dimension reduction and/or statistical machine learning algorithms.

In this paper, we will use the entire dataset without applying any dimension reduction algorithms. Specifically, we will use causal fuzzy logic rules from [9], combined with Transformer-based deep learning algorithms [10] to identify the possible causes of Gliosarcoma. To the best of our knowledge, we are the first to use Causal Fuzzy Deep Learning Algorithm to detect the causes of Gliosarcoma.

2. Causal reasoning

In causal reasoning, we are looking how or why something happened and the relationship between the causes and their effects. In causal reasoning, we examine the relationship between causes and effects, and we try to understand how or why something happened. To calculate the causes of an event, current causal models use Individual or Average Treatment Effect (I-ATE). For instance, Pearl [11] computes Average Causal Effect by subtracting the means of the treatment and control groups. Pearl uses Directed Acyclic Graphs (DAGs) to visualize and compute the associations or causal relationships between a set of elements. He also uses do operators, which are interventions on the nodes of DAGs, as well as probability theory, the Markov assumptions, and other concepts/methods/tools [11].

However, in [12] the authors showed that ATE describes the linear relationship between variables and in some examples cannot reflect the causality. Also, Pearl’s approach to causation does not allow reasoning in cases of degrees of uncertainty [9]. Since do operators cut the relation between two nodes, Pearl’s approach cannot answer gradient questions such as: given that you smoke a little, what is the probability that you have cancer to a certain degree? To solve Pearl’s do operator problem the authors in [9,13], used fuzzy logic rules which implement human language nuances such as “small” instead of mere zero or one. Furthermore, Janzing [14] showed that at a macro level, Pearl’s causal model works well with situations that are rare, such as rare medical conditions but, at a micro level, fails with bidirectional nodes. Authors in [12] showed that Janzing’s model [15] works well with bidirectional nodes, but fails with situations that are rare [12].

In [9] which uses fuzzy logic as the fundamental part of their causal model, there are two types of rules which we call association and causal rules. An association rule can be of type A $\to$ B. A causal rule can be ~A & B. That is, tell me what happens to B when A is missing. In other words, using fuzzy logic rules, we can estimate more than fourteen values in the presence and the absence of each element (in the dataset context this becomes columns or variables).

In [9], instead of cutting the relationships between the treatment and its confounding parents, the model using fuzzification method assigns fuzzy membership values, such as very low, low, medium, and high, to the Treatment. To automatically assign membership values, the authors used fuzzy c-mean algorithm [16] which is widely used for clustering the datasets using fuzzy logic.

Once, the membership assignments step is done, the model then applies causal fuzzy interventional rules from [9] as following $max(1-\mu (a),\mu (b\left)\right)$. Doing so, it can calculates more than fourteen different possible membership degree based fuzzy counterfactual values for each variable using different causal fuzzy rules such as $max(1-\mu (a),\mu (b\left)\right)$, where $\mu \left(a\right)$ and $\mu \left(b\right)$ are the highest membership degrees of $a$ and $b$ among all considered fuzzy attributes for $a$ and $b$, respectively [9]. We would like to put more emphasis that using the following causal rules, we are applying interventions and estimate fuzzy counterfactuals at the same time.

Here are some of our fuzzy interventional rules.

$max(1-\mu (a),\mu (b\left)\right)$,

$min(1-\mu (a),\mu (b\left)\right)$,

$max(\mathrm{m}\mathrm{i}\mathrm{n}(1-\mu \left(a\right),1-\mu \left(b\right)),b)$,

We note that based on the model introduced in [9], each of the above rules has a meaning based on a subjective random selection. For instance, $min(1-\mu (b),\mu (a\left)\right)$ could be seen as the probability of subjectively selecting $A=a$ as $F_A$ and $B=b$ as $F(\neg B)$, where $F_A$ and $F_B$ are the fuzzy attributes that $\mu \left(a\right)$ and $\mu \left(b\right)$ come from, respectively. The model then applies defuzzification to the dataset in order to obtain the outputs. The big changes in the fuzzy output values can indicate possible causes.

3. Model description

Causal models can reason but cannot learn, and Deep Learning algorithms can learn but have limited capacity for reasonings [17]. Causal fuzzy rules for the first time were integrated with DLs such as Variational Autoencoders [17] and Big Bird architecture [10]- the latter outperformed ChatGpt 2 in reasoning [10]. To handle long sequences, Big Bird is equipped with a sparse attention mechanism which reduces the computational complexity of self-attention from quadratic to linear.

Causal fuzzy rules, fuzzification and defuzzification steps were integrated with Big Bird as follows: the model first creates a matrix W with randomly generated values between zero and one (Figure 1). W will represent the potential causal relationships between the columns of the dataset D [17]. To do this, the model first initialized matrix W with size (n*n) where n is the number of columns in D. It then multiplied W by the dataset. The result of this step was then fed into the Transformer’s Embedding module. The output of the Embedding module was then fed into the Transformer’s Encoder module which then partitioned using the Fuzzy c-means algorithm [16] (shown as FC-mean in Figure 1). This partitioning was done to automatically find fuzzy membership intervals without requiring an expert to define them. Using the output of the Fuzzy c-means algorithm, the architecture then used Fuzzification method (Figure 1) to fuzzify the dataset. In the next step, the Fuzzy causal rule set engine (Figure 1) applies the above causal fuzzy rules from [9] to each of the dataset D’s columns. Finally, the architecture applies Defuzzification (Figure 1).

The next step was to incorporate fuzzy rules into the Big Bird’s loss function. According to [5], this was done by dividing the Big Bird Loss Function by the variations of fuzzy causal reasoning rules obtained from the previous defuzzification step.

The result of the previous step was then multiplied by the original dataset before being fed into the Transformer’s Encoder module. These steps were repeated until the architecture converged. The elements of the matrix W, which resulted from the previous steps, are possible causes.

5. Results

In this section, we compare the Causal Fuzzy Big Bird Architecture (CFT) [10] (code on GitHub) and the GPT-2 architecture [18] in terms of features selection and accuracy. GPT-2 architecture is a powerful deep learning algorithm which is described in the following address[1]. We applied both CFT and GPT-2 to analyze the radiomics dataset. The objective was to identify the key radiomics confounders of the causal relationship of Edema and Gliosarcoma. As these confounding features can inform the distinction of GSM from GBM. Based on the analysis, the radiomics features were grouped into four categories (Please see Appendix): Group A, Group B, Group C, and Singleton D, each representing varying probabilities of being an important potential confounder of the relationship between Edema and GSM.

We added a new embedding layer of size 20K to GPT-2, a language model, so that it could learn from numerical data. We used the same embedding size for CFT. We split the dataset into 70% for training and 30% for testing. Since the dataset had only 183 rows, we augmented it by 200 folds. We trained both models for 500 epochs on a Nvidia A100 with 24 GB RAM.

Since, Big Bird and GTP-2 are both language models, we used perplexity formula [19] to to measure the accuracy of both models. As we can see, GPT-2’s perplexity score is higher than CFT. However, this cannot be considered as a good measure for causal inference.

One way to support this claim is to compare how GPT-2 and CFT performance on detecting the ‘original_shape_Sphericity’, which is the most important radiomics feature. GPT-2 fails to detect it, while CFT succeeds, even though CFT has a lower test accuracy than GPT-2.

In the following we provided the result of the CFT and GPT-2 and their comparison. Our results agree with the findings in [20], who identified the radiomics feature original_shape_Sphericity as one of the key factors informing the distinction of GSM from GBM. That is this feature is a strong potential confounder of the relationship of Edema and GSM.

Once more, we wish to underscore the methodology distinction. In prior research, including the work referenced in [20], the standard practice involved the initial application of dimension reduction techniques followed by the utilization of statistical and machine learning algorithms, such as Lasso [8], to derive their findings. In contrast, our approach for this study involved the direct utilization of the entire dataset with our architecture, without any alterations or preprocessing steps.

The Appendix shows the groups A, B, and C of variables that are most likely to contribute to the distinction of GSM from GBM by CFT and GPT-2. However, GPT-2 detected fewer variables than CFT in each group.