Multi-Armed Bandit-Based Online Model Selection for Resource-Aware Incremental Phishing Detection

1. Introduction

Phishing remains one of the most persistent and damaging cybersecurity threats, as attackers continuously exploit both social engineering and technical deception to obtain sensitive user information such as login credentials, financial details, and personal data. Phishing attacks are commonly delivered through fraudulent emails, manipulated URLs, spoofed websites, instant messages, SMS messages, and other communication channels that imitate trusted organizations. By replicating legitimate branding elements, website layouts, and communication styles, attackers make phishing content difficult for users to distinguish from authentic services. Recent phishing detection studies continue to emphasize that phishing websites and URLs remain highly effective attack vectors because they can be rapidly generated, visually disguised, and distributed at scale [1,2]. Therefore, phishing detection should be viewed as a continuously evolving classification problem rather than a static one.

Traditional phishing detection techniques, particularly blacklist-based methods, depend on databases of previously identified malicious URLs or domains. Although such methods can detect known threats efficiently, they are often ineffective against newly generated phishing websites, rapidly changing attack patterns, and adversarial techniques such as URL obfuscation, domain variation, and automated domain generation. The dynamic nature of phishing attacks makes it difficult to maintain complete and up-to-date blacklists. Consequently, machine learning-based phishing detection has received increasing attention because of its ability to learn discriminative patterns from URL, webpage, content, and structural features and classify suspicious instances as either legitimate or phishing [1,3,4]. Publicly available phishing datasets, such as the UCI phishing websites dataset, have further supported the development and evaluation of machine learning-based phishing detectors [5].

Despite the success of machine learning in phishing detection, many existing approaches rely on batch or offline learning. In batch learning, a model is trained using a fixed dataset and then deployed for future prediction. This approach has several limitations in real-world phishing environments. First, batch models require periodic retraining when new phishing patterns emerge. Second, they assume that the training and testing data follow a relatively stable distribution, which is often unrealistic in cybersecurity applications. Third, retraining can be computationally expensive and may not be suitable for real-time or resource-constrained detection systems. These limitations are particularly important because phishing attacks evolve continuously, leading to changes in feature distributions and model performance over time. This issue is closely related to concept drift, which refers to changes in the relationship between input features and target labels over time [6]. Therefore, adaptive learning is important for phishing detection systems that must operate continuously under changing stream conditions.

Incremental machine learning provides a promising alternative by allowing models to learn continuously from streaming data. Instead of retraining from scratch, incremental learners update their knowledge instance by instance or batch by batch as new data become available. This makes them suitable for real-time phishing detection, where new URLs and webpage-related observations may arrive continuously. Stream learning algorithms such as Hoeffding Trees are designed to process high-speed data streams efficiently, while adaptive methods such as Adaptive Random Forests are developed for evolving data stream classification [7,8]. In addition, modern stream learning libraries such as River provide practical tools for implementing online learning, streaming evaluation, incremental preprocessing, and adaptive model updates in Python [9]. These capabilities make incremental learning a suitable foundation for phishing detection systems that must process incoming instances continuously.

However, relying on a single incremental learning algorithm may still be limiting. Different incremental learners may behave differently depending on the stream order, feature representation, data distribution, and model update mechanism. A learner that performs well under one setting may not remain the most suitable under another setting. Selecting one model before deployment assumes that the same learner will remain preferable throughout the stream, which may not hold in dynamic cybersecurity environments. On the other hand, using a full ensemble of incremental models may improve robustness in some cases, but it can also increase computational overhead, memory consumption, and prediction latency. Ensemble learning is widely recognized as useful for data stream classification, but its computational cost can be challenging in high-speed or resource-constrained environments [10,11].

To address this challenge, this paper investigates a multi-armed bandit (MAB)-based online model selection framework for incremental phishing detection. In the proposed framework, heterogeneous incremental learning pipelines are treated as candidate arms. At each time step, the bandit policy selects one learner to classify the incoming instance. After the true label becomes available, a reward is computed based on the prediction outcome, and the policy updates its future arm-selection behavior. This allows the detection system to balance exploration of alternative learners with exploitation of learners that have shown strong recent performance. MAB methods are well suited to sequential decision-making problems in which the system must learn which action performs best over time under uncertainty [12,13]. More recently, MAB-based online model selection has been investigated for adaptive stream classification by treating candidate learners as competing arms in non-stationary environments [11].

The objective of this study is not only to determine whether MAB-based online model selection can achieve high phishing detection performance, but also to examine the trade-offs it introduces. Therefore, the proposed framework is compared with individual incremental learners and full ensemble prediction using multiple predictive metrics, including accuracy, precision, recall, F1-score, and Cohen’s kappa. In addition, the study evaluates computational behavior in terms of arm selection time, model prediction time, reward/policy update time, model update time, total processing time, and memory footprint. The analysis also considers how different update strategies, namely selected-only update, hybrid update, and all-model update, affect predictive performance and computational cost. Furthermore, the study examines sensitivity to stream order and feature representation, including the removal of RT/rule-based features.

The main contributions of this paper are summarized as follows:

• It proposes a MAB-based online model selection framework for incremental phishing detection, where heterogeneous incremental learning pipelines are treated as competing arms.
• It provides a comparative evaluation of MAB-based model selection against individual incremental learners and full ensemble prediction using accuracy, precision, recall, F1-score, and Cohen’s kappa.
• It investigates multiple MAB policies, including random selection, Epsilon-Greedy, UCB, Bayes-UCB, Thompson Sampling, EXP3, and LinUCB, to analyze their exploration–exploitation behavior in phishing detection streams.
• It studies the computational trade-offs of MAB-based model selection by measuring arm selection time, model prediction time, reward/policy update time, model update time, total processing time, and memory footprint.
• It analyzes how arm selection behavior varies across MAB policies and update strategies, and whether the selected arms reflect the relative suitability of candidate incremental learners.
• It examines the sensitivity of the framework to update strategy, stream order, and feature representation, including the removal of RT/rule-based features.

The remainder of this paper is organized as follows. Section 2 reviews related work on phishing detection, incremental machine learning, concept drift, ensemble learning, and MAB-based online model selection. Section 3 provides the necessary background on multi-armed bandits, including the exploration–exploitation trade-off and the main policies considered in this study. Section 4 presents the proposed methodology, including the stream representation, candidate arms, MAB policies, update strategies, baselines, and evaluation metrics. Section 5 discusses the experimental results, including predictive performance, computational efficiency, timing–memory trade-offs, policy behavior, arm selection patterns, and sensitivity analysis. Finally, Section 6 concludes the paper and outlines possible directions for future work.

2. Literature Review

Phishing detection has received significant research attention because phishing remains one of the most persistent cybersecurity threats affecting online services, digital communication, cloud platforms, and electronic transactions. Phishing attacks exploit both technical deception and human trust by impersonating legitimate organizations through fraudulent websites, manipulated URLs, spoofed emails, fake login pages, and visually similar interfaces. Since attackers continuously modify their strategies, phishing detection should not be treated as a purely static classification problem. Instead, effective detection systems must be able to recognize previously unseen phishing attempts, adapt to changing attack patterns, and operate under real-time constraints [14,15,16].

Existing phishing detection techniques can be broadly categorized into traditional list-based and heuristic methods, machine learning-based methods, deep learning-based methods, incremental or online learning methods, and adaptive model selection methods. The following subsections review these research directions and position the present study within the emerging area of multi-armed bandit-based online model selection for incremental phishing detection.

2.1. Traditional Phishing Detection Techniques

Traditional phishing detection techniques include list-based, visual similarity-based, and heuristic approaches. List-based methods rely mainly on whitelists and blacklists to allow or block access to specific URLs or domains. A whitelist contains legitimate websites that users are allowed to access, whereas a blacklist contains known phishing or malicious URLs. These methods are simple and efficient when the phishing website is already known. However, they are less effective against newly generated URLs, short-lived domains, and modified URL structures. Attackers can evade blacklist-based systems by frequently changing domains, applying URL obfuscation, and using automated domain generation techniques. Therefore, list-based techniques remain limited when dealing with zero-hour phishing attacks and rapidly evolving campaigns [15,17].

Visual similarity-based methods attempt to detect phishing websites by comparing the visual appearance of a suspicious website with that of a legitimate website. These methods may analyze logos, screenshots, colors, layout, HTML structure, CSS properties, and other visual identity elements. Since phishing websites often imitate trusted brands, visual similarity can provide useful evidence for identifying impersonation attempts. However, visual comparison methods may require access to reference websites, stored templates, or brand-specific information. They may also be computationally expensive when image processing or screenshot comparison is required. In addition, attackers can reduce visual similarity while still maintaining a convincing phishing page, which limits the robustness of purely visual methods [16].

Heuristic and feature-based approaches detect phishing attempts by analyzing predefined characteristics extracted from URLs, webpage content, source code, domain information, SSL certificates, DNS records, and website behavior. These features may include URL length, use of special characters, IP-address-based URLs, suspicious subdomains, redirection behavior, HTTPS usage, and domain age. Compared with blacklists, heuristic approaches are more flexible because they can detect suspicious behavior even when the URL has not been previously reported. However, their effectiveness depends strongly on the quality and stability of the selected features. As attackers continuously modify their techniques, some handcrafted features may become less effective over time [15,16,18].

2.2. Machine Learning-Based Phishing Detection

Machine learning has become one of the most widely adopted approaches for phishing detection because it can learn discriminative patterns from labeled datasets and classify URLs, websites, or emails as legitimate or phishing. In most studies, phishing detection is formulated as a binary classification problem, where features extracted from URLs, website content, webpage structure, or email text are mapped to phishing or benign labels. Traditional machine learning classifiers such as Support Vector Machines, Decision Trees, Random Forest, Naive Bayes, Logistic Regression, K-Nearest Neighbors, AdaBoost, and ensemble models have been widely investigated for this purpose [3,4,19].

Several studies have shown that ensemble-based models, particularly Random Forest and boosting-based methods, often achieve strong phishing detection performance. URL-based phishing detection studies have demonstrated that lexical, host-based, content-based, and hybrid features can significantly improve classification accuracy when combined with suitable classifiers [18,20]. Feature engineering and feature selection also play an important role in improving detection accuracy and reducing computational cost. For example, benchmark-oriented studies have emphasized the importance of reproducible datasets and comprehensive feature groups for reliable comparison across phishing detection models [18]. Recent neural-network-based empirical evaluations also show that model performance depends strongly on feature representation, data quality, and the selected classifier architecture [1,2].

Despite the strong performance reported by many machine learning-based phishing detection studies, most of these approaches rely on batch or offline learning. In batch learning, the model is trained using a fixed dataset and then deployed for future classification. This assumption is problematic in phishing detection because the data distribution can change over time as attackers introduce new URL structures, page designs, redirection techniques, domain patterns, and social engineering strategies. As a result, a classifier trained on historical phishing samples may gradually lose effectiveness when exposed to newer phishing campaigns. Periodic retraining can partially address this issue, but retraining requires updated labeled datasets, additional computational resources, and repeated deployment. Therefore, batch learning does not fully satisfy the requirements of real-time phishing detection in dynamic environments.

2.3. Deep Learning-Based Phishing Detection

Deep learning techniques have increasingly been applied to phishing detection because of their ability to automatically learn complex feature representations from raw or semi-structured data. Deep Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, Long Short-Term Memory networks, Gated Recurrent Units, autoencoders, and hybrid deep architectures have been used to analyze URLs, email text, webpage content, and transaction behavior [21,22]. These models can reduce reliance on manual feature engineering by learning character-level, token-level, semantic, or structural representations directly from the input data. For example, CNN-based models can capture local character patterns in URLs, while LSTM-based models can learn sequential dependencies in URL strings or phishing email text [22].

Although deep learning models can achieve high detection performance, they also introduce several challenges. Many deep models require large labeled datasets, extensive computational resources, and careful hyperparameter tuning. Their training process is often performed offline, which limits their ability to adapt continuously to new phishing strategies. Deep models may also be less suitable for resource-constrained or real-time environments if prediction or retraining costs are high. Consequently, while deep learning provides powerful representation-learning capabilities, additional mechanisms are needed to support continuous adaptation, efficient online updating, and dynamic model selection in evolving phishing environments [2,21].

2.4. Incremental and Online Learning for Adaptive Phishing Detection

Incremental learning, also referred to as online learning or stream learning, provides a promising alternative to batch-based phishing detection. Instead of training a model once using a fixed dataset, incremental learning updates the model continuously as new instances arrive. This makes it suitable for real-time phishing detection, where URLs and website samples may arrive sequentially and where phishing strategies may change over time. Incremental learning is also useful for handling concept drift, which occurs when the statistical relationship between input features and class labels changes over time [6,7,9]. In phishing detection, concept drift may result from changes in attacker behavior, domain registration strategies, URL obfuscation methods, website structures, or targeted brands.

Data stream learning algorithms such as Hoeffding Trees and Adaptive Random Forests have been widely studied for high-speed and evolving data streams [7,8]. Ensemble learning has also been shown to be effective for data stream classification because ensembles can improve robustness by combining multiple learners [10]. However, ensemble methods may increase memory consumption and computational cost, which is an important concern for real-time phishing detection systems.

The authors’ previous work addressed the need for adaptive phishing detection by systematically evaluating a wide range of incremental machine learning algorithms for phishing website detection [23]. The evaluated models included ensemble-based methods, forest models, linear classifiers, Naive Bayes, nearest-neighbor models, and tree-based classifiers. The study also examined the effect of different feature scaling strategies, including standard scaling, min-max scaling, and no scaling. The models were evaluated using prequential evaluation and assessed using accuracy, precision, recall, F1-score, Cohen’s kappa, and model size. The results showed that models such as Aggregated Mondrian Forest, Extremely Fast Decision Trees, and Logistic Regression achieved strong classification performance, while lightweight models such as ALMA achieved very low memory consumption. These findings highlight an important performance–memory trade-off in incremental phishing detection.

Although previous incremental learning work provides useful insight into the suitability of different online learners for phishing detection, evaluating individual models separately does not guarantee that one model will remain the best throughout the entire data stream. In dynamic phishing environments, different models may perform better at different stages of the stream. For example, a tree-based model may perform well when phishing patterns are highly nonlinear, while a linear model may be more efficient and sufficiently accurate during stable periods. Similarly, an adaptive forest model may provide strong performance but require more memory, whereas a lightweight model may be preferable in resource-constrained settings. Therefore, selecting a single fixed incremental learner before deployment may be suboptimal.

2.5. Multi-Armed Bandits for Online Model Selection

Online model selection provides a suitable solution to the limitation of relying on a single fixed incremental learner. Instead of committing to one classifier or evaluating all models for every instance, an online model selection mechanism dynamically chooses one model from a pool of candidate learners during stream processing. Multi-armed bandits provide a mathematically grounded framework for this type of sequential decision-making problem. In a multi-armed bandit formulation, each candidate model is treated as an arm. At each time step, the bandit policy selects one arm, receives a reward based on the selected model’s performance, and updates its future selection strategy. The objective is to maximize cumulative reward by balancing exploitation of models that have performed well in the past with exploration of alternative models that may perform better in the future [12,24,25].

Several MAB policies are relevant to online model selection. Random selection provides a non-adaptive baseline by selecting arms uniformly. Epsilon-greedy balances exploration and exploitation by selecting the empirically best arm most of the time while occasionally selecting a random arm. Upper Confidence Bound selects the arm with the highest optimistic reward estimate and provides finite-time regret guarantees in stochastic bandit settings [12]. Bayesian UCB extends this idea by selecting arms according to posterior reward quantiles [26]. Thompson Sampling selects arms by sampling from posterior reward distributions, naturally balancing exploration and exploitation through uncertainty [27,28]. EXP3 is designed for adversarial or non-stationary bandit environments, making it especially relevant to phishing detection, where attackers may strategically alter their behavior to evade detection [29]. Contextual bandit methods such as LinUCB further extend the MAB setting by using the current input features when selecting an arm [30].

Recent work has shown that MAB-based online model selection can be effective for concept-drift adaptation. In such frameworks, candidate learners are modeled as competing arms, and the bandit policy dynamically allocates prediction opportunities according to observed performance [11]. Non-stationary bandit studies also show that concept drift can be explicitly considered in the design and empirical evaluation of bandit policies [13]. This formulation is highly relevant to phishing detection because phishing data streams are non-stationary and may be influenced by adversarial behavior. A model that performs well against one phishing campaign may become less effective when attackers introduce new URL obfuscation patterns, modify page structures, or target different organizations. A MAB-based model selector can respond to such changes by gradually shifting selection toward models that receive higher recent rewards.

Compared with full ensemble methods, MAB-based online model selection offers a potentially more resource-efficient alternative. Full ensembles may improve robustness by combining multiple learners, but they can increase memory usage, prediction time, and update cost because several models must be maintained and evaluated. In contrast, a bandit-based selector can reduce computational overhead by selecting a single model, or a subset of models, for each incoming instance. This makes the approach suitable for real-time phishing detection systems where fast and efficient classification is required. Furthermore, contextual bandit variants can potentially select different models for different types of phishing instances, allowing instance-dependent model selection.

2.6. Research Gap and Motivation

Based on the reviewed literature, phishing detection has progressed from static list-based and heuristic systems toward machine learning, deep learning, and adaptive learning methods. These developments have improved detection accuracy and expanded the range of detectable phishing indicators. However, most existing approaches remain batch-oriented or rely on fixed model configurations. Batch learning models require periodic retraining, while fixed incremental learners assume that one selected model will remain suitable throughout the full data stream. This assumption is often unrealistic in phishing detection because attack patterns, URL structures, webpage contents, and adversarial strategies evolve over time.

Incremental learning improves adaptability by allowing models to update continuously, but most incremental phishing detection studies still evaluate learners independently or select one model for deployment. This creates a clear research gap: there is limited work on adaptive online model selection for phishing detection using multi-armed bandits. In particular, existing studies do not sufficiently investigate whether a bandit-based selector can dynamically choose among heterogeneous incremental learners to improve the balance between detection accuracy, adaptability, memory efficiency, and computational cost.

To address this gap, the present study proposes a multi-armed bandit-based online model selection framework for incremental phishing detection. The proposed framework treats candidate incremental classifiers as competing arms and dynamically selects the most suitable learner during streaming operation. By comparing multiple bandit policies against individual incremental learners and ensemble baselines, the study aims to determine whether MAB-based online model selection can provide a more adaptive and resource-efficient defense mechanism against evolving phishing attacks.

3. Background on Multi-Armed Bandits

Multi-armed bandits (MABs) provide a mathematical framework for sequential decision-making under uncertainty. In a classical MAB problem, an agent repeatedly selects one action, commonly referred to as an arm, from a finite set of candidate arms. After each selection, the agent observes a reward associated only with the selected arm. The objective is to maximize the cumulative reward over time while learning which arms are most beneficial. This creates the exploration–exploitation trade-off, where the agent must exploit arms that have produced high rewards in the past while still exploring other arms that may provide better rewards in the future [12,24,25].

Let

$$\mathcal{A}=\{1,2,\dots ,K\}$$

(1)

denote a finite set of K arms. At each time step $t\in \{1,2,\dots ,T\}$, the agent selects one arm

$$a_t\in \mathcal{A}.$$

(2)

After selecting arm $a_t$, the agent observes a reward

$$r_t=r_t\left(a_t\right),$$

(3)

where $r_t\left(a_t\right)$ is the reward obtained from the selected arm at time t.

In the stochastic MAB setting, each arm a is associated with an unknown reward distribution with expected value

$${\mu }_a=\mathbb{E}\left[r_t\left(a\right)\right].$$

(4)

The optimal arm is defined as

$$a^{*}=arg\underset{a\in \mathcal{A}}{max}{\mu }_a,$$

(5)

and the expected reward of the optimal arm is

$${\mu }^{*}=\underset{a\in \mathcal{A}}{max}{\mu }_a.$$

(6)

In the above equations, $\mathcal{A}$ is the set of available arms, K is the number of arms, t is the time index, T is the time horizon, $a_t$ is the arm selected at time t, $r_t\left(a_t\right)$ is the observed reward, ${\mu }_a$ is the expected reward of arm a, $a^{*}$ is the optimal arm, and ${\mu }^{*}$ is the maximum expected reward among all arms.

The performance of a MAB policy is commonly evaluated using regret, which measures the loss incurred by not always selecting the optimal arm. The cumulative regret over T time steps is defined as

$$R_T=T{\mu }^{*}-{\displaystyle \sum _{t=1}^T}\mathbb{E}\left[r_t\left(a_t\right)\right].$$

(7)

Equivalently, if

$${\Delta }_a={\mu }^{*}-{\mu }_a$$

(8)

denotes the optimality gap of arm a, then the expected regret can be expressed as

$$R_T=\sum _{a\in \mathcal{A}}{\Delta }_a\mathbb{E}\left[N_a\left(T\right)\right],$$

(9)

where $N_a\left(T\right)$ is the number of times arm a has been selected up to time T. A desirable MAB policy therefore aims to minimize regret by quickly identifying high-reward arms while preserving sufficient exploration of uncertain arms [12,24].

In the proposed phishing detection framework, each arm can be interpreted as an incremental machine learning model. Therefore, the bandit policy does not directly decide whether an instance is phishing or legitimate. Instead, it decides which online classifier should be used to process the current instance. This makes MABs suitable for online model selection, where the best-performing classifier may change over time due to evolving phishing strategies, data distribution changes, or concept drift.

3.1. Random Selection Policy

The random selection policy is the simplest MAB strategy. It selects each arm with equal probability, regardless of past rewards. The probability of selecting arm a is given by

$$P(a_t=a)={\displaystyle \frac{1}{K}},\forall a\in \mathcal{A}.$$

(10)

Although this policy does not learn from previous observations, it is useful as a baseline because it represents model selection without adaptive learning. In the context of online phishing detection, random selection can be used to determine whether reward-guided bandit policies provide a clear advantage over non-adaptive model selection.

3.2. Epsilon-Greedy Policy

The epsilon-greedy policy is a simple and widely used approach for balancing exploration and exploitation. It selects the empirically best arm with probability $1-ϵ$ and selects a random arm with probability $ϵ$, where $ϵ\in [0,1]$ is the exploration parameter [24,25].

The empirical mean reward of arm a after t time steps is estimated as

$${\widehat{\mu }}_a\left(t\right)={\displaystyle \frac{1}{N_a\left(t\right)}}{\displaystyle \sum _{\tau =1}^t}{r}_{\tau }\mathbb{I}(a_{\tau }=a),$$

(11)

where $N_a\left(t\right)$ is the number of times arm a has been selected up to time t, $r_{\tau }$ is the reward observed at time $\tau$, and $\mathbb{I}(·)$ is an indicator function that equals 1 when the condition is true and 0 otherwise.

The arm-selection rule is defined as

$$a_t=\left\{\begin{array}{cc}arg\underset{a\in \mathcal{A}}{max}{\widehat{\mu }}_a\left(t\right),\hfill & \mathrm{with}\mathrm{probability}1-ϵ,\hfill \\ \mathrm{Uniform}\left(\mathcal{A}\right),\hfill & \mathrm{with}\mathrm{probability}ϵ.\hfill \end{array}\right.$$

(12)

A larger value of $ϵ$ increases exploration, while a smaller value increases exploitation. In online model selection, epsilon-greedy allows the system to mostly select the currently best-performing incremental classifier while still occasionally testing other classifiers that may become more suitable as phishing patterns evolve.

3.3. Upper Confidence Bound

The Upper Confidence Bound (UCB) policy selects the arm with the highest optimistic estimate of its reward. Instead of exploring randomly, UCB assigns each arm a confidence bonus that is larger for arms selected fewer times. A common UCB selection rule is

$$a_t=arg\underset{a\in \mathcal{A}}{max}\left[{\widehat{\mu }}_a\left(t\right)+c\sqrt{{\displaystyle \frac{lnt}{N_a\left(t\right)}}}\right],$$

(13)

where ${\widehat{\mu }}_a\left(t\right)$ is the empirical mean reward of arm a, $N_a\left(t\right)$ is the number of times arm a has been selected, and $c>0$ is a parameter controlling the strength of exploration.

The term ${\widehat{\mu }}_a\left(t\right)$ encourages exploitation by favoring arms with high observed rewards, while the confidence term

$$c\sqrt{{\displaystyle \frac{lnt}{N_a\left(t\right)}}}$$

(14)

encourages exploration by assigning larger uncertainty bonuses to less frequently selected arms. UCB is theoretically important because it provides finite-time regret guarantees under stochastic reward assumptions [12]. In phishing detection, UCB is useful because it can systematically test underexplored classifiers rather than relying on purely random exploration.

3.4. Bayesian Upper Confidence Bound

Bayesian Upper Confidence Bound, or Bayes-UCB, is a Bayesian variant of the UCB principle. Instead of using only empirical averages and confidence bonuses, Bayes-UCB maintains a posterior distribution over the expected reward of each arm and selects the arm with the highest posterior quantile [26].

For binary rewards, where a correct classification may be considered a success and an incorrect classification a failure, the reward probability of arm a can be modeled using a Beta distribution:

$${\theta }_a\sim \mathrm{Beta}({\alpha }_a,{\beta }_a),$$

(15)

where ${\alpha }_a$ and ${\beta }_a$ are posterior parameters associated with the number of successes and failures observed for arm a.

At time t, Bayes-UCB computes a high posterior quantile for each arm:

$$q_a\left(t\right)=Q_a\left(1-{\displaystyle \frac{1}{t}}\right),$$

(16)

where $Q_a(·)$ is the quantile function of the posterior distribution of arm a. The selected arm is then

$$a_t=arg\underset{a\in \mathcal{A}}{max}{q}_a\left(t\right).$$

(17)

This policy is useful for online model selection because it provides a probabilistic mechanism for selecting models under uncertainty. A classifier with limited observations may still be selected if its posterior distribution suggests that it could have high reward.

3.5. Thompson Sampling

Thompson Sampling is another Bayesian MAB policy. Instead of selecting an upper confidence bound, it samples a possible reward parameter from the posterior distribution of each arm and selects the arm with the largest sampled value. Thompson Sampling has been studied both theoretically and empirically for balancing exploration and exploitation in bandit problems [27,28].

For binary rewards, the posterior distribution of arm a may be represented as

$${\theta }_a\left(t\right)\sim \mathrm{Beta}({\alpha }_a\left(t\right),{\beta }_a\left(t\right)).$$

(18)

At time t, Thompson Sampling draws one sample from each arm posterior:

$${\tilde{\theta }}_a\left(t\right)\sim \mathrm{Beta}({\alpha }_a\left(t\right),{\beta }_a\left(t\right)),\forall a\in \mathcal{A}.$$

(19)

The selected arm is then

$$a_t=arg\underset{a\in \mathcal{A}}{max}{\tilde{\theta }}_a\left(t\right).$$

(20)

After observing the reward, the posterior parameters of the selected arm are updated. For Bernoulli rewards, the update can be written as

$${\alpha }_{a_t}(t+1)={\alpha }_{a_t}\left(t\right)+r_t,$$

(21)

$${\beta }_{a_t}(t+1)={\beta }_{a_t}\left(t\right)+1-r_t.$$

(22)

Thompson Sampling is attractive because it naturally balances exploration and exploitation through posterior uncertainty. In phishing detection, this allows models with uncertain but potentially promising performance to continue receiving selection opportunities.

3.6. EXP3

The Exponential-weight algorithm for Exploration and Exploitation, commonly known as EXP3, is designed for adversarial bandit settings [29]. Unlike stochastic bandit methods, EXP3 does not assume that each arm’s reward is generated from a fixed probability distribution. This makes it suitable for environments where rewards may be non-stationary, unstable, or influenced by adversarial behavior. Such conditions are relevant to phishing detection because attackers may continuously change their strategies to evade existing detection models.

In EXP3, each arm a is assigned a positive weight $w_a\left(t\right)$ at time t. Initially, all arms are commonly assigned equal weights:

$$w_a\left(1\right)=1,\forall a\in \mathcal{A}.$$

(23)

At each time step, these weights are converted into a probability distribution over arms:

$$p_a\left(t\right)=(1-\gamma ){\displaystyle \frac{w_a\left(t\right)}{{\sum }_{j=1}^K{w}_j\left(t\right)}}+{\displaystyle \frac{\gamma }{K}},$$

(24)

where $p_a\left(t\right)$ is the probability of selecting arm a, and $\gamma \in [0,1]$ controls the amount of uniform exploration.

The arm is then sampled according to this probability distribution:

$$a_t\sim p\left(t\right).$$

(25)

Because the reward is observed only for the selected arm, EXP3 constructs an importance-weighted reward estimate:

$${\widehat{r}}_a\left(t\right)=\left\{\begin{array}{cc}{\displaystyle \frac{r_t\left(a_t\right)}{p_a\left(t\right)}},\hfill & \mathrm{if}a=a_t,\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(26)

The arm weight is then updated exponentially:

$$w_a(t+1)=w_a\left(t\right)exp\left({\displaystyle \frac{\gamma {\widehat{r}}_a\left(t\right)}{K}}\right).$$

(27)

The exponential update increases the probability of selecting arms that obtain high rewards while preserving a nonzero probability of exploring all arms. In the proposed phishing detection context, EXP3 is particularly relevant because the performance of individual incremental classifiers may degrade when attackers introduce new phishing patterns, URL obfuscation methods, or webpage manipulation techniques.

3.7. LinUCB

The policies described above are non-contextual bandit methods because their decisions are based mainly on historical rewards. In contrast, LinUCB is a contextual bandit method that uses the current input features to guide arm selection [30]. This is particularly relevant in phishing detection because each incoming sample has a feature vector describing properties of the URL, webpage, or email.

Let ${\mathbf{x}}_t\in {\mathbb{R}}^d$ denote the context vector observed at time t, where d is the number of features. In a disjoint LinUCB formulation, each arm a has its own linear reward model:

$$\mathbb{E}[r_t\left(a\right)\mid {\mathbf{x}}_t]={\mathbf{x}}_t^{\top }{\mathit{\theta }}_a,$$

(28)

where ${\mathit{\theta }}_a\in {\mathbb{R}}^d$ is the unknown parameter vector associated with arm a.

For each arm, LinUCB maintains a design matrix ${\mathbf{A}}_a$ and a response vector ${\mathbf{b}}_a$. The parameter vector is estimated as

$${\widehat{\mathit{\theta }}}_a={\mathbf{A}}_a^{-1}{\mathbf{b}}_a.$$

(29)

The arm-selection rule is then given by

$$a_t=arg\underset{a\in \mathcal{A}}{max}\left[{\mathbf{x}}_t^{\top }{\widehat{\mathit{\theta }}}_a+\alpha \sqrt{{\mathbf{x}}_t^{\top }{\mathbf{A}}_a^{-1}{\mathbf{x}}_t}\right],$$

(30)

where $\alpha >0$ controls the exploration strength.

After observing reward $r_t$, the selected arm is updated as

$${\mathbf{A}}_{a_t}\leftarrow {\mathbf{A}}_{a_t}+{\mathbf{x}}_t{\mathbf{x}}_t^{\top },$$

(31)

$${\mathbf{b}}_{a_t}\leftarrow {\mathbf{b}}_{a_t}+r_t{\mathbf{x}}_t.$$

(32)

The first term in the LinUCB selection rule estimates the expected reward of selecting a model for the current instance, while the second term adds an uncertainty bonus. Unlike non-contextual policies, LinUCB can select different models for different types of phishing samples. For example, one classifier may be preferred for URL-based patterns, while another may be preferred for webpage-content-based patterns.

3.8. Bandit-Based Online Model Selection for Phishing Detection

In the proposed online phishing detection framework, the set of arms corresponds to a pool of incremental classifiers:

$$\mathcal{M}=\{M_1,M_2,\dots ,M_K\}.$$

(33)

Each classifier $M_k$ is mapped to one arm $a_k$. At time step t, the system receives a streaming instance

$$({\mathbf{x}}_t,y_t),$$

(34)

where ${\mathbf{x}}_t$ is the feature vector and $y_t\in \{0,1\}$ is the true label. The label $y_t=1$ denotes a phishing instance, while $y_t=0$ denotes a legitimate instance.

The bandit policy ${\pi }_t$ selects one model according to the historical reward information and, in the case of contextual bandits, the current feature vector:

$$a_t={\pi }_t({\mathcal{H}}_{t-1},{\mathbf{x}}_t),$$

(35)

where ${\mathcal{H}}_{t-1}$ denotes the historical information available before time t. The selected model $M_{a_t}$ then produces the prediction

$${\widehat{y}}_t=M_{a_t}\left({\mathbf{x}}_t\right).$$

(36)

After the true label becomes available, the system computes a reward:

$$r_t=\mathcal{R}(y_t,{\widehat{y}}_t),$$

(37)

where $\mathcal{R}(·)$ is the reward function.

A simple accuracy-based reward can be defined as

$$r_t=\mathbb{I}({\widehat{y}}_t=y_t).$$

(38)

However, phishing detection is often more sensitive to false negatives because classifying a phishing instance as legitimate may expose users to severe security risks. Therefore, a cost-sensitive reward can be defined as

$$r_t=\left\{\begin{array}{cc}1,\hfill & {\widehat{y}}_t=y_t,\hfill \\ -{\lambda }_{\mathrm{FN}},\hfill & y_t=1\mathrm{and}{\widehat{y}}_t=0,\hfill \\ -{\lambda }_{\mathrm{FP}},\hfill & y_t=0\mathrm{and}{\widehat{y}}_t=1,\hfill \end{array}\right.$$

(39)

where ${\lambda }_{\mathrm{FN}}$ and ${\lambda }_{\mathrm{FP}}$ are penalty parameters for false negatives and false positives, respectively. Since false negatives are generally more harmful in phishing detection, ${\lambda }_{\mathrm{FN}}$ is typically set greater than ${\lambda }_{\mathrm{FP}}$.

A resource-aware reward may also be used:

$$r_t=\alpha \mathbb{I}({\widehat{y}}_t=y_t)-\beta C_t,$$

(40)

where $C_t$ represents the computational cost of using the selected model, such as prediction time, update time, or memory usage. The parameters $\alpha$ and $\beta$ control the relative importance of classification correctness and computational efficiency.

After reward computation, the selected model is updated incrementally:

$$M_{a_t}\leftarrow \mathrm{Update}(M_{a_t},{\mathbf{x}}_t,y_t).$$

(41)

This process allows the system to adaptively allocate prediction opportunities to competing incremental classifiers. Instead of relying on a single fixed model or evaluating a full ensemble at every time step, the MAB framework dynamically selects models according to their observed performance. This is particularly suitable for phishing detection, where the most effective model may change over time due to evolving attack behavior and concept drift.

4. Methodology

This section presents the methodology used to evaluate the proposed multi-armed bandit (MAB)-based online model selection framework for incremental phishing website detection. The framework is designed to study whether adaptive model selection can provide a useful balance between phishing detection performance and computational efficiency in a streaming environment. Instead of deploying a fixed incremental learner or evaluating all learners through a full ensemble at every time step, the proposed approach allows a bandit policy to select one candidate learner dynamically based on the feedback collected during stream processing.

The methodology is organized around five main components. First, the phishing dataset is represented as an online data stream and evaluated using a prequential test-then-train protocol. Second, a compact set of candidate model–scaling pipelines is selected to serve as MAB arms. Third, several MAB policies are evaluated to compare different exploration–exploitation strategies. Fourth, three model update strategies are studied: selected-only update, hybrid update, and all-model update. Finally, the framework is evaluated using predictive performance metrics, timing measurements, memory footprint estimates, and arm selection behavior. This design directly supports the research questions related to predictive performance, policy behavior, computational trade-offs, arm selection, and sensitivity to stream order and feature representation.

4.1. Overview of the Proposed Framework

The proposed framework treats phishing website detection as an online binary classification problem. At each time step, a new instance arrives from the stream. The MAB policy selects one candidate incremental learning pipeline from a predefined pool of arms. The selected pipeline predicts whether the current instance is phishing or legitimate. Once the true label becomes available, a reward is computed and used to update the MAB policy. The incremental learners are then updated according to the update strategy being evaluated.

Figure 1 illustrates the overall workflow of the proposed MAB-based online model selection framework. The framework receives phishing instances sequentially, applies the selected feature representation and scaling configuration, selects one candidate incremental learner using a MAB policy, computes a reward after observing the true label, and updates both the policy and the learning models according to the selected update strategy.

Let the incoming data stream be denoted as

$$\mathcal{S}=({\mathbf{x}}_1,y_1),({\mathbf{x}}_2,y_2),\dots ,({\mathbf{x}}_T,y_T),$$

(42)

where $\mathcal{S}$ is the phishing data stream, T is the total number of instances, ${\mathbf{x}}_t\in {\mathbb{R}}^d$ is the feature vector of the instance arriving at time t, d is the number of input features, and $y_t\in 0,1$ is the true class label. In this study, $y_t=1$ denotes a phishing instance, while $y_t=0$ denotes a legitimate instance.

The set of candidate arms is defined as

$$\mathcal{A}=A_1,A_2,\dots ,A_K,$$

(43)

where $\mathcal{A}$ is the set of available arms, $A_k$ is the kth arm, and K is the number of arms. Each arm corresponds to a complete incremental learning pipeline composed of a feature scaling configuration and an incremental classifier:

$$A_k=(s_k,M_k),$$

(44)

where $s_k$ is the scaling method associated with arm $A_k$, and $M_k$ is the corresponding incremental classifier.

At time step t, the MAB policy ${\pi }_t$ selects one arm using the information collected from previous predictions and rewards:

$$a_t={\pi }_t(\mathcal{H}\ast t-1),$$

(45)

where $a_t\in 1,2,\dots ,K$ is the index of the selected arm, and $\mathcal{H}\ast t-1$ denotes the historical information available before processing instance t. This history includes previously selected arms, observed rewards, and policy statistics.

The selected arm applies its scaling method to the incoming feature vector:

$${\tilde{\mathbf{x}}}_t^{\left(a_t\right)}=s_{a_t}\left({\mathbf{x}}_t\right),$$

(46)

where ${\tilde{\mathbf{x}}}_t^{\left(a_t\right)}$ is the transformed feature vector produced by the scaler of the selected arm. The selected classifier then predicts the class label:

$${\widehat{y}}_t=M_{a_t}\left({\tilde{\mathbf{x}}}_t^{\left(a_t\right)}\right),$$

(47)

where ${\widehat{y}}_t$ is the predicted label.

After observing the true label $y_t$, the framework computes a reward and updates the MAB policy. The learning models are then updated according to one of the update strategies described in Section 4.8. This separation between policy update and model update is important because the results explicitly compare how different update strategies affect predictive performance, timing cost, memory footprint, and arm selection behavior.

4.2. Dataset Description

The experiments are conducted using a phishing website dataset containing 10,000 instances [31,32]. Each instance represents a website described by 48 input features and one binary target label. The target variable indicates whether the instance is phishing or legitimate, where the phishing class is encoded as 1 and the legitimate class is encoded as 0. The dataset is balanced, with 5,000 phishing instances and 5,000 legitimate instances. This balanced distribution makes accuracy meaningful; however, accuracy alone is not sufficient for evaluating phishing detection. Therefore, precision, recall, F1-score, and Cohen’s kappa are also reported to provide a more complete assessment of detection behavior.

The feature set captures several aspects of a website and its URL structure. These include lexical URL characteristics, hostname and path information, hyperlink and external resource indicators, form-related attributes, webpage behavior indicators, and rule-threshold features. Table 1 summarizes the main feature groups and provides representative examples. This grouping is useful because different incremental learners may respond differently to different types of phishing-related evidence.

Before conducting the online-learning experiments, the dataset is inspected for duplicate and conflicting patterns. The dataset contains 419 duplicate rows, but no conflicting feature vectors are observed. In other words, identical feature vectors do not appear with different class labels. This is important because conflicting duplicate patterns could introduce ambiguity into the learning process and make performance interpretation less reliable.

The original ordering of the dataset is also inspected. The first half of the dataset contains phishing instances, while the second half contains legitimate instances. This class-blocked ordering is important because prequential online evaluation is sensitive to the order in which samples arrive. Therefore, this characteristic is explicitly considered when generating the stream-order settings described in Section 4.3.

4.3. Stream Generation and Feature Settings

The phishing detection dataset is processed as a data stream to simulate the real-time arrival of phishing and legitimate website instances. Each instance is processed sequentially using a prequential test-then-train protocol, where prediction is performed before the model learns from the current instance. This setting reflects online deployment conditions, where the detector must make a decision before the true label is used for updating.

To evaluate the sensitivity of the proposed framework to stream ordering, three stream-order settings are considered:

• Original order: the dataset is processed in its original class-blocked sequence.
• Random shuffle: the dataset is randomly shuffled before stream processing.
• Stratified shuffle: the dataset is shuffled while preserving class balance across the stream.

The original order is retained to show the behavior of the models on the dataset as provided. However, because the original sequence contains large class-homogeneous blocks, it may produce optimistic online-learning results. The random-shuffle and stratified-shuffle settings provide more realistic and conservative evaluations because phishing and legitimate instances are interleaved throughout the stream.

To evaluate the sensitivity of the framework to feature representation, two feature settings are also considered:

• Full features: all available phishing-related features are used.
• Without RT/rule features: RT and rule-based features are removed before stream processing.

The RT/rule-based features may encode expert-designed threshold behavior and can therefore be highly informative. Removing them allows the study to determine whether the proposed framework depends mainly on these rule-threshold indicators or whether the remaining lexical, host-based, hyperlink-related, form-related, and webpage-structure features still provide sufficient discriminative information.

Together, the stream-order and feature-representation settings allow the study to evaluate the proposed framework under both favorable and more realistic online conditions. They also support the sensitivity analysis reported in the results section, where the effects of stream order and RT/rule-based feature removal are examined separately.

4.4. Feature Scaling Configurations

The candidate arms are constructed as model–scaling pipelines. This is necessary because different incremental classifiers may respond differently to feature scaling. Three scaling configurations are considered when selecting candidate pipelines: no scaling, min–max scaling, and standard scaling.

For min–max scaling, the scaled value of a feature x is computed as

$$x_{\mathrm{scaled}}={\displaystyle \frac{x-x_{min}}{x_{max}-x_{min}}},$$

(48)

where x is the original feature value, $x_{\mathrm{scaled}}$ is the scaled feature value, $x_{min}$ is the running minimum value of the feature, and $x_{max}$ is the running maximum value of the feature.

For standard scaling, the scaled value is computed as

$$x_{\mathrm{scaled}}={\displaystyle \frac{x-\mu }{\sigma }},$$

(49)

where $\mu$ is the running mean of the feature and $\sigma$ is the running standard deviation.

In the no-scaling setting, the original feature vector is passed directly to the classifier:

$${\tilde{\mathbf{x}}}_t={\mathbf{x}}_t,$$

(50)

where ${\mathbf{x}}_t$ is the original feature vector and ${\tilde{\mathbf{x}}}_t$ is the feature vector used by the classifier.

4.5. Selection of Candidate Arms

The arm pool is designed to be compact but diverse. Instead of using every evaluated incremental model as a candidate arm, the strongest representative model-scaling pipeline from each model family is selected, guided by the results presented in [23]. This reduces unnecessary computational overhead while preserving diversity across model types.

The selected arms represent linear, probabilistic, neighbor-based, tree-based, forest-based, and ensemble-based incremental learning families. The selection is guided by balanced predictive performance, including accuracy, precision, recall, F1-score, and Cohen’s kappa. Recall is particularly important because false negatives correspond to phishing websites incorrectly classified as legitimate. When models have comparable predictive performance, computational efficiency and memory footprint are considered as secondary criteria.

The selected arms are summarized in Table 2.

The arm set is therefore defined as

$$\mathcal{A}=A_{\mathrm{LR}},A_{\mathrm{NB}},A_{\mathrm{KNN}},A_{\mathrm{EFDT}},A_{\mathrm{AMF}},A_{\mathrm{ADB}},$$

(51)

where $A_{\mathrm{LR}}$ denotes the Logistic Regression arm, $A_{\mathrm{NB}}$ denotes the Gaussian Naive Bayes arm, $A_{\mathrm{KNN}}$ denotes the KNN arm, $A_{\mathrm{EFDT}}$ denotes the Extremely Fast Decision Tree arm, $A_{\mathrm{AMF}}$ denotes the Aggregated Mondrian Forest arm, and $A_{\mathrm{ADB}}$ denotes the ADWIN Bagging arm.

This selection allows the MAB policy to choose among heterogeneous learners with different strengths. For example, linear and probabilistic models may offer low computational cost, whereas forest and ensemble models may offer stronger predictive performance at higher resource cost. The MAB setting is therefore suitable for studying whether adaptive selection can provide a better operating point than relying on a fixed individual model or a full ensemble.

4.6. Multi-Armed Bandit Policies

The proposed framework evaluates several MAB policies for online model selection. Each policy uses observed rewards to decide which arm should be selected at each time step. The evaluated policies include Random selection, Epsilon-Greedy, Upper Confidence Bound (UCB), Bayesian UCB, Thompson Sampling, EXP3, and LinUCB. These policies are included to compare simple random selection, classical exploration–exploitation strategies, Bayesian strategies, adversarial bandit behavior, and contextual selection.

The Random policy selects arms uniformly:

$$P(a_t=k)={\displaystyle \frac{1}{K}},$$

(52)

where $P(a_t=k)$ is the probability of selecting arm k at time t.

The Epsilon-Greedy policy selects the empirically best arm with probability $1-ϵ$ and explores a random arm with probability $ϵ$:

$$a_t=\left\{\begin{array}{cc}arg\underset{k\in 1,\dots ,K}{max}{\widehat{\mu }}_k\left(t\right)\hfill & \mathrm{with}\mathrm{probability}1-ϵ\hfill \\ \mathrm{Uniform}\left(1,\dots ,K\right),\hfill & \mathrm{with}\mathrm{probability}ϵ,\hfill \end{array}\right.$$

(53)

where $ϵ$ is the exploration probability, and ${\widehat{\mu }}_k\left(t\right)$ is the empirical mean reward of arm k.

The empirical mean reward is computed as

$${\widehat{\mu }}_k\left(t\right)={\displaystyle \frac{1}{N_k\left(t\right)}}{\displaystyle \sum _{\tau =1}^t}{r}_{\tau }\mathbb{I}(a_{\tau }=k),$$

(54)

where $N_k\left(t\right)$ is the number of times arm k has been selected up to time t, $r_{\tau }$ is the reward observed at time $\tau$, and $\mathbb{I}(·)$ is an indicator function.

The UCB policy selects the arm with the highest optimistic estimate:

$$a_t=arg\underset{k\in 1,\dots ,K}{max}\left[{\widehat{\mu }}_k\left(t\right)+c\sqrt{{\displaystyle \frac{lnt}{N_k\left(t\right)}}}\right],$$

(55)

where c controls the strength of the confidence bonus.

Bayesian UCB selects the arm with the highest posterior reward quantile:

$$a_t=arg\underset{k\in 1,\dots ,K}{max}{Q}_k\left(1-{\displaystyle \frac{1}{t}}\right),$$

(56)

where $Q_k(·)$ is the posterior quantile function of arm k.

Thompson Sampling samples one reward parameter from the posterior distribution of each arm and selects the arm with the largest sampled value:

$${\tilde{\theta }}_k\left(t\right)\sim P_k(\theta \mid \mathcal{D}\ast k\left(t\right)),$$

(57)

$$a_t=argmax\ast k\in 1,\dots ,K{\tilde{\theta }}_k\left(t\right),$$

(58)

where ${\tilde{\theta }}_k\left(t\right)$ is the sampled reward parameter of arm k, $P_k(\theta \mid {\mathcal{D}}_k\left(t\right))$ is the posterior distribution of arm k, and ${\mathcal{D}}_k\left(t\right)$ is the reward history of arm k.

EXP3 is included to represent an adversarial-style exploration strategy. It assigns a weight $w_k\left(t\right)$ to each arm and selects arms according to

$$p_k\left(t\right)=(1-\gamma ){\displaystyle \frac{w_k\left(t\right)}{{\sum }_{j=1}^K{w}_j\left(t\right)}}+{\displaystyle \frac{\gamma }{K}},$$

(59)

where $p_k\left(t\right)$ is the probability of selecting arm k, $w_k\left(t\right)$ is the current weight of arm k, and $\gamma$ is the exploration parameter.

After observing the reward $r_t$, the importance-weighted reward estimate is computed as

$${\widehat{r}}_k\left(t\right)=\left\{\begin{array}{cc}{\displaystyle \frac{r_t}{p_k\left(t\right)}},\hfill & \mathrm{if}k=a_t,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(60)

where ${\widehat{r}}_k\left(t\right)$ is the estimated reward of arm k. The arm weight is then updated as

$$w_k(t+1)=w_k\left(t\right)exp\left({\displaystyle \frac{\gamma {\widehat{r}}_k\left(t\right)}{K}}\right).$$

(61)

LinUCB is evaluated as a contextual bandit policy because it can use the current feature vector during arm selection. For each arm k, LinUCB assumes a linear reward model:

$$\mathbb{E}[r_t\left(k\right)\mid {\mathbf{x}}_t]={\mathbf{x}}_t^{\top }{\mathit{\theta }}_k,$$

(62)

where $\mathit{\theta }\ast k$ is the unknown parameter vector associated with arm k. The selected arm is

$$a_t=argmax\ast k\in 1,\dots ,K\left[{\mathbf{x}}_t^{\top }{\widehat{\mathit{\theta }}}_k+\alpha \sqrt{{\mathbf{x}}_t^{\top }{\mathbf{A}}_k^{-1}{\mathbf{x}}_t}\right],$$

(63)

where ${\widehat{\mathit{\theta }}}_k$ is the estimated parameter vector of arm k, ${\mathbf{A}}_k$ is the design matrix of arm k, and $\alpha$ controls the exploration strength.

4.7. Reward Function

The primary reward used in this study is based on prediction correctness:

$$r_t=\mathbb{I}({\widehat{y}}_t=y_t),$$

(64)

where $r_t$ is the reward at time t, ${\widehat{y}}_t$ is the predicted label, $y_t$ is the true label, and $\mathbb{I}(·)$ is an indicator function equal to 1 when the prediction is correct and 0 otherwise.

This reward directly links the MAB policy to online classification performance. It also keeps the reward bounded in $[0,1]$, which is suitable for the evaluated bandit policies. Computational efficiency is not embedded directly into the primary reward. Instead, timing and memory are measured separately and analyzed as part of the computational trade-off evaluation. This separation makes it possible to distinguish predictive performance from resource consumption in the results.

4.8. Model Update Strategies

The update strategy determines which candidate learners are trained after each incoming instance. This is a central design choice because it directly affects predictive performance, timing cost, memory footprint, and arm selection behavior. Three update strategies are evaluated.

4.8.1. Selected-only update

In selected-only update, only the arm chosen by the MAB policy is updated:

$$M_{a_t}\leftarrow \mathrm{Update}(M_{a_t},{\tilde{\mathbf{x}}}_t^{\left(a_t\right)},y_t).$$

(65)

This strategy has the lowest computational cost because only one model is trained at each time step. However, arms that are selected less frequently may receive fewer updates and may become under-trained as the stream progresses.

4.8.2. All-model update

In all-model update, every candidate arm is updated after each instance:

$$M_k\leftarrow \mathrm{Update}(M_k,{\tilde{\mathbf{x}}}_t^{\left(k\right)},y_t),\forall k\in 1,2,\dots ,K.$$

(66)

This strategy keeps all candidate learners synchronized with the stream and allows the MAB policy to compare more fully trained arms. However, it increases model update time and memory growth because all arms are trained continuously.

4.8.3. Hybrid update

The hybrid strategy provides an intermediate option. The selected arm is always updated, while each non-selected arm is updated with probability $p_h$:

$$M_k\leftarrow \mathrm{Update}(M_k,{\tilde{\mathbf{x}}}_t^{\left(k\right)},y_t),k=a_t,$$

(67)

and for $k\ne a_t$,

$$M_k\leftarrow \mathrm{Update}(M_k,{\tilde{\mathbf{x}}}_t^{\left(k\right)},y_t)\mathrm{with}\mathrm{probability}{p}_h.$$

(68)

This strategy is designed to reduce the under-training problem of selected-only update while avoiding the full cost of all-model update. It is therefore expected to provide a practical balance between predictive reliability and computational efficiency.

4.9. Prequential Evaluation Procedure

The proposed framework is evaluated using the prequential, or interleaved test-then-train, protocol. This protocol is suitable for online learning because each instance is first used for testing and then used for model updating. This ensures that the model is always evaluated before learning from the current instance.

For each incoming instance $({\mathbf{x}}_t,y_t)$, the procedure is as follows:

1.

The MAB policy selects an arm $A_{a_t}$.

2.

The selected scaler $s_{a_t}$ transforms the feature vector.

3.

The selected classifier $M_{a_t}$ predicts the label ${\widehat{y}}_t$.

4.

The prediction is compared with the true label $y_t$.

5.

The reward $r_t$ is computed.

6.

The MAB policy is updated using $(a_t,r_t)$.

7.

The candidate learners are updated according to the selected update strategy.

8.

Classification metrics, timing metrics, memory estimates, and arm selection records are updated.

This procedure reflects real-time deployment, where the detector must make a prediction before receiving feedback from the current instance.

4.10. Baseline Methods

The proposed MAB framework is compared against two baseline groups: individual incremental learners and a full ensemble baseline.

First, each selected candidate arm is evaluated individually as a standalone incremental learning pipeline. This baseline shows how each representative learner performs when deployed independently, without adaptive model selection. It also provides a reference for determining whether MAB-based selection offers advantages over using a single fixed learner throughout the stream.

Second, the best-performing individual learner is identified based on the evaluation metrics reported in the experiments. This baseline represents the conventional approach of selecting one model before deployment and using it throughout the stream. Comparing the MAB framework with this baseline is important because adaptive selection should be evaluated not only against all individual learners, but also against the strongest standalone learner observed under the same experimental conditions.

Third, a full ensemble baseline is evaluated by combining the predictions of all selected candidate learners. This baseline represents a model-combination strategy in which all candidate learners participate in prediction. Although such an approach may improve robustness in some settings, it can also increase computational cost because multiple models must be maintained and evaluated during stream processing.

Together, these baselines allow the study to assess whether MAB-based online model selection provides a useful balance between predictive performance, adaptability, and computational efficiency.

4.11. Evaluation Metrics

The framework is evaluated using predictive performance metrics, computational efficiency metrics, and MAB behavior metrics.

The predictive metrics are accuracy, precision, recall, F1-score, and Cohen’s kappa. Accuracy is defined as

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}},$$

(69)

where $TP$ is the number of phishing instances correctly classified as phishing, $TN$ is the number of legitimate instances correctly classified as legitimate, $FP$ is the number of legitimate instances incorrectly classified as phishing, and $FN$ is the number of phishing instances incorrectly classified as legitimate.

Precision is defined as

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}},$$

(70)

and recall is defined as

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}.$$

(71)

Recall is especially important in phishing detection because false negatives represent missed phishing websites.

The F1-score is defined as

$$F_1={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}},$$

(72)

and Cohen’s kappa is defined as

$$\kappa ={\displaystyle \frac{p_o-p_e}{1-p_e}},$$

(73)

where $p_o$ is the observed agreement between predicted and true labels, and $p_e$ is the expected agreement by chance.

The computational metrics include:

• Arm selection time: the time required by the MAB policy to select an arm.
• Model prediction time: the time required by the selected model to produce a prediction.
• Reward/policy update time: the time required to compute the reward and update the MAB policy.
• Model update time: the time required to update one or more incremental models according to the update strategy.
• Total processing time: the total per-instance processing time, including selection, prediction, reward/policy update, and model update operations.
• Memory footprint: the estimated memory footprint of the individual model, full ensemble, or MAB framework.

The MAB behavior metrics include the arm selection count and arm selection percentage for each candidate learner. These metrics are used to analyze whether the MAB policies learn meaningful preferences over the candidate arms and whether the selection behavior reflects the relative suitability of the incremental learners.

4.12. Algorithmic Description

Algorithm 1 summarizes the proposed MAB-based online model selection framework.

Algorithm 1 MAB-Based Online Model Selection for Incremental Phishing Detection

Require:  Data stream $\mathcal{S}$, selected arms $\mathcal{A}$, MAB policy $\pi$, reward function $\mathcal{R}$, update strategy U Ensure:  Predictive metrics, timing metrics, memory estimates, and arm selection history 1: Initialize selected model–scaling arms $A_k=(s_k,M_k)$ for $k=1,\dots ,K$ 2: Initialize MAB policy statistics 3: Initialize classification, timing, memory, and selection tracking metrics 4: for each incoming instance $(\mathbf{x}\ast t,y_t)\in \mathcal{S}$ do 5:     Select arm $a_t\leftarrow {\pi }_t(\mathcal{H}\ast t-1)$ 6:     Apply selected scaler $\tilde{\mathbf{x}}\ast t^{\left(a_t\right)}\leftarrow s\ast a_t\left({\mathbf{x}}_t\right)$ 7:     Predict class label $\widehat{y}\ast t\leftarrow M\ast a_t\left({\tilde{\mathbf{x}}}_t^{\left(a_t\right)}\right)$ 8:     Compute reward $r_t\leftarrow \mathcal{R}(y_t,{\widehat{y}}_t)$ 9:     Update MAB policy using $(a_t,r_t)$ 10:    Update candidate learners according to update strategy U 11:    Update predictive metrics, timing metrics, memory estimates, and arm selection counts 12: end for 13: return Final metrics and arm selection history

4.13. Experimental Design and Research Questions

The experimental design consists of four phases. In the first phase, the phishing dataset is prepared under different stream-order and feature-representation settings. In the second phase, the selected model–scaling pipelines are evaluated individually and as a full ensemble baseline. In the third phase, the MAB framework is evaluated using different bandit policies and update strategies. In the fourth phase, the results are analyzed in terms of predictive performance, timing efficiency, memory footprint, and arm selection behavior.

The experiments are designed to answer the following research questions:

• RQ1: How does MAB-based online model selection compare with individual incremental learners and full ensemble prediction in terms of accuracy, precision, recall, F1-score, and Cohen’s kappa?
• RQ2: Which MAB policy provides the most effective exploration–exploitation behavior for phishing detection streams?
• RQ3: What are the computational trade-offs of MAB-based model selection compared with individual learners and full ensemble prediction in terms of prediction time, model update time, total processing time, and memory footprint?
• RQ4: How does the arm selection behavior vary across MAB policies and update strategies, and does it reflect the relative suitability of candidate incremental learners?
• RQ5: How sensitive is the proposed framework to update strategy, stream order, and feature representation, including the removal of RT/rule-based features?

This experimental design ensures that the proposed framework is evaluated not only as a classifier, but also as a resource-aware online model selection mechanism. The results are therefore presented using classification tables, timing decomposition, memory analysis, trade-off plots, policy comparisons, and arm selection heatmaps.

5. Results and Discussion

This section presents a comprehensive evaluation of the proposed MAB-based online model selection framework for incremental phishing website detection. The discussion is organized progressively to make the analysis easier to follow. First, the main detection approaches are compared under realistic shuffled-stream settings. This comparison includes the strongest standalone incremental learner, the full ensemble, and the three MAB update strategies: selected-only, hybrid, and all-model update. Second, the sensitivity of the results to stream order and RT/rule-based feature removal is examined. Third, the computational behavior of the framework is analyzed in terms of timing and memory footprint. Finally, the MAB policy behavior and arm selection distributions are examined to explain how different exploration–exploitation strategies adapt their model choices over time.

The evaluation does not rely on a single metric. Accuracy, precision, recall, F1-score, and Cohen’s kappa are reported together because each metric captures a different aspect of phishing detection. Accuracy is meaningful because the dataset is balanced, but it should not be interpreted alone. Recall is particularly important because a false negative means that a phishing website is classified as legitimate. Precision reflects false-alarm behavior, while F1-score summarizes the balance between precision and recall. Cohen’s kappa is also included because it measures agreement beyond chance and provides a more conservative indication of predictive reliability.

The experiments were conducted under three stream-order settings: the original order, a randomly shuffled stream, and a stratified shuffled stream. Two feature settings were considered: the full feature set and a reduced feature set in which RT/rule-based features were removed. Three MAB update strategies were evaluated. In selected-only update, only the arm selected by the MAB policy is updated after each instance. In all-model update, all candidate arms are updated after every instance. The hybrid strategy updates the selected arm at every instance while occasionally updating non-selected arms. Unless otherwise stated, the main comparative analysis focuses on the realistic shuffled-stream settings, since the original stream order produces optimistic results due to its class-blocked structure.

5.1. Predictive Performance under Realistic Shuffled Streams

The original stream order produced very high predictive scores, but these results are optimistic because online learning is sensitive to the order in which samples arrive. Therefore, the main performance discussion focuses on the randomly shuffled and stratified shuffled settings, which provide a more realistic view of online phishing detection.

Table 3 compares AMF, the full ensemble, and the three MAB update strategies under the realistic shuffled settings. AMF is included because it is the strongest standalone incremental learner observed in the experiments. The full ensemble is included to represent a model-combination baseline in which all candidate learners participate in prediction. The MAB rows summarize the performance obtained under the selected-only, hybrid, and all-model update strategies. The detailed policy-level behavior that leads to these results is analyzed later in the MAB policy and arm-selection subsections.

The results show that AMF achieves the strongest standalone performance across the realistic shuffled settings. It maintains high accuracy, recall, F1-score, and Cohen’s kappa while requiring substantially less processing time than the full ensemble. For example, under the full-feature random-shuffle setting, AMF achieves 0.9545 accuracy, 0.9510 recall, 0.9543 F1-score, and 0.9090 kappa. A similar pattern is observed under stratified shuffle and after removing RT/rule-based features.

The full ensemble does not provide a clear advantage over AMF. Although it evaluates all candidate learners, its predictive performance is lower than AMF in all reported shuffled settings. This indicates that majority voting can be affected by weaker learners in the model pool. In other words, using more models does not automatically produce better phishing detection decisions.

The MAB-based results show a more nuanced pattern. Selected-only MAB is computationally attractive, but its predictive performance is generally less stable because non-selected arms may receive fewer updates and become under-trained. All-model MAB achieves stronger MAB predictive performance because all candidate arms remain trained throughout the stream, but this comes with higher computational cost. Hybrid MAB provides an intermediate behavior by improving over selected-only MAB while avoiding much of the cost of all-model update.

Figure 2 provides a multi-metric comparison of the main evaluated approaches under the realistic shuffled-stream settings. The figure shows that the performance differences among the methods are not limited to F1-score, but are also reflected in accuracy, precision, recall, and Cohen’s kappa. AMF achieves the strongest overall standalone performance, with consistently high values across accuracy, precision, recall, and F1-score, while also producing the highest Cohen’s kappa. This indicates that AMF is not only accurate, but also provides more reliable agreement with the true labels beyond chance.

The full ensemble does not provide a clear advantage over the strongest standalone learner. Although it combines all candidate incremental models, its average accuracy, F1-score, and kappa are lower than those of AMF. This suggests that majority voting may be affected by weaker learners in the model pool, especially when some models are less suitable for the current stream conditions. Therefore, simply combining all models does not necessarily improve phishing detection performance.

The MAB update strategies provide different predictive behaviors. All-model MAB achieves the strongest average MAB performance and is closest to AMF across the reported metrics. This behavior is expected because all candidate arms are continuously updated, allowing the bandit policy to compare well-trained learners during arm selection. Hybrid MAB performs between selected-only and all-model MAB, showing that occasional updates of non-selected arms improve stability compared with selected-only update. Selected-only MAB is generally the weakest MAB update strategy in this figure, which indicates that updating only the selected arm may cause some candidate learners to become under-trained and less competitive over time.

The kappa values provide an important additional insight. While several methods achieve relatively close accuracy, precision, recall, and F1-score values, the differences in Cohen’s kappa are more visible. This means that kappa is more sensitive to the reliability of the predictions across the balanced classes. From this perspective, AMF and all-model MAB provide the most reliable predictive behavior, while selected-only MAB and the full ensemble show lower agreement beyond chance. Overall, the figure supports the conclusion that MAB-based selection should be evaluated as a trade-off mechanism: all-model update improves predictive reliability, hybrid update provides a balanced configuration, and selected-only update favors efficiency at the cost of reduced predictive stability.

5.2. Effect of Stream Order

Stream order is important in prequential evaluation because the model is evaluated before being updated on each incoming instance. Figure 3 illustrates the sensitivity of the MAB framework to stream order under both feature settings. The figure compares the original stream order with random and stratified shuffled streams.

The most visible pattern is the large performance gap between the original stream order and the shuffled streams. Under the original order, the MAB framework achieves near-perfect accuracy, recall, F1-score, and Cohen’s kappa for both the full-feature and no-RT/rule feature settings. In contrast, the random and stratified shuffled streams produce lower but more realistic performance values, especially in terms of Cohen’s kappa.

This behavior indicates that the original stream order creates a more favorable online-learning condition. Since the dataset is class-blocked in its original ordering, with phishing and legitimate samples appearing in large consecutive segments, the learner may benefit from a simplified temporal structure. Under such a setting, the model can adapt to a relatively homogeneous portion of the stream before encountering the next class segment. This can inflate prequential performance because the stream does not fully reflect the mixed and continuously varying arrival pattern expected in practical phishing detection scenarios.

The random and stratified shuffled settings provide a more demanding evaluation because phishing and legitimate instances are interleaved throughout the stream. This makes the online learning task less dependent on class-block structure and gives a clearer indication of how the MAB framework behaves when it must continuously distinguish between both classes. Although the scores decrease compared with the original stream order, the MAB-based approach still maintains strong performance under these realistic settings, with accuracy, recall, and F1-score remaining around the mid-0.95 range in several cases. This indicates that MAB-based model selection remains effective even when the stream is more mixed and less favorable than the original ordering.

The reduction in kappa under the shuffled settings is also informative. While the original stream produces near-perfect agreement, the shuffled streams provide a more conservative estimate of reliability. Nevertheless, the kappa values remain high enough to show that the MAB framework continues to produce meaningful predictive agreement beyond chance. Therefore, the figure does not suggest that MAB performance collapses under realistic stream conditions; rather, it shows that the original order is optimistic, while the shuffled settings provide a more credible assessment of strong but more realistic MAB behavior.

The figure also shows that removing RT/rule-based features does not substantially change the overall stream-order pattern. Both the full-feature and no-RT/rule settings achieve very high performance under the original order and lower, more conservative performance under shuffled streams. This suggests that stream order has a stronger effect on online evaluation than the presence or absence of RT/rule-based features. Overall, Figure 3 supports the decision to emphasize random and stratified shuffled streams in the main analysis, since they provide a more reliable assessment of MAB-based online phishing detection while still showing that the proposed technique performs well under realistic stream conditions.

5.3. Effect of Removing RT/Rule-Based Features

The RT/rule-based features are important to examine because they may already encode expert-designed phishing indicators. If the models depended mainly on these rule-threshold attributes, removing them would cause a sharp decline in performance. The results show that this is not the case.

Figure 4 and Figure 5 present the effect of removing RT/rule-based features on recall and Cohen’s kappa. Recall is emphasized because it measures the ability to detect phishing websites, while kappa reflects predictive reliability beyond chance. The comparison is evaluated under realistic shuffled-stream settings to avoid drawing conclusions from the optimistic original stream order.

The ablation results show that removing RT/rule-based features does not collapse performance. In several shuffled settings, the reduced-feature configuration remains close to the full-feature configuration. This suggests that the remaining lexical, host-based, hyperlink, and webpage-structure features contain strong discriminative information. Therefore, the high predictive performance is not solely explained by rule-threshold indicators.

5.4. Timing and Computational Efficiency Analysis

Timing is central to this study because the proposed framework targets online phishing detection. In a streaming setting, a model must not only make accurate predictions but also process each instance quickly. Table 4 reports the average timing decomposition of the three MAB update strategies under realistic shuffled streams.

The most important insight is that the MAB policy itself is not the computational bottleneck. Arm selection and reward/policy update require relatively little time. The dominant cost is model updating, especially under all-model update. This is expected because all-model update trains every candidate learner after each incoming instance.

Figure 6 makes this point clear. Selected-only update is the fastest because it updates only one learner. All-model update gives stronger predictive performance, but its model update time is much higher. Hybrid update sits between these two cases, improving model stability while avoiding most of the cost of all-model update.

This timing behavior is important for interpreting the MAB framework. The selected-only strategy is attractive for low-latency deployment, but it may under-train non-selected arms. The all-model strategy provides stronger predictive reliability, but its cost is closer to maintaining a fully trained model pool. The hybrid strategy provides a more practical compromise.

5.5. Memory Efficiency Analysis

Memory footprint is also relevant for online deployment, especially in resource-constrained environments. The memory values reported here are implementation-level estimates based on serialized model objects. Therefore, they should be interpreted as relative memory indicators rather than exact runtime RAM measurements.

Figure 7 compares the estimated memory footprint of AMF, the full ensemble, and the three MAB update strategies. The full ensemble and all-model MAB have relatively high memory requirements because they maintain all candidate learners. Selected-only MAB may have a smaller memory footprint because non-selected learners receive fewer updates and therefore grow less. However, this lower memory usage should be interpreted carefully. A smaller model footprint may reflect under-trained candidate arms rather than a purely positive efficiency gain.

The memory analysis complements the timing results. All-model update gives better predictive performance because all arms remain trained, but this also increases both update time and memory footprint. Selected-only update is lighter but less stable. Hybrid update again provides a middle-ground configuration.

5.6. Timing–Memory–Performance Trade-off

The central message of the trade-off analysis is that no single method is optimal across all dimensions. The strongest standalone learner provides a favorable balance between predictive reliability and processing time, while the full ensemble requires substantially higher computational cost without consistently improving predictive performance. Within the MAB framework, the update strategy determines the operating point of the system. Selected-only MAB favors computational efficiency, all-model MAB favors predictive reliability, and hybrid MAB provides an intermediate configuration that balances detection performance and resource consumption.

Table 5 summarizes the average predictive and timing performance of the MAB update strategies across the realistic shuffled settings. The results show a clear relationship between update frequency and predictive reliability. Selected-only update achieves the lowest average processing time of 5.65 ms per instance, but it also has the lowest average Cohen’s kappa among the three MAB update strategies. This is expected because only the selected arm is updated, while non-selected arms may receive insufficient training and become less competitive as the stream progresses. In contrast, all-model update achieves the highest average accuracy, recall, F1-score, and kappa, but its average processing time increases to 24.89 ms per instance because all candidate learners are updated after each instance. Hybrid update lies between these two extremes, achieving better predictive reliability than selected-only update while remaining much faster than all-model update.

Compared with selected-only update, all-model update improves average kappa from 0.8694 to 0.9033 and average recall from 0.9338 to 0.9505. However, this improvement increases average processing time from 5.65 ms to 24.89 ms, which is more than a fourfold increase. This indicates that the additional predictive reliability obtained by all-model update is not free; it is mainly achieved by keeping all arms continuously trained. Hybrid update provides a more moderate improvement, increasing average kappa to 0.8834 and recall to 0.9376 while requiring only 8.01 ms per instance. From a deployment perspective, this makes hybrid update attractive when the system must improve reliability without incurring the high cost of updating all models.

Figure 8 and Figure 9 visualize the relationship between predictive performance and processing time. Cohen’s kappa is used to represent predictive reliability beyond chance agreement, while recall is used to represent phishing detection capability. In these figures, the point annotations use compact abbreviations: F-Rnd denotes the full feature set with random shuffle, F-Str denotes the full feature set with stratified shuffle, N-Rnd denotes the no-RT/rule feature setting with random shuffle, and N-Str denotes the no-RT/rule feature setting with stratified shuffle.

Figure 8 shows that the full ensemble occupies a high-cost region but does not achieve the highest kappa. This suggests that evaluating all candidate learners through ensemble voting is not necessarily cost-effective. The strongest standalone learner appears in a favorable region, combining high kappa with relatively low processing time. The MAB update strategies form a continuum of operating points. Selected-only MAB is located in the lower-cost region, but its kappa is generally lower because some arms may be under-trained. All-model MAB shifts toward the higher-cost region but achieves stronger kappa because all learners remain updated throughout the stream. Hybrid MAB occupies the intermediate region, providing a practical compromise between reliability and processing cost.

Figure 9 provides a security-oriented view of the same trade-off. Recall is particularly important in phishing detection because false negatives correspond to phishing websites that are incorrectly classified as legitimate. The figure shows that stronger recall can be achieved by maintaining more frequently updated learners, especially under all-model update. However, the recall improvement must be interpreted together with processing time. A small gain in recall may require a substantial increase in update cost, depending on the selected update strategy. This reinforces the importance of evaluating phishing detectors not only by their detection scores, but also by the computational cost required to obtain those scores.

Figure 10 and Figure 11 extend the trade-off analysis to memory efficiency. The memory values should be interpreted as implementation-level estimates based on serialized model objects rather than exact runtime RAM measurements. Nevertheless, they are useful for comparing the relative memory demand of the evaluated configurations. In Figure 10, Cohen’s kappa is plotted against estimated memory footprint. This view shows whether stronger reliability is associated with larger model storage requirements. In Figure 11, processing time and memory footprint are shown together, while marker size represents Cohen’s kappa. This allows the three dimensions of online deployment—predictive reliability, latency, and memory demand—to be interpreted jointly.

The memory-oriented figures further clarify the cost of keeping multiple learners active. The full ensemble and all-model MAB require larger memory footprints because they maintain all candidate learners. Selected-only MAB may appear more memory-efficient because non-selected learners are updated less frequently and therefore grow more slowly. However, this apparent memory advantage has to be interpreted carefully: lower memory consumption may also indicate that some candidate learners are under-trained. Hybrid update reduces this issue by allowing some updates to non-selected arms while avoiding the full memory and timing cost of all-model update.

Overall, the timing–memory–performance analysis shows that the proposed framework should be viewed as a multi-objective online detection system. If the main objective is predictive reliability, all-model MAB and the strongest standalone learner are strong candidates. If the main objective is low processing time, selected-only MAB is attractive but may sacrifice reliability. If a balanced deployment setting is required, hybrid MAB provides the most practical MAB configuration because it improves predictive performance compared with selected-only update while avoiding much of the timing and memory overhead of all-model update.

5.7. MAB Policy Behavior

The previous subsections compare the update strategies and their trade-offs. This subsection examines the MAB policies themselves to explain which exploration–exploitation strategies are most suitable for online phishing detection. Figure 12 and Figure 13 report the average policy performance across the realistic shuffled-stream settings, namely the random-shuffle and stratified-shuffle streams, under both feature settings. The original stream order is excluded from this policy-level summary because it produces overly optimistic results due to its class-blocked structure. Therefore, these figures focus on the more realistic settings in which phishing and legitimate instances are interleaved throughout the stream.

Figure 12 shows that adaptive policies generally achieve higher Cohen’s kappa than random selection, indicating that reward-guided arm selection improves predictive reliability beyond chance agreement. Bayes-UCB provides the strongest average reliability under selected-only and hybrid update, while Thompson Sampling achieves the strongest average reliability under all-model update. This suggests that Bayesian exploration strategies are effective because they maintain uncertainty-aware estimates of arm quality instead of relying only on point estimates or fixed exploration rates.

Figure 13 provides a complementary security-oriented view. Since recall measures the ability to identify phishing instances, it reflects the extent to which a policy avoids false negatives. The recall comparison shows that the stronger policies are generally those that balance exploration and exploitation without spreading selections too uniformly across weak arms. Random selection is consistently weaker because it does not use reward feedback to concentrate selections on better-performing learners. Epsilon-Greedy and EXP3 provide exploration, but their behavior is less stable in this setting: Epsilon-Greedy depends strongly on the fixed exploration rate, while EXP3 may continue allocating probability mass to weaker arms due to its adversarial exploration mechanism.

The influence of the update strategy is also clear. Under selected-only update, policy decisions have a stronger effect because only the selected arm receives training. This can amplify early selection choices and may cause some non-selected arms to become under-trained. Under all-model update, all arms remain trained throughout the stream, allowing policies such as Thompson Sampling and Bayes-UCB to compare more reliable arm estimates. Hybrid update provides an intermediate condition by reducing the under-training effect while avoiding the full cost of updating all arms at every instance. Overall, the policy comparison indicates that MAB behavior is shaped jointly by the exploration–exploitation strategy and the model update mechanism, rather than by the policy alone.

5.8. Arm Selection Behavior

Predictive metrics show how well each policy performs, but they do not fully explain why. Figure 14 presents the average arm selection distribution across MAB policies and update strategies. The values are averaged over the realistic shuffled-stream settings, including random shuffle and stratified shuffle, under both full-feature and no-RT/rule feature settings.

Figure 14 shows that stronger policies frequently allocate more selections to AMF, which is consistent with AMF being the strongest standalone learner. This indicates that reward-guided MAB policies are able to learn meaningful arm preferences rather than selecting models uniformly. The concentration on AMF helps explain the strong kappa and recall achieved by policies such as Bayes-UCB and Thompson Sampling.

The selection pattern also depends on the update strategy. Under selected-only update, only the selected arm is trained after each instance. This can create a self-reinforcing effect: arms selected early receive more updates, become more competitive, and are selected again, while rarely selected arms may become under-trained. This explains why selected-only MAB is efficient but less stable.

Under all-model update, all arms are continuously trained, so the policy compares learners under more equal training conditions. This makes the learned selection preferences more reliable, but increases computational and memory cost. Hybrid update provides a compromise by updating the selected arm while occasionally updating non-selected arms, reducing under-training without fully incurring the cost of all-model update.

Overall, Figure 14 shows that MAB performance is shaped by both the policy and the update mechanism. Strong policies perform well because they increasingly favor suitable learners, especially AMF, while maintaining enough exploration to avoid poor early commitments.

5.9. Overall Discussion

This subsection discusses the research questions based on the achieved experimental results. The aim is to connect the main findings to the objectives of the study and to clarify what the results reveal about the predictive performance, computational efficiency, policy behavior, and sensitivity of the proposed MAB-based online model selection framework.

RQ1: How does MAB-based online model selection compare with individual incremental learners and full ensemble prediction in terms of predictive performance?

The results show that MAB-based online model selection provides competitive predictive performance compared with individual incremental learners and full ensemble prediction. However, the comparison is nuanced. AMF remains the strongest standalone incremental learner across the realistic shuffled-stream settings, achieving consistently high accuracy, precision, recall, F1-score, and Cohen’s kappa. All-model MAB and hybrid MAB approach AMF performance and generally outperform selected-only MAB. The full ensemble, despite combining all candidate learners, does not consistently outperform AMF or the stronger MAB update strategies. This suggests that using all models through majority voting is not necessarily better than selecting a suitable learner adaptively. Therefore, MAB-based selection should not be interpreted simply as a method that always exceeds the best individual learner, but rather as an adaptive mechanism that can provide strong and flexible performance when the best learner is not known in advance.

RQ2: Which MAB policy provides the most effective exploration–exploitation behavior for phishing detection streams?

The policy-level results indicate that Bayesian exploration strategies provide the most effective exploration–exploitation behavior for phishing detection streams. Bayes-UCB performs strongly under selected-only and hybrid update strategies, while Thompson Sampling performs strongly under all-model update. These policies achieve stronger reliability than random selection and generally provide better behavior than simpler strategies such as Epsilon-Greedy and EXP3. This suggests that uncertainty-aware exploration is useful in this problem because the suitability of candidate learners may differ across stream settings and update strategies. In contrast, random selection does not learn from feedback and consistently performs weakest, which confirms the value of adaptive arm selection.

RQ3: What are the computational trade-offs of MAB-based model selection compared with individual learners and full ensemble prediction?

The timing and memory analyses show that the computational benefit of MAB-based model selection depends strongly on the update strategy. Selected-only MAB has the lowest processing cost because it predicts and updates only one selected arm at each time step. Hybrid MAB increases the update cost moderately but improves predictive stability by allowing non-selected arms to receive occasional updates. All-model MAB provides the strongest MAB predictive performance, but it substantially increases model update time and memory footprint because all arms remain actively trained. Compared with the full ensemble, MAB-based approaches provide more flexible operating points: selected-only and hybrid MAB reduce computational overhead, while all-model MAB behaves closer to a fully maintained model pool. Therefore, the main efficiency advantage of MAB is not universal; it appears most clearly when the update strategy avoids updating all models at every instance.

RQ4: How does the arm selection behavior vary across MAB policies and update strategies?

The arm selection analysis shows that MAB policies adapt their selections according to the relative usefulness of the candidate learners. Stronger policies frequently select AMF, which is also the strongest standalone learner in the experiments. This indicates that the learned arm preferences are meaningful and broadly reflect model suitability. However, the selection behavior is also affected by the update strategy. Under selected-only update, arms selected early receive more training, while non-selected arms may become stale. This can create a self-reinforcing effect in which the policy continues to favor arms that were selected and trained more often, even if other learners could become competitive with sufficient updates. Hybrid and all-model update reduce this issue by keeping more candidate learners trained, leading to more reliable arm comparisons.

RQ5: How sensitive is the proposed framework to update strategy, stream order, and feature representation?

The sensitivity analysis shows that the proposed framework is affected by update strategy, stream order, and feature representation. Stream order has a clear effect: the original order produces optimistic results, while random and stratified shuffled streams provide a more realistic evaluation of online learning performance. The update strategy also plays a major role. Selected-only update favors efficiency but may reduce predictive stability; all-model update improves reliability but increases processing time and memory demand; and hybrid update provides the most practical balance between these two extremes. The feature-ablation results show that removing RT/rule-based features does not cause a major performance collapse, indicating that the remaining lexical, host-based, hyperlink, and webpage-structure features still contain strong discriminative information. This suggests that the framework is not solely dependent on rule-threshold indicators.

Overall, the findings show that the proposed MAB framework is best understood as a resource-aware adaptive model selection mechanism. Its main contribution is not that it always outperforms the best individual learner selected after evaluation, but that it provides a flexible way to adapt model choice during online detection while making the trade-offs among predictive reliability, recall, processing time, and memory footprint explicit. Among the tested configurations, hybrid MAB offers the most practical balance, while all-model MAB represents a performance-oriented configuration and selected-only MAB represents a low-cost configuration.

6. Conclusions

This paper investigated a multi-armed bandit-based online model selection framework for incremental phishing website detection. The motivation behind the study is that phishing detection is not only a predictive classification problem, but also an online decision-making problem in which the detector must adapt to incoming data while controlling computational cost. Instead of relying on a single fixed incremental learner or evaluating all learners through a full ensemble, the proposed framework treats heterogeneous incremental learning pipelines as candidate arms and dynamically selects one learner during stream processing.

The experimental results provide several important insights. First, MAB-based online model selection achieves competitive predictive performance compared with individual incremental learners and full ensemble prediction. However, the results also show that MAB does not necessarily outperform the strongest standalone learner selected after evaluation. This is an important and realistic finding. The value of MAB lies not in always exceeding the best individual model, but in providing an adaptive selection mechanism when the best learner is not known in advance or may change across stream conditions.

Second, the choice of MAB policy matters. Bayesian exploration strategies, particularly Bayes-UCB and Thompson Sampling, generally provide the most reliable exploration–exploitation behavior across the evaluated settings. These policies are better able to identify useful candidate learners than random selection and simpler exploration mechanisms. The arm selection analysis further shows that strong policies tend to favor learners that also perform well as standalone models, indicating that the learned selection behavior is meaningful.

Third, the computational analysis confirms that update strategy is a central design factor. Selected-only update provides the lowest processing cost because only the selected learner is updated after each instance, but it may cause non-selected arms to become under-trained. All-model update improves predictive reliability by keeping all candidate learners updated, but it increases model update time and memory footprint. Hybrid update provides the most practical compromise by improving predictive stability compared with selected-only update while avoiding much of the overhead of all-model update.

Fourth, the sensitivity analysis shows that stream order and feature representation affect online evaluation. The original stream order can produce optimistic results, while random and stratified shuffled streams provide a more realistic assessment of online detection performance. Removing RT/rule-based features does not cause a major collapse in performance, suggesting that the remaining lexical, host-based, hyperlink-related, and webpage-structure features still provide meaningful discriminative information.

Overall, the findings suggest that MAB-based online model selection is best understood as a resource-aware adaptive mechanism for incremental phishing detection. It provides multiple operating points: selected-only MAB for low-cost deployment, all-model MAB for performance-oriented deployment, and hybrid MAB for balanced deployment. This makes the framework useful in practical settings where detection reliability, false-negative risk, processing time, and memory footprint must be considered together.

Future work will consider extending the proposed framework to a federated learning setting for privacy-preserving and distributed phishing detection. This would allow multiple clients or organizations to collaboratively improve detection while keeping their local phishing data private.