Severity-Regularized Deep Support Vector Data Description with Application to Intrusion Detection in Cybersecurity

1. Introduction

Anomaly detection is a machine learning technique that has been widely applied in many domains, including cybersecurity, finance, healthcare, the industrial internet of things, and disaster management [1,2,3,4,5]. The objective of anomaly detection methods is to detect anomalous behavior by capturing a proper representation of normal operating conditions. However, anomalies are often heterogeneous in terms of their severity and consequences, where some have a low impact, while others are high-severity events that expose safety, privacy, or system availability. Failing to detect high-severity anomalies can lead to catastrophic damage.

Various deep learning methods have been developed for anomaly detection that utilize normal observation labels in training the method. A deep variant of support vector data description (SVDD) was developed by Ruff et al. [6]. To improve one-class separation, deep autoencoding of SVDD was proposed [7]. To preserve local data, structured deep structure preservation SVDD was developed [8]. Variational autoencoder (VAE)-based deep support vector data description (DeepSVDD) has been introduced to shape the latent space before applying the SVDD objective for effective feature representation [9]. Contrastive DeepSVDD was developed to enhance the feature representation from large data sets [10]. To utilize labeled anomaly information in combination with labeled normal in DeepSVDD, Deep semi-supervised anomaly detection (DeepSAD) was proposed [11]. While effective, these methods do not incorporate anomaly labels in their loss function or implicitly assume that all anomalies have the same level of severity and impact. As such, an approach that assigns a higher priority to severe anomaly detection can enhance critical anomaly detection.

In practice, several deep learning-based anomaly detection methods have been applied to cybersecurity. In aviation, a VAE-based DeepSVDD model on automatic dependent surveillance-broadcast (ADS-B) has been applied to detect communication tampering and anomalies [12]. Temporal convolutional networks and transformer-based SVDD were used to detect anomalies in ADS-B generated by denial of service (DoS) attacks [13]. In vehicular networks, intrusion detection systems (IDS), a collaborative DeepSVDD method of multiple agents, were proposed to improve vehicular network protection by reducing false negative rates [14]. A convolutional autoencoder combined with SVDD was introduced for IDS to improve the performance of industrial control systems (ICS) [15]. A lightweight network intrusion detection system for industrial IoT based on incremental SVDD was introduced to improve detection accuracy [16]. In smart electric grids, DeepSVDD has been applied to detect false data injection [17]. An adaptive policy-based anomaly control for cybersecurity has been introduced to reduce distributed DoS attacks that uses an out-of-distribution detector for neural networks combined with DeepSVDD [18]. A deep one-class support vector machine coupled with an autoencoder has been applied to IDS to predict deviations from normal network behavior at inference [19]. Despite improved intrusion detection performance, these methods assume that all types of attacks and anomalies have the same levels of impact and severity, while rarely using their labels in training. As such, assigning the same detection priority to all anomaly types, whereas in real-world applications, the anomalies vary based on their impact and severity. In cybersecurity risk management, for example, undetected attacks that create unauthorized (super)user accounts enable persistent footholds and escalation that would amplify damage, increase data loss and breach risk, complicate containment and raise recovery costs [20]. Therefore, an intrusion detection model that can prioritize the detection of more severe and impactful cyberattacks is essential for effective cybersecurity management.

Motivated by these challenges, this paper presents Severity-Regularized Deep SVDD (SR-DeepSVDD). Specifically, we propose a novel DeepSVDD variant that incorporates anomaly severity across multiple anomaly types based on application-specific importance assigned to each type. In the proposed loss function, we introduce regularizing parameters that control the importance assigned to each anomaly type. To our knowledge, we develop the first application of SR-DeepSVDD for intrusion detection in cybersecurity to improve detection rates of highly severe anomalies using the Australian Defence Force Academy Linux Dataset (ADFA-LD). Experiments are carried out on a simulated dataset and a real-world intrusion detection case study to demonstrate SR-DeepSVDD’s effectiveness in detecting highly severe anomalies while maintaining competitive overall performance.

The remainder of the paper is organized as follows. Section 2 proposes the newly developed SR-DeepSVDD method. Section 3 presents our experiments on the simulation dataset. An Intrusion Detection System based on SR-DeepSVDD is introduced in the case study in Section 4. Section 5 draws conclusions and future work directions.

2. Severity-Regularized Deep Support Vector Data Description

In this section, we develop a severity-aware extension of Deep SVDD. The primary concept is to maintain the Deep SVDD decision boundary within a deep latent space while incorporating anomaly-type severity regularization into the training objective. We first establish the notation, then briefly recap Deep SVDD and its limitations, and finally introduce a severity-regularized objective and an optimization procedure to obtain an effective decision boundary.

We consider anomaly detection with labeled normal observations and labeled anomaly observations partitioned into K anomaly types. Let ${\mathcal{X}}_{+}={\left\{{\mathbf{x}}_i\right\}}_{i=1}^N$ denote the N normal training observations, where ${\mathbf{x}}_i\in {\mathbb{R}}^d$. For each type of anomaly $k\in \{1,\dots ,K\}$, let ${\mathcal{X}}_{-}^{\left(k\right)}={\left\{{\mathbf{x}}_j^{\left(k\right)}\right\}}_{j=1}^{M_k}$ denote the $M_k$ labeled anomalies of that type. A deep encoder $\varphi (\mathbf{x};\mathbf{W}):{\mathbb{R}}^d\to {\mathbb{R}}^q$ maps the inputs into a latent space, producing embeddings $\mathbf{z}=\varphi (\mathbf{x};\mathbf{W})\in {\mathbb{R}}^q$, in which we learn a hypersphere of normality with center $\mathbf{c}\in {\mathbb{R}}^q$ and radius $R\ge 0$.

2.1. Introduction to Deep SVDD

The goal of Deep SVDD is to learn a hypersphere representation from normal observation embeddings centered around a fixed latent point such that proximity to that center indicates normality [6]. In the fixed center (one-class) formulation, the encoder parameters $\mathbf{W}$ are learned by minimizing the average squared distance of normal embeddings to the center with weight decay.

$$\underset{\mathbf{W}}{min}{\displaystyle \frac{1}{N}}\sum _{\mathbf{x}\in {\mathcal{X}}_{+}}\parallel \varphi (\mathbf{x};\mathbf{W})-\mathbf{c}{\parallel }_2^2+\lambda \sum _{\ell }\parallel {\mathbf{W}}^{(\ell )}{\parallel }_F^2,$$

(1)

where $\lambda >0$ is the layerwise weight-decay coefficient.

A soft-boundary approach concurrently learns a nonnegative radius and penalizes only the normals that lie outside the sphere. The parameter $\nu \in (0,1]$ governs the permissible fraction of violations.

$$\underset{\mathbf{W},R\ge 0}{min}{R}^2+{\displaystyle \frac{1}{\nu N}}\sum _{\mathbf{x}\in {\mathcal{X}}_{+}}max\left\{0,\parallel \varphi (\mathbf{x};\mathbf{W})-\mathbf{c}{\parallel }_2^2-R^2\right\}+\lambda \sum _{\ell }\parallel {\mathbf{W}}^{(\ell )}{\parallel }_F^2.$$

(2)

In practice, the center $\mathbf{c}$ is initialized from a warm-up pass on normals (e.g., the mean of initial embeddings) and then fixed to stabilize optimization, and bias terms are removed from the network to prevent degenerate collapse [6].

Although effective as a one-class learner, the original formulation does not incorporate labeled anomalies or distinguish among anomaly types with different operational consequences. As such, highly severe anomalies can be overlooked and treated as normal in the monitoring phase. This limitation drives the development of a severity-aware extension that structures the latent space and boundary to preferentially identify higher-severity anomaly types.

2.2. Severity-Regularized Deep SVDD

In order to distinguish between anomaly types while preserving the SVDD geometry, we propose a new soft-boundary Deep SVDD objective with type-specific severity-regularized push-out penalties.

$$\begin{array}{cc}\hfill \underset{\mathbf{W},R\ge 0}{min}\mathcal{L}(\mathbf{W},R)=& R^2+{\displaystyle \frac{1}{\nu N}}\sum _{i=1}^{N}max\left\{0,\parallel \varphi ({\mathbf{x}}_i;\mathbf{W})-\mathbf{c}{\parallel }_2^2-R^2\right\}\hfill \\ \hfill & +\sum _{k=1}^K{\displaystyle \frac{{\Omega }_k}{M_k}}\sum _{j=1}^{M_k}max\left\{0,m_k+R^2-\parallel \varphi ({\mathbf{x}}_j^{\left(k\right)};\mathbf{W})-\mathbf{c}{\parallel }_2^2\right\}\hfill \\ \hfill & +\lambda \sum _{\ell }\parallel {\mathbf{W}}^{(\ell )}{\parallel }_F^2,\hfill \end{array}$$

(3)

where $\nu$ is the same soft-boundary parameter as in (2); ${\Omega }_k>0$ encodes the importance of the anomaly type k, and $m_k\ge 0$ is an optional clearance margin for that type.

In the proposed formulation, the $R^2$ term discourages an unnecessarily large hypersphere, while the hinge in the second term activates only for normal embeddings that fall outside the boundary; thus, those outlying normals are pulled inward. The parameter $\nu \in (0,1]$ controls how tight the sphere is allowed to be by setting the tolerated fraction of such violations, a smaller $\nu$ yields a tighter hypersphere (fewer allowed outliers), and a larger $\nu$ permits more normal observations outside the hypersphere, obtaining a wider boundary. Normalization by N keeps the contribution per normal observation roughly scale-free as the training dataset grows larger. The third term adds the severity-regularized anomaly push-out per anomaly type. The hinge activates only when an anomaly of type k is inside the sphere or within the type-specific margin of width $m_k$. When active, it pushes those embeddings outward, either increasing their distance from $\mathbf{c}$ in the latent space or, through the joint optimization, discouraging an increase of R that would otherwise encapsulate them. The weight ${\Omega }_k>0$ scales the importance of anomaly type k where a larger ${\Omega }_k$ pushes type k anomalies out of the hypersphere, and the margin $m_k\ge 0$ allows for an increased gap between type k anomalies and the boundary. Dividing by $M_k$ balances the per-type influence so that abundant or rare anomaly types do not dominate simply due to sample count. The last term is a layerwise weight decay that regularizes the network parameters $\mathbf{W}$. Crucially, these additions do not alter the decision geometry, and the score persists as $s\left(\mathbf{x}\right)={\parallel \varphi (\mathbf{x};\mathbf{W})-\mathbf{c}\parallel }_2^2-R^2$. This formulation enhances the representation and the hypersphere to facilitate stronger separation for more severe anomaly types.

If all severity weights are set to zero, i.e., ${\Omega }_k=0$ for every $k\in \{1,\dots ,K\}$, the push-out term is eliminated and $\mathcal{L}(\mathbf{W},R)$ reduces to

$$R^2+{\displaystyle \frac{1}{\nu N}}\sum _{i=1}^{N}max\left\{0,\parallel \varphi ({\mathbf{x}}_i;\mathbf{W}){-\mathbf{c}\parallel }^2-R^2\right\}+\lambda \sum _{\ell }\parallel {\mathbf{W}}^{(\ell )}{\parallel }_F^2,$$

which is exactly the soft-boundary Deep SVDD objective in (2). Thus, Deep SVDD is recovered as a special case of the proposed formulation.

Alternatively, setting $R=0$ produces a center-fixed SR-DeepSVDD with score $s\left(\mathbf{x}\right)=\parallel \varphi (\mathbf{x};{\mathbf{W}}^{*}){-\mathbf{c}\parallel }_2^2$ and anomaly push-out $max\{0,m_k-\parallel \varphi ({\mathbf{x}}_j^{\left(k\right)};\mathbf{W})-\mathbf{c}{\parallel }_2^2\}$. Its objective is

$$\begin{array}{cc}\hfill \underset{\mathbf{W}}{min}\mathcal{L}\left(\mathbf{W}\right)=& {\displaystyle \frac{1}{N}}\sum _{i=1}^N\parallel \varphi ({\mathbf{x}}_i;\mathbf{W})-\mathbf{c}{\parallel }_2^2\hfill \\ \hfill & +\sum _{k=1}^K{\displaystyle \frac{{\Omega }_k}{M_k}}\sum _{j=1}^{M_k}max\left\{0,m_k-\parallel \varphi ({\mathbf{x}}_j^{\left(k\right)};\mathbf{W})-\mathbf{c}{\parallel }_2^2\right\}\hfill \\ \hfill & +\lambda \sum _{\ell }\parallel {\mathbf{W}}^{(\ell )}{\parallel }_F^2.\hfill \end{array}$$

(4)

For smoother magnitude-aware gradients, one may square the hinge terms in (3) (and (4)). Specifically, replace $max\{0,\parallel \varphi ({\mathbf{x}}_i;\mathbf{W})-\mathbf{c}{\parallel }_2^2-R^2\}$ and $max\{0,m_k+R^2-\parallel \varphi ({\mathbf{x}}_j^{\left(k\right)};\mathbf{W})-\mathbf{c}{\parallel }_2^2\}$ by their squared forms

$$\begin{array}{cc}\hfill \underset{\mathbf{W},R\ge 0}{min}\mathcal{L}(\mathbf{W},R)=& R^2+{\displaystyle \frac{1}{\nu N}}\sum _{i=1}^N{\left(max\left\{0,\parallel \varphi ({\mathbf{x}}_i;\mathbf{W})-\mathbf{c}{\parallel }_2^2-R^2\right\}\right)}^2\hfill \\ \hfill & +\sum _{k=1}^K{\displaystyle \frac{{\Omega }_k}{M_k}}\sum _{j=1}^{M_k}{\left(max\left\{0,m_k+R^2-\parallel \varphi ({\mathbf{x}}_j^{\left(k\right)};\mathbf{W})-\mathbf{c}{\parallel }_2^2\right\}\right)}^2\hfill \\ \hfill & +\lambda \sum _{\ell }\parallel {\mathbf{W}}^{(\ell )}{\parallel }_F^2.\hfill \end{array}$$

(5)

This preserves zero loss for non-violations and leaves the decision score $s\left(\mathbf{x}\right)$ unchanged, while imposing a quadratic penalty on margin violations that makes the (L2-loss) objective differentiable and yields gradients that scale linearly with the violation—properties known to simplify and stabilize optimization [21,22,23]; see also the motivation for smoothing non-smooth hinge losses to

For a test point ${\mathbf{x}}_{\mathrm{test}}\in {\mathbb{R}}^d$, we score anomalies by the signed squared radial distance from its embedding to the center. Let ${\mathbf{W}}^{*}$ and $R^{*}$ denote the trained network parameters and radius; then

$$s\left({\mathbf{x}}_{\mathrm{test}}\right)=\parallel \varphi ({\mathbf{x}}_{\mathrm{test}};{\mathbf{W}}^{*})-\mathbf{c}{\parallel }_2^2-{R^{*}}^2.$$

(6)

According to this scoring convention, points located outside the hypersphere (anomalies) have $s>0$, inliers (normals) have $s<0$, and boundary points correspond to $s=0$. In the center-fixed variant ($R^{*}=0$), this reduces to the unsigned distance $s\left({\mathbf{x}}_{\mathrm{test}}\right)={\parallel \varphi ({\mathbf{x}}_{\mathrm{test}};{\mathbf{W}}^{*})-\mathbf{c}\parallel }_2^2$. In addition, training data are not required at the prediction time. The model is entirely defined by ${\mathbf{W}}^{*}$, the fixed $\mathbf{c}$, and the $R^{*}$ in the soft-boundary variant, which facilitates low-memory and rapid inference by a single forward pass through $\varphi (·;{\mathbf{W}}^{*})$.

Alternatively, an inference mechanism can be defined post-training and during validation. Given scores $s\left(\mathbf{x}\right)$, a threshold $\tau$ can be selected using a held-out validation set where common choices could be to fix a target false-alarm rate $\rho$ on validation normals and set $\tau$ to the corresponding quantile, or maximizing a utility that reflects type importance, e.g., ${\sum }_{k=1}^K{\Omega }_k{\mathrm{DR}}_k\left(\tau \right)$. This post-training calibration does not alter the learned representation or decision geometry.

2.3. Optimization of Severity-Regularized Deep SVDD

The network parameters $\mathbf{W}$ are optimized by stochastic gradient methods such as stochastic gradient descent (SGD), Adam, and AdamW on the squared-hinge SR-DeepSVDD objective in (5), using backpropagation and standard mini-batch training. This training procedure is in line with the deep SVDD training practices and scales linearly with the number of batches [6,24,25]. We first run a short warm-up on normal data to initialize the center $\mathbf{c}$ as the mean embedding and then keep $\mathbf{c}$ fixed. To avoid trivial “hypersphere collapse,” we use bias-free layers as suggested for Deep SVDD [6]. Rather than backpropagating through the radius, we adopt a lightweight alternating update: within each epoch, R is held fixed while $\mathbf{W}$ is updated; after the epoch, we refresh R by setting $R^2$ to the empirical $(1-\nu )$-quantile of the squared distances of training normals, consistent with the soft-boundary $\nu$-property (roughly $\nu$ of normals may lie on or outside the sphere) [6,26]. Mini-batches may include both normals and any labeled anomalies; the anomaly push-out term is applied per type with weights ${\Omega }_k$, margins $m_k$, and normalization by $M_k$. This epoch-wise quantile step is a fast block-coordinate update for R that empirically tracks the line-search update advocated in the original Deep SVDD when using the plain hinge [6]. In our implementation, we use the squared hinge in (5) for smoother, magnitude-aware gradients while leaving the decision score $s\left(x\right)={\parallel \varphi (x;\mathbf{W})-\mathbf{c}\parallel }_2^2-R^2$ unchanged.

2.4. Calibration of Severity Weights and Margins

The severity weights ${\Omega }_k>0$, importance of anomaly type k, and margins $m_k\ge 0$, desired clearance for type k are tuned on held-out validation set to prioritize the detection of more consequential and severe anomaly types while controlling false alarms

$$\left(\left\{{\Omega }_k\right\},\left\{m_k\right\}\right)\in arg\underset{{\Omega }_k\ge 0,m_k\ge 0}{max}\sum _{k=1}^K{\Omega }_k{\mathrm{DR}}_k\left(\tau \right)\mathrm{s}.\mathrm{t}.\mathrm{FAR}\left(\tau \right)\le \rho ,$$

where ${\mathrm{DR}}_k\left(\tau \right)$ is the detection rate for type k at threshold $\tau$ and $\rho$ is a target false alarm rate (FAR) on validation normals.

A practical approach is to maximize an application utility, for example, a weighted sum of per anomaly type detection rates, subject to a target false-alarm rate on validation normals. In choosing search spaces, ${\Omega }_k$ can be explored on a logarithmic grid and $m_k$ on a compact grid in latent-distance units with application-dependent bounds. Performance is typically most sensitive near small magnitudes and exhibits diminishing returns as values grow. While holding the other hyperparameters fixed, the type-k detection rate ${\mathrm{DR}}_k$ generally increases monotonically with ${\Omega }_k$ and $m_k$ until saturation. Increasing multiple ${\Omega }_k$ simultaneously can reduce the distinction between types of anomalies and should reflect the true severity or operational cost.

The soft-boundary parameter $\nu$ also shapes calibration. A larger$\nu$ tightens the boundary (smaller R) and could increase detection at the cost of higher false alarms. In contrast, a smaller$\nu$ widens the boundary (larger R), reducing false alarms but potentially lowering detection unless ${\Omega }_k$ and/or $m_k$ are increased.

In calibration procedures, it is essential to consider class imbalance to ensure that infrequent but critical types are not dominated by more common ones, which can be achieved through per anomaly type application-based utility or normalization. Selections should aim for operating points where further increases in ${\Omega }_k$ or $m_k$ result in small marginal gains in detection. Throughout, these choices preserve the standard SVDD decision score $s\left(\mathbf{x}\right)={\parallel \varphi (\mathbf{x};\mathbf{W})-\mathbf{c}\parallel }_2^2-R^2$ and its interpretation.

Algorithm 1 summarizes SR-DeepSVDD training while Algorithm 2 describes inference via scoring and optional validation-time threshold calibration.

3. Experiments on Simulated Data

In this section, we evaluate the proposed Severity-Regularized Deep SVDD (SR-DeepSVDD) against three baseline methods: Deep SVDD [6], one class suport vetor machines (OC-SVM) [27], and classical SVDD with an RBF kernel (SVDD-RBF)[26]. We also compare it to an ablation of SR-DeepSVDD (SR-DeepSVDD-NS) where we do not distinguish between anomaly types, which can be considered a variant of DeepSAD without unlabeled data [11]. The study uses a two-dimensional simulated dataset that induces an irregular normal region (banana shape) and two anomaly types with different severities.

3.1. Experimental Setup

We generate a total of $1,100$ observations in ${\mathbb{R}}^2$: 700 normals and $200+200$ anomalies across two types. Normal samples with a mean vector ${[0,0]}^{\top }$ are sampled from banana-shaped data. We define severe anomalies as Type-1, drawn from $\mathcal{N}({\mu }_{a1},{\Sigma }_{a1})$ with ${\mu }_{a1}={[0,4]}^{\top }$ and diagonal covariance ${\Sigma }_{a1}=\mathrm{diag}(0.5,0.9)$ (off-diagonal entries are zero). Non-severe anomalies are defined as Type-2, sampled from $\mathcal{N}({\mu }_{a2},{\Sigma }_{a2})$ with ${\mu }_{a2}={[0,-5]}^{\top }$ and diagonal covariance ${\Sigma }_{a2}=\mathrm{diag}(0.5,0.9)$ (off-diagonal entries are zero). Data are split into train, validation, and test sets with stratified ratios $0.6/0.2/0.2$ over the normal class and each anomaly type. A representative training split is shown in Figure 1.

For evaluation, we report per–anomaly-type recall, precision, and F1 using the anomaly-type index $k\in \{1,2\}$. For each k, we compute the confusion counts $({\mathrm{TP}}_k,{\mathrm{FP}}_k,{\mathrm{FN}}_k)$ on the subset $\left\{\mathrm{normals}\right\}\cup \left\{\mathrm{anomalies}\mathrm{of}\mathrm{type}k\right\}$. The typewise metrics are

$${\mathrm{Recall}}_k={\displaystyle \frac{{\mathrm{TP}}_k}{{\mathrm{TP}}_k+{\mathrm{FN}}_k}},{\mathrm{Precision}}_k={\displaystyle \frac{{\mathrm{TP}}_k}{{\mathrm{TP}}_k+{\mathrm{FP}}_k}},\mathrm{F}{1}_k={\displaystyle \frac{2·{\mathrm{Precision}}_k·{\mathrm{Recall}}_k}{{\mathrm{Precision}}_k+{\mathrm{Recall}}_k}}.$$

In addition, we report the FAR on normals,

$$\mathrm{FAR}={\displaystyle \frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}}},$$

where $\mathrm{FP}$ and $\mathrm{TN}$ are computed over all normal samples. Here, $\mathrm{TP}=\mathrm{True}\mathrm{Positives}$, $\mathrm{TN}=\mathrm{True}\mathrm{Negatives}$, $\mathrm{FP}=\mathrm{False}\mathrm{Positives}$, and $\mathrm{FN}=\mathrm{False}\mathrm{Negatives}$.

We compare SR-DeepSVDD against Deep SVDD, OC-SVM, and SVDD-RBF methods. All deep methods share the same encoder capacity for fairness—a bias-free multi-layer perceptron (MLP) $\varphi (·;\mathbf{W})$ with two hidden layers (64, 32) and a 16-dimensional embedding—and are trained with AdamW for 50 epochs using mini-batches of size 128. For SR-DeepSVDD we use the soft-boundary objective with squared hinges and $\nu =0.05$, severity weights and margins are set to ${\Omega }_1=5,{\Omega }_2=2$ and $m_1=m_2=0.5$, and optimization uses AdamW (learning rate ${10}^{-3}$ with weight decay ${10}^{-6}$). At validation, the threshold $\tau$ is chosen to maximize the Type-1 detection rate subject to a false alarm cap $\mathrm{FAR}\le 0.05$ on validation normals. Deep SVDD uses the same encoder and $\nu =0.05$ with a squared hinge loss, trained on labeled normals only with AdamW (learning rate ${10}^{-4}$, weight decay ${10}^{-6}$); its $\tau$ is selected on validation to maximize F1 under the same FAR cap. OC-SVM employs an RBF kernel with $\nu =0.05$ and ${\gamma }_{RBF}=2$, and makes decisions by its native decision boundary. SVDD-RBF uses $\nu =0.05$ with ${\gamma }_{RBF}=2$ and also makes decisions using its native decision boundary. Moreover, to evaluate the effectiveness of severity weights and margins, we report an ablation, SR-DeepSVDDNS, which is identical to SR-DeepSVDD in terms of architecture, optimization, and validation policy but sets anomaly-specific parameters ${\Omega }_1={\Omega }_2=1$ and $m_1=m_2=0.5$, thereby does not differentiate between anomaly types.

To assess robustness, we repeat the entire procedure over 10 random seeds. For all methods, we select—per seed—the configuration according to the validation policy of each method described above. We report the Type-1 $\left(\mathrm{F}{1}_1\right)$ score as mean ± standard deviation (SD) across seeds. We also show the performance over a single selected seed for all performance metrics.

3.2. Results and Analysis

Table 1 lists the overall and per–anomaly-type metrics (including FAR) on a representative seed, while Table 2 summarizes the overall F1 across the 10 seeds as mean ± SD. The decision boundaries for each method are visualized in Figure 2, where the heat map shows the anomaly score $s\left(x\right)$ and the dashed curve denotes the selected threshold $\tau$.Table 1 lists the overall and per–anomaly-type metrics (including FAR) on a representative seed, while Table 2 summarizes the overall F1 across the 10 seeds as mean ± SD. The corresponding score fields and calibrated decision boundaries for each method are visualized in Figure 2, where the heat map shows the anomaly score $s\left(x\right)$ and the dashed curve denotes the selected threshold $\tau$.

The OC-SVM algorithm establishes a wide, radially shaped boundary (Figure 2 a) that identifies numerous anomalies but also inaccurately categorizes peripheral normals as anomalies, aligning with the elevated False Alarm Rate (FAR) detailed in Table 1.SVDD-RBF adjusts its boundary to align more closely with the normal distribution density (Figure 2 b), resulting in a more balanced precision-recall trade-off compared to OC-SVM and generating fewer false alarms on normals. DeepSVDD, trained using normal observations, establishes an excessively broad decision boundary (Figure 2 c). This boundary successfully encompasses distant normal observations but also inadvertently includes Type-1 anomalies, resulting in a compromise. Consequently, this negatively impacts its typewise F1 score, despite maintaining reasonable precision, and reduces the detection rate for Type-1 anomalies. Including anomaly information into decision boundary training, although without considering anomaly type severity, SR-DeepSVDD-NS improves the deep boundary as the boundary expands more symmetrically around the normal region while trying to push anomalies out (Figure 2 d), producing solid, uniform gains across types but without explicitly prioritizing the severe class.

In contrast, SR-DeepSVDD contracts the decision boundary to be more precise, especially in areas where severe Type-1 anomalies are concentrated (Figure 2 e). This method effectively isolates the top cluster of severe anomalies better than the ablation (SR-DeepSVDD-NS) approach, as shown by the improved detection rate for the Type-1 anomaly, while still providing strong detection rates for lower-priority anomaly types. This underscores the importance of the newly introduced severity importance weights, which emphasize the detection of severe anomalies. Moreover, the seed-level results in Table 2 further indicate that SR-DeepSVDD is consistently the most robust across random initializations, with SR-DeepSVDD-NS following behind.

3.3. Effect of Severity Importance Weight ${\Omega }_k$ on Performance

We study how the severity weight placed on Type-1 anomalies (${\Omega }_1$) affects SR-DeepSVDD while keeping all other settings fixed. Figure 3 plots the Type-1 detection rate (${\mathrm{DR}}_1$) on the left y-axis colored with a blue line and the overall F1 score on the right y-axis colored with a red against ${\Omega }_1$ on the x-axis. When ${\Omega }_1$ is very small, the loss under-weights Type-1 violations and the model behaves similarly to a non-distnguishing variant, producing weak ${\mathrm{DR}}_1$ and only moderate F1. As ${\Omega }_1$ increases to a moderate range, both ${\mathrm{DR}}_1$ and F1 improve sharply and remain high over a wide range, indicating that giving meaningful preference to the severe class helps the network push Type-1 anomalies farther beyond the boundary while maintaining overall balance. Pushing ${\Omega }_1$ to very large values eventually degrades both curves, as excessive emphasis on the severe class distorts the boundary, trading off precision and harming overall F1. These results suggest careful selection of ${\Omega }_k$ is required over the validation set where moderate values deliver a favorable trade-off between prioritizing detection of type k anomalies and preserving overall performance.

4. Case Study: Severity-Regularized Deep SVDD for Intrusion Detection in Cybersecurity

This section presents our case study on host-based intrusion detection. We analyzed Linux system-call sessions from the Australian Defence Force Academy Linux Dataset (ADFA-LD) [28,29]. It captures normal activity and multiple attack families under controlled conditions. The data set has family-level labels for both attack and normal sessions.

4.1. Intrusion Detection Experiment Setup

To reflect operational priorities in cybersecurity risk management, we introduce a severity importance mechanism over attack families. According to a report prepared by the National Institute of Standards and Technology, undetected intrusions that create unauthorized (super)user accounts allow for persistent presence and escalation, potentially exacerbating harm, enhancing data loss and breach risk, complicating containment efforts, and increasing recovery expenses. [20]. Therefore, the "Adduser" attack family that resembles this type of threat in ADFA-LD has been considered as the severe anomaly type (Type-1 anomalies) in our case study experiments. In contrast, the rest of the families have been considered as non-severe (Type-2 anomalies) for the purpose of this experiment. Thus, the objective is to be able to detect "Adduser" intrusions with higher accuracy without sacrificing the overall performance of the intrusion detection model.

We represent each session as a fixed-length vector. Traces are tokenized, and term frequency–inverse document frequency (TF–IDF) is computed over n-gram lengths 1–4 with sublinear TF. The TF–IDF model is fit once on all training sessions, including normal and anomaly traces. To reduce out-of-vocabulary drift, the fitted transform is applied unchanged to validation and testing sessions. In parallel, we compute the length of the session, the logarithmic length, the number of unique tokens, the Shannon entropies of the unigram/bigram Shannon, the proportion of the most frequent token and the cumulative proportion of the three most frequent tokens, the Gini unigram coefficient, burstiness (count coefficient of variation in non-overlapping windows of 50), and an n-gram diversity ratio. These features are standardized using means and standard deviations from training normals only. Finally, TF–IDF is densified and concatenated with the standardized tabular block to produce a single input vector per session. This follows common intrusion detection systems practice while ensuring reproducibility and compatibility with deep boundary learners that require fixed-dimensional inputs [30,31,32,33].

The dataset is partitioned at the session level where normals arae randomly partitioned as $60\%,20\%$, and $20\%$ into training, validation, and test sets, respectively, while anomalies are split, per attack family, with $20\%$,$20\%$, and $60\%$ into training, validation, and testing sets. Training of the baseline methods uses the training normals only, while SR-DeepSVDD training uses the same training normals together with labeled anomalies. All deep models share the same MLP encoder (two hidden layers of sizes 512 and 256 with a 64-dimensional latent representation). All methods use $\nu =0.05$ for training.

All methods use the same operating-point calibration policy as in our simulated study. SR-DeepSVDD and its no-severity ablation, SR-DeepSVDD-NS, share the training setup with AdamW optimizer (learning rate $1\times {10}^{-3}$, refresh rate 50 epochs, batch size 128, weight decay $1\times {10}^{-6}$). They differ only in severity settings, where SR-DeepSVDD uses $({\Omega }_1=1.5,{\Omega }_2=0.1)$ with margins $(m_1=m_2=0.5)$, whereas SR-DeepSVDD-NS sets $({\Omega }_1={\Omega }_2=1)$. DeepSVDD uses AdamW optimizer (learning rate $1\times {10}^{-4}$, refresh rate 10 epochs, batch size 128, and weight decay $1\times {10}^{-6}$). OC-SVM is trained with ${\gamma }_{RBF}=\mathtt{scale}$ and SVDD-RBF with ${\gamma }_{RBF}=1$.

4.2. Intrusion Detection Results and Analysis

Table 3 summarizes the overall and typewise performance on ADFA-LD where Type-1 denotes the highest severity targeted anomaly (Adduser attack family) and Type-2 denotes the lowest severity attack sessions. OC-SVM shows poor performance in terms of F1 and detection rate for the severe attack session, while also underperforming in capturing normal sessions successfully due to the increased FAR. SVDD-RBF shows better performance in detecting the severe attack sessions, as it builds a narrower decision boundary; however, this causes an increase in normal sessions being detected as attacks, thereby increasing its FAR.

Since DeepSVDD does not include an anomaly push-out term in its loss function, it produces a conservative operating point that broadens the decision boundary to capture normal sessions effectively but sacrifices recall (detection rate) on both types of anomalies, thereby reducing its typewise F1 despite reasonable precision.

Incorporating attack sessions into boundary learning (without severity weighting), the ablation, SR-DeepSVDD-NS, constructs a tighter boundary around normal sessions compared to DeepSVDD, pushing attack sessions outward, producing enhanced uniform gains across attack session types at a low FAR, yet without explicitly prioritizing the adduser attack family.

Meanwhile, the severity-aware SR-DeepSVDD contracts and sharpens the decision boundary where the severe attack sessions are concentrated, isolating these instances more effectively than the ablation, while maintaining strong performance on lower-priority attack families and keeping FAR competitively low. The improvements align with the severity importance weights that prioritize the detection of severe attack sessions. These results indicate the added advantages of using SR-DeepSVDD in cybersecurity risk management, specifically intrusion detection.

5. Conclusions

This paper presents Severity-Regularized Deep SVDD, a principled extension of Deep SVDD that incorporates anomaly-type severity into the training objective. This is achieved by introducing anomaly type-specific push-out penalties, which have adjustable importance weights and margins. The formulation retains the well-known SVDD decision geometry and scoring system while allowing for specific control over the balance between FAR and the prioritization of detecting anomalies with greater impact. In both a demanding simulated environment and a host-based intrusion detection cybersecurity case study utilizing ADFA-LD, SR-DeepSVDD consistently achieved the most robust overall performance compared to other one-class and deep baseline algorithms. Crucially, the approach enhanced the detection of the targeted (severe) anomaly type without increasing false positive rates on normal instances, while maintaining superior performance on lower-priority anomaly types. The ablation study without severity weights, SR-DeepSVDD-NS, validated the benefit of integrating labeled anomalies into boundary learning.

From an operational perspective, SR-DeepSVDD provides practitioners with a distinct severity control: the importance weights (${\Omega }_k$) and margins ($m_k$) can be adjusted using validation data to prioritize domain-critical attacks, such as account-creation intrusions, while ensuring that the overall FAR remains within acceptable limits. This approach does not introduce additional overhead inference time beyond the standard forward pass and threshold comparison, making it suitable for real-time monitoring.

SR-DeepSVDD utilizes labeled anomaly types to guide severity-aware training. In domains with scarce labels, semi-supervised approaches for uncovering and assigning importance to latent anomaly clusters represent a promising future research direction. Future work could include learning ${\Omega }_k$ and $m_k$ based on task-level utilities or costs, and the implementation of dynamic or data-dependent reweighting that adapts to drift during online monitoring. Collectively, these advancements can further enhance severity-based anomaly detection in applications where safety and security are critical.