Encrypted Social Media Traffic Fingerprinting Under Temporal Shift: A Public Benchmark and Multi-Flow Evaluation

1. Introduction

Mobile social media applications such as Facebook, Instagram, TikTok, WhatsApp, Telegram, YouTube, Reddit, Snapchat, Twitter, and LinkedIn have become integral to modern communication and generate substantial volumes of encrypted mobile network traffic. Beyond everyday communication, these applications can also provide useful evidence in criminal investigations, cyber incident response, and digital forensic examinations. When device artifacts are unavailable, deleted, or otherwise inaccessible, network traffic may provide supporting evidence for reconstructing application activity and communication patterns.

Modern mobile applications primarily rely on encryption protocols such as Transport Layer Security (TLS), Quick UDP Internet Connections (QUIC), and end-to-end encryption to protect communication confidentiality [1]. Although encryption prevents direct inspection of network payloads, it does not completely conceal communication behavior. Observable characteristics such as packet timing, flow duration, packet-size distributions, transport-layer behavior, and encrypted-session fingerprints remain available for analysis. Previous studies have shown that such characteristics can be used to distinguish applications without decrypting network traffic [2,3]. From a digital forensic perspective, encrypted network flows may therefore serve as supporting artifacts for identifying application-related network activity when payload contents are unavailable.

Despite growing interest in encrypted traffic fingerprinting, publicly available datasets specifically designed for mobile social media application identification remain limited. Existing studies often rely on proprietary datasets, different traffic environments, or evaluation procedures that make direct comparison and reproducibility difficult. In addition, performance obtained from randomly partitioned network flows may not reflect how a classifier behaves when applied to traffic collected during a later capture session.

To address these limitations, this study constructs and publicly releases a benchmark dataset for encrypted mobile social media traffic fingerprinting. The dataset contains traffic from ten widely used applications collected across five independent capture days and is accompanied by a reproducible processing pipeline for converting packet captures into labeled flow-level representations. The benchmark is evaluated using a temporally separated protocol in which Days 1–4 are used for model development, and Day 5 is reserved for final testing.

The main contributions of this work are as follows:

• We construct a publicly available benchmark dataset containing 25,116 encrypted mobile network flows from ten widely used social media applications collected across five independent capture days.
• We provide a reproducible packet-capture processing and preprocessing pipeline that transforms raw traffic into flow-level statistical representations while excluding leakage-prone attributes from model training.
• We evaluate five competitive tree-based classifiers using a temporally separated protocol in which Days 1–4 are used for model development, and an unseen Day 5 capture is reserved for final testing.
• We conduct a validation-only multi-flow ablation using window sizes of 1, 20, 40, 60, and 80 flows and select a common 60-flow operating window before accessing the final Day 5 results.
• We compare conventional single-flow classification with the locked 60-flow configuration on the temporally held-out Day 5 test set to quantify the value of aggregating evidence across consecutive encrypted flows.
• We apply SHAP and LIME to the highest-performing Day 5 classifier and evaluate SHAP–LIME agreement to examine the flow characteristics associated with application discrimination.
• We publicly archive the benchmark dataset through Mendeley Data [4] and provide the accompanying implementation and processing resources to support reproducibility and future comparative research.

The remainder of this paper is organized as follows. Section II reviews related work. Section III describes the dataset construction process. Section IV presents the methodology and experimental protocol. Section V reports the classification results, including the validation-based multi-flow ablation and temporally held-out Day 5 evaluation. Section VI presents the SHAP and LIME analyses. Section VII discusses the findings and their practical implications. Section VIII presents the limitations of the study. Finally, Section IX concludes the paper and outlines directions for future work.

2. Related Work

Encrypted traffic analysis has emerged as an active research area spanning traffic classification, anomaly detection, privacy analysis, asset identification, and cybersecurity monitoring. Shen et al. survey machine learning approaches for encrypted traffic analysis across traffic characterization, malicious activity detection, and user behavior inference [5]. Agrawal et al. similarly show that encrypted mobile traffic can reveal application usage, device behavior, and user activity through observable flow characteristics [6]. Together, these studies establish encrypted metadata as an important source of operational and forensic information when packet payloads are inaccessible.

Application fingerprinting seeks to infer application identities from encrypted traffic without inspecting payload contents. Early approaches relied on transport metadata, DNS information, packet statistics, and temporal correlations. He et al. showed that encrypted mobile applications retain distinguishable behavioral signatures despite payload encryption [7]. Fernandez-Terrasa et al. demonstrated that JA4+ TLS fingerprints remain useful for mobile application identification even as traditional metadata such as Server Name Indication (SNI) becomes less visible [8]. Yang et al. further showed that explicitly modeling packet size, direction, and inter-arrival time can improve fine-grained encrypted traffic classification [9].

Machine learning is widely used for encrypted traffic classification because it can capture complex relationships among observable traffic characteristics. Tree-based ensemble methods such as Random Forest and Gradient Boosting remain strong choices for structured flow-level data. Montieri et al. demonstrated the effectiveness of engineered flow-level features for encrypted mobile traffic classification [10]. Akbari et al. likewise showed that flow statistics, packet size, timing, and direction can support classification across HTTPS and QUIC traffic [11]. Their findings also demonstrated that classifiers may rely heavily on identifying TLS metadata such as SNI, emphasizing the importance of controlling leakage-prone attributes when evaluating encrypted application classification.

Multi-flow analysis has emerged as an alternative to treating every network flow as an independent classification instance. Chen et al. showed that single-flow classification may overlook relationships among related communications and proposed a “flow bunch” approach for multi-flow encrypted traffic classification [12]. Mengmeng et al. similarly demonstrated that relationships among multiple encrypted flows can improve application and behavior classification over isolated-flow approaches [13]. More recently, Wang et al. proposed MFSI, which represents multiple encrypted flows using a Multi-Flow Multi-Relational Graph and relational graph convolution to learn service-level traffic patterns [14]. These studies demonstrate the value of combining information from multiple flows and motivate further examination of how the amount of aggregated traffic context affects classification.

Explainable Artificial Intelligence (XAI) has also received increasing attention in network security, where predictive performance alone may provide limited insight into the characteristics responsible for model decisions. SHAP and LIME are widely used post-hoc techniques for examining feature contributions to machine learning predictions [15,16]. Recent forensic research has further demonstrated the complementary use of SHAP and LIME for examining feature relevance and model behavior [17]. Such analysis is particularly relevant to security and forensic applications because it allows analysts to examine the observable traffic characteristics associated with classification decisions rather than relying solely on predictive scores [18].

Reproducible research in encrypted mobile traffic classification is also constrained by the availability of suitable public datasets. Zhao et al. note that accurate application-level labeling and public release remain challenging because of privacy and traffic-collection requirements [19]. The MIRAGE dataset introduced by Montieri et al. provides an important public benchmark for mobile encrypted traffic classification [10], but it targets a broader collection of mobile applications rather than a benchmark specifically designed around social media applications. Mobile sensor forensic datasets [20] provide another source of mobile evidence but do not contain encrypted network-flow traffic for application fingerprinting. Previous encrypted traffic studies have also considered individual social media and messaging applications, but these applications are generally examined as subsets of broader application collections or under different traffic-classification objectives.

Despite these advances, the literature reviewed in this study does not provide a dedicated public benchmark that examines encrypted mobile social media applications in the same capacity considered here. In particular, limited attention has been given to a benchmark combining a social-media-specific application set, multi-day traffic collection, contemporary TLS/QUIC communication, leakage-controlled flow representations, temporal holdout evaluation, systematic multi-flow window analysis, and feature-level explanation within a single reproducible study. This distinction is important because social media applications frequently generate heterogeneous groups of encrypted connections for messaging, media delivery, advertisements, content retrieval, API communication, and background services, making their identification from isolated flows particularly challenging.

This study addresses this gap by constructing and publicly releasing a benchmark dataset containing encrypted network flows from ten widely used mobile social media applications collected across five independent capture days. Beyond dataset construction, the benchmark is used to examine how application discrimination changes when moving from isolated flows to consecutive-flow aggregation. In this way, the study provides both a reusable social-media-specific encrypted traffic dataset and a reproducible experimental baseline for evaluating application fingerprinting across capture sessions.

3. Benchmark Dataset Construction

This section describes the construction of the benchmark dataset used for encrypted mobile social media traffic fingerprinting. The dataset was designed to provide a reproducible collection of labeled, encrypted network flows suitable for machine-learning-based application fingerprinting and network traffic analysis.

3.1. Experimental Setup

Network traffic was collected using a controlled experimental environment consisting of an Android mobile device, a passive hardware network tap, and a dedicated monitoring workstation running Wireshark for packet capture. The Android device served as the traffic-generating endpoint and was used to interact with selected mobile social media applications under realistic usage conditions.

Internet connectivity between the mobile device and the broadband connection was routed through an ETAP-2003 passive network tap. The tap enabled packet-level monitoring without packet injection, modification, or active interference. A monitoring workstation connected to the tap’s monitor port captured raw network traffic in PCAP format using Wireshark. This passive architecture ensured transparent observation of traffic while preserving natural application behavior.

Figure 1 illustrates the experimental setup used for traffic capture.

3.2. Traffic Collection Protocol

Traffic collection focused on ten widely used social media applications: Facebook, Instagram, LinkedIn, Reddit, Snapchat, Telegram, TikTok, Twitter, WhatsApp, and YouTube. These applications were selected due to their widespread usage, diverse communication behavior, and increasing relevance in digital forensic investigations.

For each application, multiple traffic capture sessions were conducted across five collection periods to capture variability in user interaction and network conditions. During each session, realistic user activities were performed, including feed browsing, media streaming, content uploads, messaging, voice or video interactions where applicable, and application synchronization events. Traffic was collected separately for each application to ensure reliable label assignment during dataset construction.

3.3. Packet Capture and Flow Extraction

Captured network traces were stored in PCAP or PCAPNG format and processed using an automated Python-based pipeline based on NFStream. NFStream was selected because it provides efficient bidirectional flow generation together with rich statistical flow features suitable for network traffic analysis and machine learning applications [21]. The pipeline automatically identified all capture files in the input directory and extracted the application label and capture day from the filename. For example, filenames following the convention facebook_day1.pcapng and instagram_day3.pcapng were parsed to assign the application labels facebook and instagram and the capture-day identifiers day1 and day3, respectively.

Each packet capture was converted into bidirectional flow-level records using NFStream. NFStream was configured with statistical analysis, SPLT analysis, and up to 20 packet dissections using statistical_analysis=True, splt_analysis=True, and n_dissections=20. The resulting flow representation included statistical, temporal, packet-size, packet-inter-arrival-time, transport-layer, SPLT, and NFStream-derived application and protocol metadata.

Following flow extraction, three dataset metadata fields were added to each record: Label, capture_day, and capture_file. The Label field identifies the application associated with the controlled capture, capture_day identifies the collection day, and capture_file preserves the source capture filename. The Label field was repositioned as the first column, and each processed capture was exported as an individual CSV file.

3.4. Data Cleaning and Labeling

Application labels were assigned using the controlled data-collection procedure and capture-file naming convention rather than NFStream’s inferred application labels. Each capture file represented traffic collected during a controlled session for a designated social media application, and the corresponding application label and capture-day identifier were extracted automatically from the filename.

Importantly, the flow-extraction pipeline did not select or discard flows based on NFStream’s application_name field. All flows extracted from each controlled capture were retained at this stage. This avoids constructing the ground-truth dataset using the same application-identification information that could otherwise reveal the target class.

After extraction, the individual capture CSV files were grouped by application and concatenated to produce one merged CSV file per application. Schema consistency was checked during merging, while the application label was preserved as the first column. This resulted in ten application-level datasets corresponding to Facebook, Instagram, LinkedIn, Reddit, Snapchat, Telegram, TikTok, Twitter, WhatsApp, and YouTube.

3.5. Dataset Characteristics

Each application was captured across five independent collection days, producing 50 capture files that were subsequently processed with NFStream and merged by application into ten application-level CSV files. Although all applications were captured using the same collection protocol and approximately equivalent user interactions, the number of extracted bidirectional flows differs across applications. Because NFStream constructs flow records from the observed packet exchanges within each application session, the number of extracted flows depends on the characteristics of the captured traffic rather than the collection procedure itself [21]. Consequently, the resulting class imbalance reflects differences in the extracted network-flow records rather than inconsistencies in the data collection process. The final benchmark contains 25,116 labeled flow records, with the application-level distribution summarized in Table 1.

The extracted schema contains flow identifiers, endpoint and protocol metadata, bidirectional flow statistics, packet-size statistics, packet-inter-arrival-time statistics, transport-layer flag counts, SPLT sequence information, NFStream application metadata, and dataset metadata such as capture_day and capture_file. Application-level metadata extracted by NFStream was retained in the raw dataset at this stage but was subsequently treated separately during machine-learning feature preparation to prevent direct target leakage.

4. Methodology

This section describes the feature preparation, preprocessing, benchmark models, temporal evaluation strategy, multi-flow aggregation procedure, validation-based operating-point selection, training configuration, explainability analysis, and evaluation metrics used in this study.

4.1. Feature Selection and Preprocessing

Before model training, features that could introduce target leakage, capture-session memorization, or direct endpoint identification were removed. The complete list of excluded features and their corresponding removal categories is summarized in Table 2. These include the target label, capture metadata, NFStream application-identification outputs, endpoint identifiers, raw SPLT sequence fields, and absolute flow timestamps, all of which could leak application identity or capture-specific information into the learning process.

In particular, the NFStream application-identification fields and server-name information were removed because they may directly or indirectly reveal the target application. Likewise, source and destination endpoint identifiers were excluded to prevent models from memorizing capture-specific endpoints rather than learning generalizable encrypted traffic characteristics. Raw SPLT sequence fields and absolute timestamp attributes were also removed because they may encode collection-specific or temporal information that does not represent intrinsic application behavior.

After the predefined exclusions, infinite values were converted to missing values. Feature quality checks were then performed using the training partition only. Features containing exclusively missing values or having no variation in the training data were removed. This training-only procedure prevented information from the validation partition or held-out test day from influencing feature selection.

Numerical and categorical attributes were processed separately. Missing numerical values were replaced using median imputation, while missing categorical values were replaced using the most frequent category. To control categorical dimensionality, categories occurring fewer than ten times in the training data were mapped to a common __RARE__ category. Categorical variables were subsequently transformed using one-hot encoding, with previously unseen categories ignored during validation and testing.

All preprocessing transformations, including imputation, rare-category identification, categorical encoding, and training-based feature-quality filtering, were fitted exclusively on the training partition and then applied unchanged to the validation and held-out test data.

4.2. Benchmark Models

Five supervised machine-learning classifiers were evaluated: XGBoost, LightGBM, HistGradientBoosting, CatBoost, and Random Forest. These models were selected to provide representative tree-based ensemble and gradient-boosting approaches suited to heterogeneous tabular network-flow data. XGBoost and LightGBM provide scalable gradient-boosting implementations for large tabular datasets [22,23]. HistGradientBoosting implements histogram-based gradient boosting within the scikit-learn framework [24]. CatBoost uses ordered boosting to reduce prediction shift during training [25], while Random Forest was included as a robust bagging-based ensemble baseline [26].

The same leakage-controlled feature representation and preprocessing procedure were used for all five classifiers. The classifiers were trained at the individual-flow level; multi-flow aggregation was applied to their predicted class probabilities during validation and final testing rather than by retraining separate models on concatenated flow windows.

4.3. Temporal Development and Test Strategy

To evaluate generalization to traffic collected during an unseen period, a temporal holdout strategy was adopted. Capture Day 5 was reserved in its entirety as the final test set. Day 5 was not used to select the multi-flow window, select the focal model for explainability analysis, fit preprocessing transformations, or train the classifiers.

Data collected during Days 1–4 formed the model-development partition. Rather than randomly distributing individual flows between training and validation, each original capture file was first ordered according to its flow sequence. Within every Days 1–4 capture, the chronologically first 85% of flows were assigned to training and the final 15% were assigned to validation. The resulting structure was therefore

$${\mathcal{D}}_{\mathrm{train}}\cup {\mathcal{D}}_{\mathrm{val}}\subseteq \{\mathrm{Day}1,\mathrm{Day}2,\mathrm{Day}3,\mathrm{Day}4\},$$

(1)

with

$${\mathcal{D}}_{\mathrm{test}}=\left\{\mathrm{Day}5\right\}.$$

(2)

The contiguous within-capture split was used because the multi-flow analysis requires consecutive observations. A conventional random flow-level validation split would scatter flows from the same capture across training and validation and would disrupt the natural sequence required for constructing validation windows. The 85/15 split therefore preserves temporal ordering while ensuring that every development capture contributes observations to both training and validation.

The train-validation assignment was fixed for all experiments. Model training was repeated using five random seeds: 42, 123, 456, 789, and 2026. The seeds affected model stochasticity but did not alter which flows belonged to the training, validation, or Day 5 test partitions.

4.4. Multi-Flow Probability Aggregation

In addition to conventional single-flow classification, a multi-flow probability-aggregation strategy was evaluated to determine whether accumulating evidence from multiple consecutive encrypted flows improves mobile application identification.

The classifiers were trained exclusively at the individual-flow level. For an individual flow i, a trained classifier produces a probability vector over the $C=10$ application classes:

$${\mathbf{p}}_i=[p_{i,1},p_{i,2},\dots ,p_{i,C}],$$

(3)

where $p_{i,c}$ denotes the predicted probability that flow i belongs to application class c.

For multi-flow inference, consecutive flows belonging to the same original capture file were ordered according to their original flow sequence and partitioned into non-overlapping windows. Windows were never permitted to cross capture-file boundaries. For a complete window containing W consecutive flows, the predicted class probabilities were averaged as

$${\overline{\mathbf{p}}}_W={\displaystyle \frac{1}{W}}{\displaystyle \sum _{i=1}^W}{\mathbf{p}}_i,$$

(4)

where ${\overline{\mathbf{p}}}_W$ represents the aggregated probability vector for the window.

The final application prediction was obtained as

$${\widehat{y}}_W=arg\underset{c\in \{1,\dots ,C\}}{max}{\overline{p}}_{W,c}.$$

(5)

Only complete windows were retained. Residual flows that could not form a complete window at the end of a capture were discarded. Because each capture file represented a controlled collection session for a single target application, each constructed window was additionally checked for label consistency.

4.5. Validation-Based Multi-Flow Window Selection

The number of flows used for multi-flow aggregation was treated as an operating parameter and selected exclusively from the validation partition. The candidate window sizes were

$$W\in \{1,20,40,60,80\}.$$

(6)

Here, $W=1$ represents the conventional single-flow condition, while progressively larger values quantify the effect of accumulating additional encrypted-flow evidence.

All candidate window sizes were evaluated on the fixed validation partition for each of the five classifiers and each of the five random seeds. For a model m and candidate window W, validation Macro-F1 was first averaged across the five seeds:

$${\overline{F1}}_{m,W}^{\mathrm{val}}={\displaystyle \frac{1}{S}}{\displaystyle \sum _{s=1}^S}F{1}_{m,W,s}^{\mathrm{val}},$$

(7)

where $F{1}_{m,W,s}^{\mathrm{val}}$ represents the validation Macro-F1 obtained by model m at window size W using random seed s, and $S=5$ denotes the total number of random-seed runs. Thus, ${\overline{F1}}_{m,W}^{\mathrm{val}}$ represents the mean validation performance of a particular model–window combination across the five seeds, reducing the influence of variation caused by a single random initialization or training run.

To avoid selecting a window that was optimized for only one classifier, a common validation score was subsequently calculated by averaging the seed-averaged Macro-F1 values across all five benchmark models:

$$F{1}_W^{\mathrm{common}}={\displaystyle \frac{1}{M}}{\displaystyle \sum _{m=1}^M}{\overline{F1}}_{m,W}^{\mathrm{val}},$$

(8)

where $M=5$ represents the number of benchmark classifiers. Here, $F{1}_W^{\mathrm{common}}$ measures the average validation Macro-F1 achieved by all five models at a given window size W. This provides a model-independent criterion for comparing the candidate window sizes rather than allowing the strongest result from a single classifier to determine the operating window.

The common operating window was then selected as

$$W^{*}=arg\underset{W}{max}F{1}_W^{\mathrm{common}},$$

(9)

where $W^{*}$ denotes the candidate window size with the highest common validation Macro-F1. In other words, the selected operating window is the window that provides the strongest average validation performance across all five classifiers after accounting for variation across the five random seeds.

Once selected, $W^{*}$ was locked before final Day 5 performance was examined. This procedure separates operating-point selection from temporal test evaluation and prevents the held-out test day from influencing the choice of multi-flow configuration.

4.6. Final Held-Out Day Evaluation

Following validation-based selection, all five classifiers were evaluated on the held-out Day 5 data. Final testing was restricted to two predefined conditions:

$$W\in \{1,W^{*}\}.$$

(10)

The $W=1$ condition provides the single-flow baseline, while $W=W^{*}$ represents the validation-selected multi-flow operating point. Candidate window sizes that were not selected during validation were not used as final Day 5 operating points.

This design permits a direct comparison between conventional single-flow classification and the locked multi-flow strategy while preserving the independence of Day 5 from window selection. The same locked $W^{*}$ was applied to all five classifiers to ensure a common operating condition rather than selecting a separate test configuration for each model.

For both final conditions, performance was computed independently for each of the five random-seed runs and summarized using the mean and standard deviation. The number of complete evaluation windows was also recorded because larger values of W reduce the number of independent classification units.

4.7. Training Configuration

The benchmark models were configured using the hyperparameters summarized in Table 3. All five classifiers used the same fixed training, validation, and Day 5 test partitions and the same training-fitted preprocessing pipeline. The five random seeds were used to quantify variation attributable to model stochasticity rather than to generate alternative dataset partitions.

XGBoost, LightGBM, and CatBoost used the fixed external validation partition for early stopping. HistGradientBoosting instead used an internal 15% subset of the training partition solely for early stopping, while its reported validation performance and multi-flow window selection were computed using the same fixed external validation partition as the other models. Random Forest did not use early stopping.

4.8. Explainability Analysis

Post-hoc explainability analysis was performed using SHAP and LIME after completion of the final Day 5 evaluation. The model achieving the highest mean Macro-F1 across the five random-seed runs was selected as the focal model for explainability analysis.

The explainability stage was therefore conducted after model evaluation and did not influence model training, hyperparameter configuration, validation-based window selection, or the reported Day 5 performance. Its purpose was to characterize the feature-level behavior of the strongest final classifier under the locked multi-flow configuration.

For reproducible explanation generation, the selected model was refitted using seed 42 under the same preprocessing and training configuration. SHAP and LIME were then applied to correctly classified individual Day 5 flows. The explanations characterize the flow-level classifier that produces the probability vectors subsequently combined during multi-flow inference, rather than treating each aggregated window as a separately trained model.

For SHAP, up to 100 correctly classified Day 5 flows per application class were analyzed. Features were ranked according to their mean absolute SHAP values, and the 15 highest-ranked features were retained for each class.

For LIME, up to 25 correctly classified Day 5 flows per application class were explained using 3,000 perturbation samples per instance. Absolute local feature contributions were aggregated using zero-inclusive averaging, meaning that features not selected in a particular local explanation contributed a value of zero to the class-level average. The 15 highest-ranked features were retained for each application class.

Agreement between SHAP and LIME was examined using Jaccard similarity between their class-specific top-15 feature sets. Both exact agreement, based on individual encoded feature names, and semantic agreement, based on their underlying feature families, were evaluated to determine the extent to which the two explanation methods identified similar traffic characteristics.

4.9. Evaluation Metrics

Performance was evaluated using accuracy, macro-averaged precision, macro-averaged recall, Macro F1-score, Weighted F1-score, and multiclass log loss. Macro F1 served as the primary metric for validation-based operating-window selection because it gives equal importance to each application class irrespective of class frequency.

Macro F1 was computed as

$$\mathrm{Macro}-\mathrm{F}1={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{\mathrm{F}1}_c,$$

(11)

where C is the number of application classes.

Weighted F1 was computed as

$$\mathrm{Weighted}-\mathrm{F}1={\displaystyle \sum _{c=1}^C}{\displaystyle \frac{n_c}{N}}{\mathrm{F}1}_c,$$

(12)

where $n_c$ is the number of observations belonging to class c and N is the total number of observations.

Multiclass log loss was additionally calculated from the predicted class probabilities:

$${\mathcal{L}}_{\log }=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{c=1}^C}{y}_{i,c}log\left(p_{i,c}\right),$$

(13)

where $y_{i,c}$ is an indicator equal to 1 when observation i belongs to class c, and $p_{i,c}$ is the predicted probability assigned to that class.

Validation performance was summarized across the five random-seed runs for every model and candidate window. Final Day 5 performance was reported only for the single-flow baseline ($W=1$) and the validation-selected common operating window ($W=W^{*}$), using mean and standard deviation across the five seeds. Per-class precision, recall, and F1-scores and class-normalized confusion matrices were additionally used to examine application-specific behavior and systematic confusion between application classes.

5. Results

This section presents the experimental results obtained under the validation-controlled temporal evaluation protocol. Results are reported across five random seeds for the five benchmark models: CatBoost, HistGradientBoosting, LightGBM, Random Forest, and XGBoost.

Multi-flow window selection was performed exclusively on the validation partition derived from Days 1–4. Five candidate window sizes, $W\in \{1,20,40,60,80\}$, were evaluated, where $W=1$ represents conventional single-flow classification and larger windows aggregate the predicted probability distributions of consecutive flows from the same capture file. A single common operating window, $W^{*}$, was selected using the mean validation Macro-F1 across the five benchmark models and five random-seed runs.

After selection, $W^{*}$ was locked before final evaluation on the temporally held-out Day 5 data. Consequently, the final Day 5 evaluation reports only two conditions for each model: the predefined single-flow baseline ($W=1$) and the validation-selected multi-flow configuration ($W=W^{*}$). This separation ensures that the held-out Day 5 test set does not influence the selection of the multi-flow operating window.

5.1. Validation-Based Multi-Flow Ablation Results

Table 4 reports the multi-flow ablation results obtained on the validation partition for all five benchmark models and six candidate window sizes, $W\in \{1,20,40,60,80\}$. Results are reported as mean ± standard deviation across the five random-seed runs. The validation experiment was used exclusively to determine the common multi-flow operating window before evaluation on the held-out Day 5 test set.

The single-flow condition ($W=1$) produced relatively modest validation performance across all five classifiers, with mean Macro F1-scores ranging from $0.3635$ for CatBoost to $0.3841$ for XGBoost. Substantial improvement was observed when multiple consecutive flows were aggregated. At $W=20$, Macro F1 increased to between $0.7392$ and $0.8359$, demonstrating that combining evidence from multiple encrypted flows considerably improved application discrimination.

Performance improved further for several models at $W=40$. Random Forest achieved the highest Macro F1 at this window with $0.9227$, followed by LightGBM with $0.8960$. The strongest and most consistent overall validation performance was observed at $W=60$, where LightGBM and Random Forest both achieved a Macro F1 of $0.9467$, while CatBoost, HistGradientBoosting, and XGBoost also exceeded $0.90$.

Increasing the aggregation window to $W=80$ did not provide further improvement. Macro F1 decreased across all five classifiers despite relatively high accuracy values. The evaluation at this larger window also contained only 14 complete validation windows, reducing class coverage and making aggregate accuracy less informative. These results indicate that the benefit of increasing multi-flow context peaked at approximately 60 consecutive flows under the validation setting.

To avoid selecting a window based on the performance of a single classifier, a common validation score was calculated by averaging the seed-averaged Macro F1 of all five models at each candidate window. The resulting common validation scores are summarized in Table 5.

The common validation Macro F1 reached its maximum at $W=60$, with a score of $0.9236$. Consequently, the 60-flow configuration was selected as the common operating window:

$$W^{*}=60.$$

(14)

This operating point was locked before final evaluation on the held-out Day 5 test set. The final temporal evaluation therefore compares the predefined single-flow baseline ($W=1$) against the validation-selected multi-flow configuration ($W=60$) for all five benchmark models.

5.2. Held-Out Day 5 Test Results

Following validation-based window selection, the locked multi-flow configuration ($W^{*}=60$) was evaluated on the temporally held-out Day 5 test set. No additional window-size selection or tuning was performed using Day 5. Final performance was therefore evaluated under only two conditions: the predefined single-flow baseline ($W=1$) and the validation-selected multi-flow configuration ($W=60$). Figure 2 and Figure 3 compare the mean test accuracy and Macro F1-score, respectively, across the five random-seed runs.

Under the single-flow condition ($W=1$), performance on the temporally held-out Day 5 data was relatively modest across all five classifiers. Accuracy ranged from $0.3187$ for CatBoost to $0.3431$ for LightGBM, while Macro F1 ranged from $0.2979$ to $0.3285$. LightGBM achieved the strongest single-flow performance, with an accuracy of $0.3431\pm 0.0015$ and a Macro F1-score of $0.3285\pm 0.0016$. These results demonstrate the difficulty of identifying mobile social media applications from individual encrypted network flows under temporal separation between the development and test data.

As shown in Figure 2 and Figure 3, applying the validation-selected 60-flow aggregation substantially improved performance for every classifier. Mean accuracy increased to between $0.6511$ and $0.6832$, while Macro F1 increased to between $0.6671$ and $0.7317$. Importantly, these improvements were obtained without retraining the classifiers for multi-flow inference; the same flow-level models were used, with their predicted probability distributions aggregated across consecutive Day 5 flows.

XGBoost achieved the highest Macro F1 at the locked $W=60$ operating point, increasing from $0.3155\pm 0.0018$ under single-flow classification to $0.7317\pm 0.0077$. CatBoost produced a closely comparable Macro F1 of $0.7281\pm 0.0090$, compared with $0.2979\pm 0.0021$ at $W=1$. HistGradientBoosting increased from $0.3191\pm 0.0092$ to $0.7047\pm 0.0116$, while LightGBM increased from $0.3285\pm 0.0016$ to $0.6929\pm 0.0114$. Random Forest similarly improved from $0.3189\pm 0.0011$ to $0.6671\pm 0.0041$.

For accuracy, LightGBM, Random Forest, and XGBoost each achieved a mean of $0.6832$ at $W=60$, compared with single-flow accuracies of $0.3431$, $0.3319$, and $0.3342$, respectively. CatBoost and HistGradientBoosting also exhibited substantial gains, reaching mean accuracies of $0.6511$ and $0.6672$, respectively.

The consistent improvement across all five classifiers indicates that the observed benefit is not specific to a particular learning algorithm. Instead, the results support the central premise of the multi-flow approach: aggregating predictions across consecutive encrypted flows provides a more informative representation of application behavior than relying on isolated flows. Although performance on Day 5 remained below the corresponding validation results, the substantial improvement over the single-flow baseline demonstrates that the validation-selected operating point generalized to traffic collected during an unseen capture period.

5.3. Per-Class Performance

Detailed per-class results for all five models under the two final Day 5 evaluation conditions, the single-flow baseline ($W=1$) and the validation-selected multi-flow configuration ($W=60$), are provided in Appendix A.1. The appendix reports the mean and standard deviation of precision, recall, and F1-score across the five random seeds for each of the ten application classes.

These per-class results complement the aggregate Day 5 performance presented in Figure 2 and Figure 3 by showing how classification performance varies across individual applications under single-flow and multi-flow inference. This class-level analysis is particularly important for interpreting the Macro F1-score, which assigns equal importance to each of the ten application classes and can therefore reveal class-specific differences that may not be apparent from overall accuracy alone. Comparing $W=1$ and $W=60$ also provides insight into which applications benefit most from aggregating evidence across multiple consecutive encrypted flows.

5.4. Confusion Matrix Analysis

To examine the class-specific effect of multi-flow aggregation, Figure 4 presents the class-normalized confusion matrices for XGBoost under the two final Day 5 evaluation conditions: the single-flow baseline ($W=1$) and the validation-selected multi-flow configuration ($W=60$). XGBoost was selected for this detailed analysis because it achieved the highest Macro F1-score on the held-out Day 5 test set at the locked $W=60$ operating point.

The single-flow confusion matrix reveals substantial cross-application confusion on the temporally held-out Day 5 data. At $W=1$, only Facebook and Reddit exceeded 50% class-specific recall, with diagonal rates of 60.1% and 58.7%, respectively. Several applications were considerably more difficult to identify from individual encrypted flows. Instagram, LinkedIn, Snapchat, WhatsApp, and YouTube achieved diagonal rates of only 21.8%, 19.5%, 20.3%, 22.7%, and 28.0%, respectively. Telegram, TikTok, and Twitter performed comparatively better, but their correct-classification rates remained below 50%.

The $W=60$ configuration substantially changed this pattern. Facebook, Reddit, Telegram, TikTok, and Twitter achieved 100% diagonal rates, while Snapchat and YouTube increased to 85.7% and 87.0%, respectively. Instagram improved from 21.8% at $W=1$ to 51.4% at $W=60$, and LinkedIn increased from 19.5% to 57.1%. These changes show that aggregating evidence across consecutive encrypted flows substantially reduced the diffuse cross-application confusion observed under single-flow classification.

The improvement was particularly pronounced for applications that were difficult to distinguish from isolated flows. For example, Telegram increased from 38.0% to 100%, TikTok from 47.1% to 100%, Twitter from 46.2% to 100%, and YouTube from 28.0% to 87.0%. Snapchat similarly increased from 20.3% to 85.7%. Thus, the gain in aggregate Macro F1 at $W=60$ reflects improvements across multiple application classes rather than being driven by a single dominant class.

Despite these improvements, some class-specific ambiguity remained. WhatsApp was the most difficult application under the multi-flow configuration, with a diagonal rate of 31.2%. Its largest residual confusion was with Twitter (37.5%), followed by LinkedIn (14.6%) and Reddit (11.2%). Instagram also retained confusion with Facebook (25.7%) and Twitter (22.9%), while LinkedIn exhibited residual confusion with Instagram, Telegram, and WhatsApp (14.3% each). YouTube was predominantly identified correctly, although 8.7% of its windows were classified as WhatsApp and 4.3% as Twitter.

Overall, the confusion-matrix comparison demonstrates that the improvement from single-flow to multi-flow classification is associated with substantially stronger class separation for most applications. At the same time, the remaining errors, particularly those involving WhatsApp, Instagram, and LinkedIn, indicate that some encrypted application behaviors remain difficult to distinguish under temporal separation. These residual confusions provide a more nuanced interpretation of the Day 5 Macro F1 result and identify specific application pairs for which additional contextual information may be beneficial.

6. Explainable AI Analysis

Explainable AI analysis was performed using SHAP and LIME to examine the feature-level behavior of the highest-performing classifier in the final temporally held-out Day 5 evaluation. The operating window was first selected and locked at $W=60$ using only the Days 1–4 validation partition. All five classifiers were subsequently evaluated on the held-out Day 5 test set at the locked operating window, and the classifier achieving the highest mean Macro F1-score across the five random-seed runs was selected for post-hoc explainability analysis.

XGBoost achieved the highest Day 5 Macro F1-score at the locked $W=60$ configuration and was therefore selected as the focal model for SHAP and LIME analysis. Importantly, this post-hoc selection was used only to determine which final classifier to interpret. It did not influence model training, hyperparameter configuration, validation-based window selection, or the Day 5 predictions and performance measures reported in the primary classification evaluation.

6.1. SHAP Analysis

SHAP (SHapley Additive exPlanations) is a post-hoc explainability method based on game-theoretic principles that assigns feature-level contributions to model predictions [27].

SHAP was applied to the XGBoost classifier to identify the features contributing most strongly to its correctly classified predictions on the temporally held-out Day 5 test set. For each application class, mean absolute SHAP values were calculated across up to 100 correctly classified flows, and the 15 highest-ranked features were retained.

Table 6 presents the top-15 SHAP feature rankings for the ten application classes. The analysis provides a class-specific view of the network-flow characteristics associated with the model’s classification decisions. In particular, the rankings capture the relative influence of the statistical, protocol, timing, and encrypted fingerprint features represented in the feature space. Differences in the resulting rankings across application classes provide insight into the heterogeneous encrypted-traffic characteristics used by XGBoost when distinguishing mobile social media applications.

6.2. LIME Analysis

LIME (Local Interpretable Model-agnostic Explanations) is a post-hoc, model-agnostic explainability method that explains an individual prediction by approximating the behavior of a complex classifier with a simpler interpretable model in the local neighborhood of the instance [27].

LIME was applied to the XGBoost classifier as a complementary local explanation technique. For each application class, up to 25 correctly classified Day 5 flows were explained. The absolute local LIME weights were aggregated using zero-inclusive averaging, such that features not selected in a particular local explanation contributed a value of zero to the corresponding class-level average. Table 7 reports the 15 highest-ranked LIME features for each application class. The occurrence rate indicates the proportion of the analyzed local explanations in which each feature was selected.

The LIME results complement the SHAP findings, with packet-size statistics, directional traffic characteristics, timing information, protocol-related features, and encrypted client/server fingerprints appearing among the influential attributes across application classes. Differences in the exact rankings are expected because LIME explains model behavior around individual instances, whereas the reported SHAP rankings summarize feature contributions across multiple correctly classified flows. The recurrence of similar feature categories across the two methods provides complementary evidence about the encrypted traffic characteristics influencing XGBoost’s classification decisions.

6.3. SHAP–LIME Agreement

The consistency of the SHAP and LIME explanations was evaluated using Jaccard similarity between their respective top-15 feature sets. Two forms of agreement were calculated. Exact agreement compared the individual encoded feature names directly, whereas semantic agreement grouped encoded categorical variants under their original feature family. For example, different encoded values of client_fingerprint were treated as members of the common semantic feature client_fingerprint.

Figure 5 presents the class-level agreement results for the selected XGBoost model. The agreement varied across application classes, with semantic agreement generally exceeding exact agreement. Twitter exhibited the highest semantic agreement at 0.2000, followed by Telegram at approximately 0.1765 and TikTok at approximately 0.1667. Reddit and Twitter showed the highest exact agreement at approximately 0.1538. In contrast, Facebook and Instagram showed no exact feature overlap between their SHAP and LIME top-15 rankings, although Facebook retained some agreement after the features were grouped by semantic family.

The differences between exact and semantic agreement reflect the large number of encoded categorical fingerprint features used by the model. SHAP and LIME may identify different individual fingerprint values as important while still emphasizing the same underlying feature family. This is particularly visible for several application classes where semantic agreement exceeds exact agreement.

The relatively modest agreement between SHAP and LIME is also consistent with their different explanation mechanisms. SHAP assigns feature contributions using Shapley-value-based attribution, whereas LIME constructs local surrogate models around individual predictions. The two methods therefore do not necessarily select identical features even when explaining the same classifier. Their agreement at the semantic level provides additional evidence that both methods capture related categories of information used by XGBoost, particularly encrypted fingerprint attributes together with traffic, timing, and protocol characteristics.

7. Discussion

The primary novelty of this work is the construction of a new benchmark dataset for encrypted mobile social media traffic fingerprinting. Rather than relying on an existing network intrusion or general traffic dataset, traffic was collected specifically from ten widely used mobile social media applications across multiple capture sessions and transformed from raw packet captures into labeled flow-level records for application fingerprinting research. The resulting benchmark captures contemporary encrypted communication, including TLS- and QUIC-based traffic, while retaining statistical, directional, protocol, timing, and encrypted fingerprint information that can be examined without accessing application payloads.

The benchmark was also designed to reduce dependence on trivial application identifiers. Application labels, endpoint information, absolute timestamps, and other attributes capable of directly revealing the capture source were excluded from the predictive feature space. The temporal evaluation further strengthens this design: Days 1–4 were used for model development, while Day 5 remained unseen until final testing. Consequently, the reported test performance reflects application discrimination across capture sessions rather than performance obtained from a conventional random split of the complete dataset.

The experiments reveal an important characteristic of the new dataset. Individual encrypted flows provided limited evidence for distinguishing the ten applications, whereas combining consecutive flows produced substantially stronger application-level discrimination. The aggregation window was determined using only validation data from Days 1–4, where the 60-flow window was selected and locked before Day 5 testing. The improvement from the single-flow baseline to the locked multi-flow configuration on Day 5 suggests that application behavior is represented more effectively by sequences of encrypted communications than by isolated flows. This is relevant because a mobile application may generate multiple connections for content delivery, API communication, advertisements, media transfer, and background services.

The SHAP and LIME analyses provide further insight into the information captured by the benchmark. Application discrimination was associated with combinations of encrypted client/server fingerprints, packet-size statistics, timing characteristics, protocol information, and directional traffic behavior. Different applications relied on different combinations of these characteristics, suggesting that identification was not driven by a single universally dominant feature.

These findings also have practical relevance. Encrypted flow analysis could provide network administrators with additional visibility into mobile application activity without decrypting communication payloads. It could support identification of applications restricted by organizational policy, characterization of application-specific network usage, and retrospective analysis of historical network-flow records. In a forensic investigation, such classification could provide supporting evidence that network activity was consistent with a particular mobile application during a period of interest. It should not, however, be interpreted as evidence of specific user actions or communication content.

8. Limitations

Several limitations should be considered when interpreting the findings of this study.

First, the dataset was collected under controlled conditions using a single Android device and network environment. Although the temporally held-out Day 5 evaluation measures performance across capture sessions, all five capture days originated from the same experimental environment. Traffic characteristics may differ across devices, operating system versions, network providers, geographic locations, and routing conditions, and cross-environment generalization therefore remains to be evaluated.

Second, the dataset covers ten social media applications across five capture days. Application updates, changes in TLS/QUIC implementations, backend infrastructure, and content delivery networks may alter observable traffic characteristics over longer periods.

Third, the dataset contains some class imbalance. Macro-averaged metrics were therefore reported alongside accuracy and Weighted F1 to provide a balanced assessment of performance across the ten application classes.

Finally, multi-flow aggregation introduces a trade-off between additional traffic context and the number of available prediction windows. The validation study selected a 60-flow window, which substantially improved classification compared with the single-flow baseline, but requires more traffic to be observed before an application-level prediction can be produced. This trade-off should be considered in applications where rapid identification from limited traffic is required.

Future work will extend data collection across additional devices, operating systems, network environments, geographic locations, and longer time periods to evaluate the generalization of the identified application fingerprints beyond the conditions examined in this study.

9. Conclusion

This study introduced a publicly available benchmark dataset for fingerprinting encrypted mobile social media traffic. The benchmark was constructed from ten mobile applications collected across multiple capture sessions and provides flow-level statistical, directional, timing, protocol, and encrypted fingerprint features while excluding direct application identifiers and other leakage-prone information.

The experiments showed that individual encrypted flows provide limited information for application identification, while aggregating consecutive flows substantially improves performance. A 60-flow window was selected using validation data from Days 1–4 and locked before evaluation on the temporally held-out Day 5 test set. At this operating point, all five classifiers improved considerably over their single-flow baselines, with XGBoost achieving the highest test Macro F1-score.

SHAP and LIME analysis of the XGBoost classifier showed that application identification was influenced by combinations of encrypted client/server fingerprints, packet-size statistics, timing characteristics, protocol information, and directional traffic behavior. These findings demonstrate that useful application-specific patterns remain observable in encrypted TLS/QUIC traffic without inspecting communication payloads. The benchmark therefore provides a reproducible resource for studying encrypted mobile application fingerprinting and its potential use in network monitoring and digital investigations.

Appendix A.1. Per-Class Results

Table A1. Per-class performance of CatBoost at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3511\pm 0.0110$	$0.6356\pm 0.0214$	$0.4523\pm 0.0137$	60	Facebook	$0.8790\pm 0.0277$	$1.0000\pm 0.0000$	$0.9354\pm 0.0154$
1	Instagram	$0.2244\pm 0.0026$	$0.1728\pm 0.0039$	$0.1952\pm 0.0025$	60	Instagram	$0.9000\pm 0.1369$	$0.4000\pm 0.0639$	$0.5471\pm 0.0635$
1	LinkedIn	$0.1153\pm 0.0050$	$0.1552\pm 0.0034$	$0.1323\pm 0.0044$	60	LinkedIn	$0.7733\pm 0.0596$	$0.5714\pm 0.0000$	$0.6564\pm 0.0229$
1	Reddit	$0.4711\pm 0.0117$	$0.5760\pm 0.0056$	$0.5183\pm 0.0083$	60	Reddit	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Snapchat	$0.1426\pm 0.0030$	$0.1609\pm 0.0041$	$0.1512\pm 0.0024$	60	Snapchat	$0.6833\pm 0.0864$	$0.7714\pm 0.0782$	$0.7205\pm 0.0498$
1	Telegram	$0.1135\pm 0.0006$	$0.3450\pm 0.0037$	$0.1708\pm 0.0009$	60	Telegram	$0.6333\pm 0.0745$	$1.0000\pm 0.0000$	$0.7733\pm 0.0596$
1	TikTok	$0.4591\pm 0.0077$	$0.4142\pm 0.0049$	$0.4355\pm 0.0059$	60	TikTok	$0.9800\pm 0.0447$	$1.0000\pm 0.0000$	$0.9895\pm 0.0235$
1	Twitter	$0.2195\pm 0.0034$	$0.4404\pm 0.0026$	$0.2930\pm 0.0035$	60	Twitter	$0.2316\pm 0.0045$	$1.0000\pm 0.0000$	$0.3760\pm 0.0059$
1	WhatsApp	$0.3317\pm 0.0019$	$0.2230\pm 0.0011$	$0.2667\pm 0.0013$	60	WhatsApp	$0.6606\pm 0.0136$	$0.2917\pm 0.0000$	$0.4046\pm 0.0026$
1	YouTube	$0.6340\pm 0.0102$	$0.2554\pm 0.0070$	$0.3641\pm 0.0066$	60	YouTube	$1.0000\pm 0.0000$	$0.7826\pm 0.0000$	$0.8780\pm 0.0000$

Table A1. Per-class performance of CatBoost at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3511\pm 0.0110$	$0.6356\pm 0.0214$	$0.4523\pm 0.0137$	60	Facebook	$0.8790\pm 0.0277$	$1.0000\pm 0.0000$	$0.9354\pm 0.0154$
1	Instagram	$0.2244\pm 0.0026$	$0.1728\pm 0.0039$	$0.1952\pm 0.0025$	60	Instagram	$0.9000\pm 0.1369$	$0.4000\pm 0.0639$	$0.5471\pm 0.0635$
1	LinkedIn	$0.1153\pm 0.0050$	$0.1552\pm 0.0034$	$0.1323\pm 0.0044$	60	LinkedIn	$0.7733\pm 0.0596$	$0.5714\pm 0.0000$	$0.6564\pm 0.0229$
1	Reddit	$0.4711\pm 0.0117$	$0.5760\pm 0.0056$	$0.5183\pm 0.0083$	60	Reddit	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Snapchat	$0.1426\pm 0.0030$	$0.1609\pm 0.0041$	$0.1512\pm 0.0024$	60	Snapchat	$0.6833\pm 0.0864$	$0.7714\pm 0.0782$	$0.7205\pm 0.0498$
1	Telegram	$0.1135\pm 0.0006$	$0.3450\pm 0.0037$	$0.1708\pm 0.0009$	60	Telegram	$0.6333\pm 0.0745$	$1.0000\pm 0.0000$	$0.7733\pm 0.0596$
1	TikTok	$0.4591\pm 0.0077$	$0.4142\pm 0.0049$	$0.4355\pm 0.0059$	60	TikTok	$0.9800\pm 0.0447$	$1.0000\pm 0.0000$	$0.9895\pm 0.0235$
1	Twitter	$0.2195\pm 0.0034$	$0.4404\pm 0.0026$	$0.2930\pm 0.0035$	60	Twitter	$0.2316\pm 0.0045$	$1.0000\pm 0.0000$	$0.3760\pm 0.0059$
1	WhatsApp	$0.3317\pm 0.0019$	$0.2230\pm 0.0011$	$0.2667\pm 0.0013$	60	WhatsApp	$0.6606\pm 0.0136$	$0.2917\pm 0.0000$	$0.4046\pm 0.0026$
1	YouTube	$0.6340\pm 0.0102$	$0.2554\pm 0.0070$	$0.3641\pm 0.0066$	60	YouTube	$1.0000\pm 0.0000$	$0.7826\pm 0.0000$	$0.8780\pm 0.0000$

Table A2. Per-class performance of HistGradientBoosting at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3736\pm 0.0084$	$0.6162\pm 0.0137$	$0.4651\pm 0.0092$	60	Facebook	$0.8354\pm 0.0460$	$1.0000\pm 0.0000$	$0.9098\pm 0.0278$
1	Instagram	$0.2068\pm 0.0146$	$0.1972\pm 0.0117$	$0.2015\pm 0.0087$	60	Instagram	$0.8200\pm 0.1754$	$0.4286\pm 0.0000$	$0.5582\pm 0.0424$
1	LinkedIn	$0.1227\pm 0.0047$	$0.1783\pm 0.0070$	$0.1453\pm 0.0051$	60	LinkedIn	$0.3040\pm 0.0197$	$0.5714\pm 0.0000$	$0.3966\pm 0.0167$
1	Reddit	$0.4294\pm 0.0164$	$0.5850\pm 0.0037$	$0.4951\pm 0.0115$	60	Reddit	$0.6972\pm 0.0627$	$0.9818\pm 0.0407$	$0.8137\pm 0.0436$
1	Snapchat	$0.1371\pm 0.0107$	$0.2342\pm 0.0389$	$0.1713\pm 0.0062$	60	Snapchat	$0.7373\pm 0.1293$	$0.8286\pm 0.0639$	$0.7779\pm 0.0964$
1	Telegram	$0.1919\pm 0.1139$	$0.3772\pm 0.0056$	$0.2388\pm 0.0951$	60	Telegram	$0.6333\pm 0.2173$	$1.0000\pm 0.0000$	$0.7600\pm 0.1461$
1	TikTok	$0.5110\pm 0.0192$	$0.4513\pm 0.0072$	$0.4791\pm 0.0077$	60	TikTok	$0.9600\pm 0.0548$	$1.0000\pm 0.0000$	$0.9789\pm 0.0288$
1	Twitter	$0.2574\pm 0.0102$	$0.4489\pm 0.0184$	$0.3269\pm 0.0076$	60	Twitter	$0.3400\pm 0.0310$	$1.0000\pm 0.0000$	$0.5069\pm 0.0342$
1	WhatsApp	$0.3773\pm 0.0136$	$0.2333\pm 0.0043$	$0.2881\pm 0.0025$	60	WhatsApp	$0.7570\pm 0.0508$	$0.2958\pm 0.0093$	$0.4253\pm 0.0169$
1	YouTube	$0.6313\pm 0.0258$	$0.2717\pm 0.0189$	$0.3798\pm 0.0224$	60	YouTube	$1.0000\pm 0.0000$	$0.8522\pm 0.0238$	$0.9200\pm 0.0140$

Table A2. Per-class performance of HistGradientBoosting at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3736\pm 0.0084$	$0.6162\pm 0.0137$	$0.4651\pm 0.0092$	60	Facebook	$0.8354\pm 0.0460$	$1.0000\pm 0.0000$	$0.9098\pm 0.0278$
1	Instagram	$0.2068\pm 0.0146$	$0.1972\pm 0.0117$	$0.2015\pm 0.0087$	60	Instagram	$0.8200\pm 0.1754$	$0.4286\pm 0.0000$	$0.5582\pm 0.0424$
1	LinkedIn	$0.1227\pm 0.0047$	$0.1783\pm 0.0070$	$0.1453\pm 0.0051$	60	LinkedIn	$0.3040\pm 0.0197$	$0.5714\pm 0.0000$	$0.3966\pm 0.0167$
1	Reddit	$0.4294\pm 0.0164$	$0.5850\pm 0.0037$	$0.4951\pm 0.0115$	60	Reddit	$0.6972\pm 0.0627$	$0.9818\pm 0.0407$	$0.8137\pm 0.0436$
1	Snapchat	$0.1371\pm 0.0107$	$0.2342\pm 0.0389$	$0.1713\pm 0.0062$	60	Snapchat	$0.7373\pm 0.1293$	$0.8286\pm 0.0639$	$0.7779\pm 0.0964$
1	Telegram	$0.1919\pm 0.1139$	$0.3772\pm 0.0056$	$0.2388\pm 0.0951$	60	Telegram	$0.6333\pm 0.2173$	$1.0000\pm 0.0000$	$0.7600\pm 0.1461$
1	TikTok	$0.5110\pm 0.0192$	$0.4513\pm 0.0072$	$0.4791\pm 0.0077$	60	TikTok	$0.9600\pm 0.0548$	$1.0000\pm 0.0000$	$0.9789\pm 0.0288$
1	Twitter	$0.2574\pm 0.0102$	$0.4489\pm 0.0184$	$0.3269\pm 0.0076$	60	Twitter	$0.3400\pm 0.0310$	$1.0000\pm 0.0000$	$0.5069\pm 0.0342$
1	WhatsApp	$0.3773\pm 0.0136$	$0.2333\pm 0.0043$	$0.2881\pm 0.0025$	60	WhatsApp	$0.7570\pm 0.0508$	$0.2958\pm 0.0093$	$0.4253\pm 0.0169$
1	YouTube	$0.6313\pm 0.0258$	$0.2717\pm 0.0189$	$0.3798\pm 0.0224$	60	YouTube	$1.0000\pm 0.0000$	$0.8522\pm 0.0238$	$0.9200\pm 0.0140$

Table A3. Per-class performance of LightGBM at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.4531\pm 0.0071$	$0.5490\pm 0.0094$	$0.4964\pm 0.0070$	60	Facebook	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Instagram	$0.1876\pm 0.0022$	$0.3568\pm 0.0083$	$0.2459\pm 0.0036$	60	Instagram	$0.5872\pm 0.0307$	$0.7714\pm 0.0782$	$0.6657\pm 0.0404$
1	LinkedIn	$0.1138\pm 0.0019$	$0.2998\pm 0.0104$	$0.1650\pm 0.0035$	60	LinkedIn	$0.1762\pm 0.0053$	$0.7143\pm 0.0000$	$0.2826\pm 0.0069$
1	Reddit	$0.4622\pm 0.0106$	$0.5726\pm 0.0033$	$0.5115\pm 0.0073$	60	Reddit	$0.8408\pm 0.0497$	$0.9455\pm 0.0498$	$0.8891\pm 0.0368$
1	Snapchat	$0.1500\pm 0.0060$	$0.2166\pm 0.0074$	$0.1773\pm 0.0061$	60	Snapchat	$1.0000\pm 0.0000$	$0.4286\pm 0.1010$	$0.5943\pm 0.1003$
1	Telegram	$0.1026\pm 0.0010$	$0.4174\pm 0.0056$	$0.1647\pm 0.0013$	60	Telegram	$0.2643\pm 0.0196$	$1.0000\pm 0.0000$	$0.4178\pm 0.0243$
1	TikTok	$0.3838\pm 0.0063$	$0.5207\pm 0.0038$	$0.4419\pm 0.0052$	60	TikTok	$0.8345\pm 0.0366$	$1.0000\pm 0.0000$	$0.9095\pm 0.0212$
1	Twitter	$0.3264\pm 0.0058$	$0.3826\pm 0.0031$	$0.3523\pm 0.0040$	60	Twitter	$0.6126\pm 0.0174$	$0.9800\pm 0.0447$	$0.7537\pm 0.0225$
1	WhatsApp	$0.5811\pm 0.0084$	$0.1964\pm 0.0005$	$0.2935\pm 0.0011$	60	WhatsApp	$1.0000\pm 0.0000$	$0.2917\pm 0.0000$	$0.4516\pm 0.0000$
1	YouTube	$0.5381\pm 0.0050$	$0.3670\pm 0.0102$	$0.4363\pm 0.0080$	60	YouTube	$0.9739\pm 0.0238$	$0.9565\pm 0.0000$	$0.9650\pm 0.0116$

Table A3. Per-class performance of LightGBM at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.4531\pm 0.0071$	$0.5490\pm 0.0094$	$0.4964\pm 0.0070$	60	Facebook	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Instagram	$0.1876\pm 0.0022$	$0.3568\pm 0.0083$	$0.2459\pm 0.0036$	60	Instagram	$0.5872\pm 0.0307$	$0.7714\pm 0.0782$	$0.6657\pm 0.0404$
1	LinkedIn	$0.1138\pm 0.0019$	$0.2998\pm 0.0104$	$0.1650\pm 0.0035$	60	LinkedIn	$0.1762\pm 0.0053$	$0.7143\pm 0.0000$	$0.2826\pm 0.0069$
1	Reddit	$0.4622\pm 0.0106$	$0.5726\pm 0.0033$	$0.5115\pm 0.0073$	60	Reddit	$0.8408\pm 0.0497$	$0.9455\pm 0.0498$	$0.8891\pm 0.0368$
1	Snapchat	$0.1500\pm 0.0060$	$0.2166\pm 0.0074$	$0.1773\pm 0.0061$	60	Snapchat	$1.0000\pm 0.0000$	$0.4286\pm 0.1010$	$0.5943\pm 0.1003$
1	Telegram	$0.1026\pm 0.0010$	$0.4174\pm 0.0056$	$0.1647\pm 0.0013$	60	Telegram	$0.2643\pm 0.0196$	$1.0000\pm 0.0000$	$0.4178\pm 0.0243$
1	TikTok	$0.3838\pm 0.0063$	$0.5207\pm 0.0038$	$0.4419\pm 0.0052$	60	TikTok	$0.8345\pm 0.0366$	$1.0000\pm 0.0000$	$0.9095\pm 0.0212$
1	Twitter	$0.3264\pm 0.0058$	$0.3826\pm 0.0031$	$0.3523\pm 0.0040$	60	Twitter	$0.6126\pm 0.0174$	$0.9800\pm 0.0447$	$0.7537\pm 0.0225$
1	WhatsApp	$0.5811\pm 0.0084$	$0.1964\pm 0.0005$	$0.2935\pm 0.0011$	60	WhatsApp	$1.0000\pm 0.0000$	$0.2917\pm 0.0000$	$0.4516\pm 0.0000$
1	YouTube	$0.5381\pm 0.0050$	$0.3670\pm 0.0102$	$0.4363\pm 0.0080$	60	YouTube	$0.9739\pm 0.0238$	$0.9565\pm 0.0000$	$0.9650\pm 0.0116$

Table A4. Per-class performance of RandomForest at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3977\pm 0.0057$	$0.5148\pm 0.0091$	$0.4487\pm 0.0062$	60	Facebook	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Instagram	$0.1919\pm 0.0045$	$0.3812\pm 0.0049$	$0.2553\pm 0.0037$	60	Instagram	$0.5556\pm 0.0000$	$0.7143\pm 0.0000$	$0.6250\pm 0.0000$
1	LinkedIn	$0.1228\pm 0.0043$	$0.2776\pm 0.0030$	$0.1703\pm 0.0039$	60	LinkedIn	$0.3476\pm 0.0130$	$0.7143\pm 0.0000$	$0.4675\pm 0.0119$
1	Reddit	$0.5236\pm 0.0077$	$0.5523\pm 0.0021$	$0.5376\pm 0.0049$	60	Reddit	$0.7543\pm 0.0287$	$1.0000\pm 0.0000$	$0.8597\pm 0.0185$
1	Snapchat	$0.1094\pm 0.0027$	$0.2085\pm 0.0062$	$0.1435\pm 0.0032$	60	Snapchat	$1.0000\pm 0.0000$	$0.2857\pm 0.0000$	$0.4444\pm 0.0000$
1	Telegram	$0.1012\pm 0.0038$	$0.3906\pm 0.0120$	$0.1607\pm 0.0059$	60	Telegram	$0.2714\pm 0.0196$	$1.0000\pm 0.0000$	$0.4267\pm 0.0243$
1	TikTok	$0.3843\pm 0.0036$	$0.5080\pm 0.0010$	$0.4376\pm 0.0027$	60	TikTok	$0.8182\pm 0.0000$	$1.0000\pm 0.0000$	$0.9000\pm 0.0000$
1	Twitter	$0.2813\pm 0.0046$	$0.3744\pm 0.0033$	$0.3212\pm 0.0036$	60	Twitter	$0.3571\pm 0.0000$	$1.0000\pm 0.0000$	$0.5263\pm 0.0000$
1	WhatsApp	$0.5868\pm 0.0085$	$0.1939\pm 0.0009$	$0.2915\pm 0.0020$	60	WhatsApp	$1.0000\pm 0.0000$	$0.2917\pm 0.0000$	$0.4516\pm 0.0000$
1	YouTube	$0.5361\pm 0.0047$	$0.3482\pm 0.0034$	$0.4222\pm 0.0037$	60	YouTube	$0.9576\pm 0.0010$	$0.9826\pm 0.0238$	$0.9698\pm 0.0122$

Table A4. Per-class performance of RandomForest at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3977\pm 0.0057$	$0.5148\pm 0.0091$	$0.4487\pm 0.0062$	60	Facebook	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Instagram	$0.1919\pm 0.0045$	$0.3812\pm 0.0049$	$0.2553\pm 0.0037$	60	Instagram	$0.5556\pm 0.0000$	$0.7143\pm 0.0000$	$0.6250\pm 0.0000$
1	LinkedIn	$0.1228\pm 0.0043$	$0.2776\pm 0.0030$	$0.1703\pm 0.0039$	60	LinkedIn	$0.3476\pm 0.0130$	$0.7143\pm 0.0000$	$0.4675\pm 0.0119$
1	Reddit	$0.5236\pm 0.0077$	$0.5523\pm 0.0021$	$0.5376\pm 0.0049$	60	Reddit	$0.7543\pm 0.0287$	$1.0000\pm 0.0000$	$0.8597\pm 0.0185$
1	Snapchat	$0.1094\pm 0.0027$	$0.2085\pm 0.0062$	$0.1435\pm 0.0032$	60	Snapchat	$1.0000\pm 0.0000$	$0.2857\pm 0.0000$	$0.4444\pm 0.0000$
1	Telegram	$0.1012\pm 0.0038$	$0.3906\pm 0.0120$	$0.1607\pm 0.0059$	60	Telegram	$0.2714\pm 0.0196$	$1.0000\pm 0.0000$	$0.4267\pm 0.0243$
1	TikTok	$0.3843\pm 0.0036$	$0.5080\pm 0.0010$	$0.4376\pm 0.0027$	60	TikTok	$0.8182\pm 0.0000$	$1.0000\pm 0.0000$	$0.9000\pm 0.0000$
1	Twitter	$0.2813\pm 0.0046$	$0.3744\pm 0.0033$	$0.3212\pm 0.0036$	60	Twitter	$0.3571\pm 0.0000$	$1.0000\pm 0.0000$	$0.5263\pm 0.0000$
1	WhatsApp	$0.5868\pm 0.0085$	$0.1939\pm 0.0009$	$0.2915\pm 0.0020$	60	WhatsApp	$1.0000\pm 0.0000$	$0.2917\pm 0.0000$	$0.4516\pm 0.0000$
1	YouTube	$0.5361\pm 0.0047$	$0.3482\pm 0.0034$	$0.4222\pm 0.0037$	60	YouTube	$0.9576\pm 0.0010$	$0.9826\pm 0.0238$	$0.9698\pm 0.0122$

Table A5. Per-class performance of XGBoost at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3781\pm 0.0050$	$0.6012\pm 0.0117$	$0.4642\pm 0.0073$	60	Facebook	$0.7477\pm 0.0233$	$1.0000\pm 0.0000$	$0.8555\pm 0.0153$
1	Instagram	$0.2043\pm 0.0045$	$0.2178\pm 0.0051$	$0.2109\pm 0.0044$	60	Instagram	$0.7800\pm 0.0274$	$0.5143\pm 0.0782$	$0.6182\pm 0.0664$
1	LinkedIn	$0.1352\pm 0.0032$	$0.1949\pm 0.0035$	$0.1597\pm 0.0028$	60	LinkedIn	$0.3648\pm 0.0236$	$0.5714\pm 0.0000$	$0.4450\pm 0.0175$
1	Reddit	$0.4391\pm 0.0045$	$0.5867\pm 0.0036$	$0.5023\pm 0.0041$	60	Reddit	$0.6733\pm 0.0463$	$1.0000\pm 0.0000$	$0.8040\pm 0.0332$
1	Snapchat	$0.1361\pm 0.0015$	$0.2027\pm 0.0033$	$0.1628\pm 0.0016$	60	Snapchat	$1.0000\pm 0.0000$	$0.8571\pm 0.0000$	$0.9231\pm 0.0000$
1	Telegram	$0.1001\pm 0.0019$	$0.3799\pm 0.0077$	$0.1585\pm 0.0029$	60	Telegram	$0.6667\pm 0.0000$	$1.0000\pm 0.0000$	$0.8000\pm 0.0000$
1	TikTok	$0.5201\pm 0.0040$	$0.4709\pm 0.0022$	$0.4943\pm 0.0023$	60	TikTok	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Twitter	$0.2562\pm 0.0041$	$0.4618\pm 0.0026$	$0.3296\pm 0.0033$	60	Twitter	$0.3273\pm 0.0143$	$1.0000\pm 0.0000$	$0.4930\pm 0.0162$
1	WhatsApp	$0.3979\pm 0.0032$	$0.2271\pm 0.0012$	$0.2892\pm 0.0018$	60	WhatsApp	$0.7895\pm 0.0000$	$0.3125\pm 0.0000$	$0.4478\pm 0.0000$
1	YouTube	$0.6072\pm 0.0102$	$0.2801\pm 0.0040$	$0.3833\pm 0.0056$	60	YouTube	$1.0000\pm 0.0000$	$0.8696\pm 0.0000$	$0.9302\pm 0.0000$

Table A5. Per-class performance of XGBoost at the single-flow and selected multi-flow window sizes.

Window	Class	Precision	Recall	F1	Window	Class	Precision	Recall	F1
1	Facebook	$0.3781\pm 0.0050$	$0.6012\pm 0.0117$	$0.4642\pm 0.0073$	60	Facebook	$0.7477\pm 0.0233$	$1.0000\pm 0.0000$	$0.8555\pm 0.0153$
1	Instagram	$0.2043\pm 0.0045$	$0.2178\pm 0.0051$	$0.2109\pm 0.0044$	60	Instagram	$0.7800\pm 0.0274$	$0.5143\pm 0.0782$	$0.6182\pm 0.0664$
1	LinkedIn	$0.1352\pm 0.0032$	$0.1949\pm 0.0035$	$0.1597\pm 0.0028$	60	LinkedIn	$0.3648\pm 0.0236$	$0.5714\pm 0.0000$	$0.4450\pm 0.0175$
1	Reddit	$0.4391\pm 0.0045$	$0.5867\pm 0.0036$	$0.5023\pm 0.0041$	60	Reddit	$0.6733\pm 0.0463$	$1.0000\pm 0.0000$	$0.8040\pm 0.0332$
1	Snapchat	$0.1361\pm 0.0015$	$0.2027\pm 0.0033$	$0.1628\pm 0.0016$	60	Snapchat	$1.0000\pm 0.0000$	$0.8571\pm 0.0000$	$0.9231\pm 0.0000$
1	Telegram	$0.1001\pm 0.0019$	$0.3799\pm 0.0077$	$0.1585\pm 0.0029$	60	Telegram	$0.6667\pm 0.0000$	$1.0000\pm 0.0000$	$0.8000\pm 0.0000$
1	TikTok	$0.5201\pm 0.0040$	$0.4709\pm 0.0022$	$0.4943\pm 0.0023$	60	TikTok	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$	$1.0000\pm 0.0000$
1	Twitter	$0.2562\pm 0.0041$	$0.4618\pm 0.0026$	$0.3296\pm 0.0033$	60	Twitter	$0.3273\pm 0.0143$	$1.0000\pm 0.0000$	$0.4930\pm 0.0162$
1	WhatsApp	$0.3979\pm 0.0032$	$0.2271\pm 0.0012$	$0.2892\pm 0.0018$	60	WhatsApp	$0.7895\pm 0.0000$	$0.3125\pm 0.0000$	$0.4478\pm 0.0000$
1	YouTube	$0.6072\pm 0.0102$	$0.2801\pm 0.0040$	$0.3833\pm 0.0056$	60	YouTube	$1.0000\pm 0.0000$	$0.8696\pm 0.0000$	$0.9302\pm 0.0000$