Enhancing Thai Sentiment Analysis with a Lightweight WangchanBERTa–CNN Hybrid and Optimal Kernel Configuration

1. Introduction

Sentiment analysis is an important methodology for investigating public sentiment, tracking brand reputation, and deriving actionable insights from large volumes of social media reviews. Although effective techniques have been proposed for sentiment analysis in highly researched languages such as English and Chinese, sentiment analysis in low-resource languages remains a significant challenge, especially Thai. Thai is a tonal language that lacks explicit word boundaries and exhibits complex linguistic characteristics, making text processing inherently difficult. Therefore, specialized approaches are required for sentiment analysis in this language. Sentiment analysis on Thai social media texts, characterized by creative spelling variations, emoji usage, code-mixing with English, and unique internet slang, further complicates sentiment analysis on Thai texts. Many approaches have been proposed for Thai sentiment analysis. Traditional lexicon-based sentiment analysis was first proposed, but it is ineffective at capturing the complexity of sentiment in social media text. Early machine learning-based sentiment analysis relied heavily on manually engineered features, which often lack generalizability. Consequently, deep learning approaches have emerged as a more effective solution. Various deep learning models have been proposed for sentiment analysis, such as LSTM, BiLSTM, CNN, GRU, and BiGRU. Among them, hybrid deep learning models were also proposed to enhance the performance of Thai sentiment analysis [1]. Hybrid models demonstrate significantly improved performance over single-architecture models.

Recently, transformer models have revolutionized natural language processing across many languages. Well-known representatives of this family include BERT [2], mBERT [3], RoBERTa [4], ALBERT [5], XLM-RoBERTa [6], and ELECTRA [7]. Some of these models include Thai language, such as mBERT, ALBERT, and XLM-RoBERTa. Unfortunately, previous studies have shown that models designed to handle multiple languages generally deliver weaker results than models trained and optimized for a single language [8,9,10,11] For the Thai language, the development of WangchanBERTa [12], a RoBERTa-based model pre-trained on large-scale Thai corpora, has yielded powerful contextualized representations that capture semantic and syntactic information specific to Thai. However, while these pre-trained models excel at encoding linguistic knowledge, they may not optimally extract sentiment-specific features that are crucial for fine-grained sentiment classification tasks. The combination of the WangchanBERTa and deep learning models has thus been proposed and shown promising performance for sentiment analysis in Thai [13].

Previous studies have explored various deep learning architectures to improve sentiment analysis in Thai. However, sentiment analysis of Thai social media texts remains challenging, particularly on the WISESIGHT Sentiment benchmark dataset, due to class imbalance, short text length, noisy user-generated content, and ambiguity between sentiment categories such as neutral and question. In addition, existing studies have not systematically explored CNN kernel size configurations for sentiment-specific feature extraction in Thai. This paper proposes lightweight hybrid models that integrate WangchanBERTa with deep learning architectures, including BiLSTM, BiGRU, and CNN, to enhance performance while maintaining computational efficiency. In addition, kernel size configurations for CNN-based sentiment feature extraction are systematically evaluated to assess their impact on performance across different sentiment categories. Experimental results demonstrate that the WangchanBERTa-based hybrid model with optimal selection of CNN kernel size improves performance and outperforms the state-of-the-art in terms of macro-average F1-score. Based on these findings, the main contributions of this study are as follows:

1.

A lightweight hybrid framework integrating WangchanBERTa with CNN for Thai sentiment analysis, balancing performance and computational efficiency.

2.

A systematic analysis of CNN kernel size configurations for sentiment feature extraction, providing new insights into their impact across sentiment categories.

3.

Empirical improvement over the state-of-the-art, achieving a 4.94% increase in macro-average F1-score on the WISESIGHT benchmark.

4.

Demonstration of an effective combination of contextualized embeddings (global features) and local feature extraction for handling short, noisy, and imbalanced Thai social media texts.

2. Related Work

The field of sentiment analysis in Thai has developed substantially over the last decade and a half, from lexicon-based methods to deep learning architectures. Past studies sought to improve methods for understanding sentiment in the Thai context, while more recent studies have used single transformer-based models and hybrid transformer-based models combined with other deep learning models to deeply understand the Thai language. Previous sentiment analysis studies can be grouped into two categories as follows.

The first group emphasized lexicon-based and machine learning methods. For example, Phienthrakul et al. explored SVM with multiple kernel functions for sentiment classification in Thai text [14]. The combination of different kernel functions and a single kernel function was evaluated, and it was reported that the combination could capture diverse linguistic features and improve classification performance compared to single-kernel approaches on product reviews. Lertsuksakda et al. proposed a novel approach to constructing Thai sentiment terms using the hourglass of emotions model [15]. They moved simple positive-negative polarity to capture more nuanced emotional states. Sentiment lexicons reflecting complex emotional dimensions were constructed. Chirawichitchai investigated emotion classification in Thai text using different term weighting and machine learning techniques [16]. The various term weighting schemes were explored to improve classification accuracy. The results showed that appropriately weighted features could significantly enhance the performance of machine learning models for Thai emotion classification. Chirawichitchai proposed a term weighting scheme based on term occurrence ratio optimized for sentiment analysis [17]. The term weighting scheme was proposed in mathematical formulations for calculating term importance that better reflected the sentiment characteristics of Thai words. The proposed weighting scheme improved classification performance by emphasizing terms with strong discriminative power for sentiment polarity.

Pasupa et al. applied SVMs to sentiment analysis of Thai children’s stories [18]. They extended sentiment analysis typically performed in domains such as product reviews and social media. Netisopakul and Lertsuksakda employed hypothesis testing based on observations from Thai sentiment classification experiments [19]. A more rigorous statistical approach was used to evaluate sentiment classification methods, employing hypothesis testing to validate performance differences between approaches. Haruechaiyasak et al. introduced S-Sense, which is a comprehensive framework for sentiment analysis. The framework was specifically designed for social media sensing [20]. It integrated multiple components, including preprocessing, feature extraction, and classification, tailored to the informal and often abbreviated language used in Thai social media. Porntrakoon and Moemeng introduced the SenseComp (Sentiment Compensation) technique for multi-dimensional analysis of consumer reviews in Thai [21]. They observed that consumer reviews often contain mixed sentiments across different product dimensions. As a result, a more sophisticated analysis was required than simple overall polarity classification. To address this, the proposed SenseComp technique improved sentiment analysis by evaluating multiple attributes separately. Taemung and Chirawichitchai applied SVM to analyze sentiment in Thai product reviews [22]. They focused on e-commerce applications and addressed practical challenges in classifying Thai-language customer reviews. The study showed that SVM could achieve reasonable performance on commercial sentiment analysis tasks. However, it still struggled to handle complex or ambiguous sentiment expressions.

The second group focused on deep learning and transformer approaches. For example, Vateekul and Koomsubha conducted comprehensive studies by applying deep learning techniques on Thai Twitter data [23]. Various deep learning architectures, including DCNN (Dynamic Convolutional Neural Network) and LSTM, were explored for sentiment analysis of Thai social texts. The study showed that the deep learning approaches outperformed traditional machine learning methods on Thai social media sentiment analysis. The best model was DCNN, achieving the highest accuracy with 75.35%. Pasupa and Seneewong also proposed a comparative study of deep learning techniques for Thai sentiment analysis [24]. Various deep learning architectures were systematically evaluated, and the impact of different input representations, including word embeddings, POS tags, and sentiment features, was explored. The study showed that deep learning approaches work best for Thai sentiment analysis. The CNN architecture, combined with the three feature types, could achieve the highest F1 Score of 81.70%. Thong-iad and Netisopakul compared different methods for Thai sentence sentiment tagging using Thai sentiment resources [25]. Different sentiment lexicons and tagging approaches were evaluated for the classification task. The results showed that using adverb and adjective synsets alone could give the highest accuracy of emotion classification. Traditional machine learning, SVM, Random Forest, performs well, but deep learning models, CNN, LSTM, BERT-based, often surpass them, especially with advanced techniques like hyperparameter tuning and incorporating linguistic features (POS, sentiment values). SVM offers strong performance among traditional methods, while hybrid deep learning models and transformer models (like WangchanBERTa) achieve higher accuracy

Lowphansirikul et al. developed WangchanBERTa, a transformer-based pre-trained language model specifically for Thai [12]. WangchanBERTa consistently outperforms established multilingual models like mBERT and XLM-R across a variety of benchmarks, achieving in NER (Named Entity Recognition), sentiment analysis, and POS tagging. WangchanBERTa has since become a foundation model for numerous Thai NLP tasks, including sentiment analysis. Pasupa and Seneewong proposed hybrid deep learning models that combined multiple neural architectures for Thai sentiment analysis [1]. The hybrid model, integrating CNNs and LSTMs, could capture both local features and long-range dependencies. It achieved higher performance than single-architecture models. From the proposed hybrid models, Bi-LSTM-CNN achieved the highest performance. It achieved macro-average F1-scores of 74.36% on ThaiTales, 77.07% on ThaiEconTwitter, and 55.21% on the WISESIGHT datasets. Jitboonyapinit et al. investigated sentiment analysis on Thai social media using convolutional neural networks combined with long short-term memory networks [26]. The challenges of this work are about informal language and unique linguistic patterns in Thai social media posts. The CNN-LSTM model successfully extracted spatial and temporal features. It achieved 85.00% accuracy of product reviews on social media. Khamphakdee and Seresangtakul developed an efficient deep learning approach optimized for Thai sentiment analysis [27]. This work focused on balancing model performance with computational efficiency. Modifications to the model architecture reduced training time while maintaining high accuracy.

Nokkaew et al. analyzed online public opinion regarding major infrastructure projects using advanced machine learning and deep learning for Thai sentiment analysis [28]. Their research demonstrated the application of sentiment analysis to policy-relevant social issues by analyzing public discourse on the Thailand-China high-speed train and Laos-China railway projects. Comment sentiment classification was performed using six approaches: linear regression, naive bayes, random forest, Bi-LSTM, BERT-Base-Thai, and WangchanBERTa. The WangchanBERTa model achieved 94.57% accuracy. Suraratchai and Phoomvuthisarn proposed a hybrid method combining WangchanBERTa with CNN and BiLSTM architectures for Thai sentiment analysis [13]. Their research built upon the pre-trained WangchanBERTa model by adding convolutional and bidirectional recurrent layers to capture additional linguistic features. The hybrid architecture achieved competitive performance on the WISESIGHT and the Thai Children’s Tales datasets. The Parallel Hybrid approach, WangchanBERTa-CNN-BiLSTM, achieved the highest macro-average F1-scores, reaching 62.70% on the WISESIGHT dataset and 78.59% on the Thai Children’s Tales dataset. Satjathanakul and Siriborvornratanakul focused on improving sentiment polarity classification on the Thai product reviews dataset using modern Transformer-based architectures [29]. The fine-tuned WangchanBERTa model achieved performance metrics ranging between 66% and 93% on the product reviews dataset. Emphan et al. enhanced the performance of the sentiment analysis model using GridSearchCV for hyperparameter optimization. This work focused on classifying sentiment on electric vehicle discussions in Thailand [30]. Hyperparameter tuning was used to identify optimal configurations for Thai sentiment classification models. The study demonstrated that careful hyperparameter tuning could improve performance.

Previous work has demonstrated that transformer-based models, especially WangchanBERTa, achieve high performance in contextual understanding of Thai linguistic nuances. Furthermore, combining WangchanBERTa with other deep learning techniques could improve the performance of Thai sentiment analysis even on imbalanced datasets such as WISESIGHT. As in previous work, they extensively explored various hybrid models, including WangchanBERTa, with other deep learning models for Thai sentiment analysis, but the impact of kernel size selection when combining WangchanBERTa and CNN across different sentiment categories remains underexplored. Therefore, this paper proposes a hybrid architecture combining WangchanBERTa and CNN and systematically investigates optimal kernel configurations to enhance sentiment analysis performance on the imbalanced WISESIGHT dataset. In addition, two alternative hybrid models, WangchanBERTa with BiGRU and WangchanBERTa with BiLSTM, are developed for comparative evaluation against the WangchanBERTa-CNN model.

3. Materials and Methods

Figure 1 shows an overview of the proposed hybrid deep learning framework for Thai sentiment analysis. In the pre-processing stage, two approaches were employed. In the first approach, we did not apply any imbalance-handling techniques to ensure a fair comparison with prior studies that used the original dataset without modification. In the second approach, a data balancing technique, namely random oversampling, was applied to address class imbalance and improve the representation of minority classes. The input texts are tokenized using the WangchanBERTa tokenizer, and special tokens ([CLS], [SEP]) are added. The tokens are then converted into token IDs. These IDs are fed to the hybrid deep learning models that integrate WangchanBERTa with CNN, BiLSTM, and BiGRU. The hybrid deep learning models are then evaluated to find the best-performing model. The details of each phase are described below.

3.1. The WISESIGHT Dataset

The WISESIGHT dataset has been widely used for Thai sentiment analysis in many studies. It was introduced by Suriyawongkul et al. [31]. The data was collected by WISESIGHT Thailand, one of the largest social media analysis companies that monitors social media discussions across many channels. The data comprises real messages written in Thai, collected from many popular social networking sites, including Facebook, Twitter, Instagram, YouTube, and Pantip. This publicly available dataset is available from UCI or Hugging Face. It contains social media posts annotated into four sentiment categories: positive, neutral, negative, and question. The WISESIGHT dataset is challenging for sentiment analysis competitions because it contains informal, noisy Thai social media text with slang, misspellings, and no clear word boundaries, making accurate tokenization and interpretation difficult. Additionally, short context on average (as shown in Figure 2), mixed sentiments, class imbalance, and subtle differences between categories (e.g., neutral vs. question) further complicate sentiment classification as shown in Figure 3. The WISESIGHT benchmark is split into training, validation, and testing sets for competition in prior studies, as shown in Figure 4 and Table 1.

Figure 2, Figure 3, Figure 4 and Table 1 clearly illustrate the imbalance among sentiment categories. The question class contains only 476 samples in the training set, 42 samples in the validation set, and 57 samples in the testing set. Classifying the question class is a very challenging task because it is very small. Many studies struggled with this class because of its very small size. The largest category in the dataset is the neutral class. It contains 11,795 samples in the training set, 1,291 samples in the validation set, and 1,453 samples in the testing set. The largest class size almost interferes with the smaller ones, such as the positive and negative classes, because models may learn patterns from neutral-class samples too much. In addition, the dataset contains highly informal, diverse language typical of Thai social media, including slang, abbreviations, emojis, mixed Thai-English content, and inconsistent spelling. These characteristics make the WISESIGHT dataset particularly challenging for sentiment analysis and require models to handle noisy and unpredictable real-world text. Therefore, the WISESIGHT dataset was utilized as a challenging benchmark to assess the performance of the proposed model.

3.2. Pre-Processing

The pre-processing stage is divided into two approaches. For the first approach, the input texts of the WISESIGHT dataset are directly tokenized using the WangchanBERTa tokenizer, and special tokens ([CLS], [SEP]) are added, where [CLS] represents the whole sentence and [SEP] marks the sentence boundary. The tokens are then converted into token IDs. These IDs are fed into the WangchanBERTa model. For example, the input sentence “สินค้าดีมาก จัดส่งรวดเร็ว” is tokenized into subword units using the WangchanBERTa tokenizer based on SentencePiece (e.g., [“สินค้า”, “ดี”, “นาก”, “จัดส่ง””, “รวดเร็ว”]). Special tokens [CLS] and [SEP] are added at the beginning and end of the sequence, respectively (e.g., [“[CLS]”, “สินค้า”, “ดี”, “นาก”, “จัดส่ง””, “รวดเร็ว”, “[SEP]”]). The resulting tokens are then converted into token IDs (e.g., [101, 3456, 789, 1203, 4567, 8901, 102]), which are used as input to the WangchanBERTa model.

For the second approach, random oversampling was employed to address class imbalance in the WISESIGHT dataset by duplicating instances from minority classes. This approach helps mitigate model bias toward majority classes and improves the learning of underrepresented sentiments. Moreover, unlike synthetic data generation techniques, random oversampling preserves the original linguistic characteristics of the text. It is particularly suitable for Thai sentiment analysis. However, to ensure fair evaluation, experiments were also conducted on the original imbalanced dataset. After data balancing, training, validation, and testing sets for the balanced WISESIGHT dataset are shown in Table 3. The balanced WISESIGHT dataset is fetched to the next steps as the original one.

Table 2. The Balanced WISESIGHT dataset characteristics.

Number of messages	Training	Validation	Testing
Total	47,180	5,164	5,812
#Neutral	11,795	1,291	1,453
#Negative	11,795	1,291	1,453
#Positive	11,795	1,291	1,453
#Question	11,795	1,291	1,453

Table 2. The Balanced WISESIGHT dataset characteristics.

Number of messages	Training	Validation	Testing
Total	47,180	5,164	5,812
#Neutral	11,795	1,291	1,453
#Negative	11,795	1,291	1,453
#Positive	11,795	1,291	1,453
#Question	11,795	1,291	1,453

3.3. WangchanBERTa

Currently, contextual embeddings have been shown to outperform traditional word embeddings (e.g., Word2Vec or GloVe) in text representation. Unlike traditional word embedding, contextual embeddings capture the meaning of each word based on its surrounding context, allowing the same word to have different representations depending on its usage in a sentence. Therefore, contextual embeddings provide richer semantic representations and significantly improve performance in complex tasks such as sentiment analysis, particularly for languages like Thai with ambiguous word boundaries. For Thai texts, several BERT-based models have been developed to generate Thai contextual embeddings, including Thai-specific models such as WangchanBERTa and ThaiBERT, as well as multilingual models like mBERT and XLM-R. Among these, WangchanBERTa demonstrates superior performance due to its training on large-scale Thai corpora and its ability to better capture the linguistic characteristics of Thai. Therefore, WangchanBERTa is used to efficiently contextualize embeddings and is thus suitable for short, noisy, or informal Thai text, where fine-grained expressions drive sentiment.

In our work, the token IDs obtained from the pre-processing step are used as input to the WangchanBERTa model to generate contextual representations. First, each token ID is transformed into a dense vector through the embedding layer, which consists of token embeddings, segment embeddings, and positional embeddings. These components are combined to form the initial input representation, as defined in (1), where $E_{\mathrm{token}}$, $E_{\mathrm{segment}}$, and $E_{\mathrm{position}}$ denote the word-piece, segment, and positional embeddings, respectively. The resulting embeddings are then passed through a stack of transformer encoder layers. Each layer consists of a self-attention mechanism and a position-wise feed-forward network. The self-attention mechanism, defined in (2), enables each token to attend to all other tokens in the sequence, where Q, K, and V are the query, key, and value matrices derived from the input representation, and $d_k$ denotes the dimension of the key vectors. The output of the attention layer is further refined by a feed-forward network, as defined in (3), where $W_1$, $W_2$, $b_1$, and $b_2$ are learnable parameters. After passing through all encoder layers, the model produces contextualized representations X, as defined in (4), where each token representation $x_t$ captures both semantic and contextual information, n is the sequence length, and d is the hidden dimension. These contextualized embeddings are then used as input features for downstream models, including CNN, BiLSTM, and BiGRU.

$$H^{\left(0\right)}=E_{\mathrm{token}}+E_{\mathrm{segment}}+E_{\mathrm{position}}$$

(1)

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(2)

$$\mathrm{FFN}\left(H\right)=\mathrm{GELU}(H{W}_1+b_1)W_2+b_2$$

(3)

$$H=[h_1,h_2,\dots ,h_n]\in {\mathbb{R}}^{n\times d}$$

(4)

3.4. Modeling

Traditional deep learning models such as BiLSTM, BiGRU, and CNN have been widely used for sentiment analysis due to their ability to capture sequential dependencies and local n-gram features. However, they rely on static embeddings and struggle to model rich contextual semantics, particularly in short, noisy, and linguistically complex Thai social media texts. CNN models further lack the ability to capture long-range dependencies, while recurrent models may be insufficient for handling highly ambiguous sentiment patterns (e.g., neutral vs. question) [32]. On the other hand, transformer-based models such as WangchanBERTa provide powerful contextualized representations and significantly improve Thai NLP performance, but they primarily focus on global context and may overlook fine-grained local features critical for sentiment classification in short texts [2,12]. Therefore, relying on either traditional deep learning or transformer models alone is insufficient. Therefore, integrating both approaches can effectively capture complementary global and local features.

We hybridize WangchanBERTa with BiLSTM, BiGRU, and CNN to mitigate noise in the WISESIGHT dataset and improve classification performance while maintaining a relatively small parameter size. The proposed hybrid models are considered lightweight, as they incorporate only a single additional neural layer (BiLSTM, BiGRU, or CNN) on top of the WangchanBERTa model. This design avoids deep or complex stacking to reduce the number of trainable parameters and computational overhead while preserving strong performance. All three hybrid models share WangchanBERTa as the backbone, a 12-layer model with 768 hidden dimensions and 12 attention heads [12], serving as an initial contextual encoder to produce high-level hidden representations of Thai text as contextual embeddings.

The proposed WangchanBERTa-CNN model integrates WangchanBERTa with a CNN for sentiment classification. Let $x_t\in {\mathbb{R}}^d$ denote the contextual embedding at time step t produced by WangchanBERTa. These contextual embeddings are then processed by one-dimensional convolutional layers with kernel sizes $k\in \{2,3,4\}$. Each kernel size extracts local n-gram features from the contextual embeddings [32,33,34]. Each convolution operation is performed as shown in (5), where * denotes the convolution operator, and $W^{\left(k\right)}$ and $b^{\left(k\right)}$ represent the weight and bias of the filter with kernel size k. The resulting feature maps are then subjected to a max-over-time pooling operation, as defined in (6), in order to extract the most salient features. The pooled features from different kernel sizes are concatenated to form a unified feature vector, as shown in (7), where ⊕ denotes concatenation. Finally, the resulting feature vector is passed through a fully connected layer to produce the output logits, as defined in (8).

$$C^{\left(k\right)}=\mathrm{ReLU}(W^{\left(k\right)}*H+b^{\left(k\right)})$$

(5)

$${\widehat{c}}^{\left(k\right)}=\underset{t}{max}{C}_t^{\left(k\right)}$$

(6)

$$\mathbf{c}=\left[{\widehat{c}}^{\left(2\right)}\oplus {\widehat{c}}^{\left(3\right)}\oplus {\widehat{c}}^{\left(4\right)}\right]$$

(7)

$$\mathbf{z}=W_y\mathbf{c}+b_y$$

(8)

The proposed WangchanBERTa-BiLSTM model integrates WangchanBERTa with a BiLSTM network to capture both contextual and sequential dependencies in Thai text. Let $x_t\in {\mathbb{R}}^d$ denote the contextual embedding at time step t produced by WangchanBERTa. The contextual embeddings are passed to a BiLSTM layer, which processes the sequence in both forward and backward directions, as shown in (15) and (16), respectively. Each LSTM unit within the BiLSTM architecture comprises a forget gate (9), input gate (10), candidate cell state (11), cell state (12), output gate (13), and hidden state (14), where W is the weight matrix applied to the contextual embedding $x_t$, U is the weight matrix applied to the previous hidden state $h_{t-1}$, and b is the bias vector. These components collectively facilitate the learning of long-range dependencies and rich contextual representations. The hidden states from both directions are concatenated to form the final representation at each time step, as shown in (17). To obtain a fixed-length representation, the concatenated hidden state is selected at the final time step T, where T denotes the length of the sequence. Finally, the feature vector is passed through a fully connected layer, as shown in (18), where z denotes the output logits for the target sentiment classes.

$$f_t=\sigma (W_f{x}_t+U_f{h}_{t-1}+b_f)$$

(9)

$$i_t=\sigma (W_i{x}_t+U_i{h}_{t-1}+b_i)$$

(10)

$${\tilde{c}}_t=tanh(W_c{x}_t+U_c{h}_{t-1}+b_c)$$

(11)

$$c_t=f_t\odot c_{t-1}+i_t\odot {\tilde{c}}_t$$

(12)

$$o_t=\sigma (W_o{x}_t+U_o{h}_{t-1}+b_o)$$

(13)

$$h_t=o_t\odot tanh\left(c_t\right)$$

(14)

$$\overrightarrow{h_t}={\mathrm{LSTM}}_f(x_t,\overrightarrow{h_{t-1}})$$

(15)

$$\overleftarrow{h_t}={\mathrm{LSTM}}_b(x_t,\overleftarrow{h_{t+1}})$$

(16)

$$h_t^{*}=\left[\overrightarrow{h_t}\oplus \overleftarrow{h_t}\right]\in {\mathbb{R}}^{2d}$$

(17)

$$z=W_y{h}_T^{*}+b_y$$

(18)

The proposed WangchanBERTa-BiGRU model integrates WangchanBERTa with a BiGRU network to capture both contextual and sequential features in Thai text. Let $x_t\in {\mathbb{R}}^d$ denote the contextual embedding at time step t generated by WangchanBERTa. These embeddings are then fed into a BiGRU layer, which processes the sequence in both forward and backward directions, as shown in (23) and (24), respectively. Each GRU unit operates through gating mechanisms that regulate the flow of information. Compared to LSTM, GRU adopts a simplified architecture with two primary gates: the update gate and the reset gate, as shown in (19) and (20), respectively. Based on these gates, the candidate hidden state ${\tilde{h}}_t$ is computed by combining the current input $x_t$ with the reset-modulated previous hidden state, as defined in (21). The final hidden state $h_t$ is then obtained by interpolating between the previous hidden state and the candidate hidden state using the update gate, as shown in (22). This mechanism enables GRU to effectively capture sequential dependencies while maintaining lower computational complexity than LSTM. The hidden states from both directions are concatenated at each time step, as shown in (25). To obtain a fixed-length representation, the concatenated hidden state at the final time step T is selected, where T denotes the sequence length. Finally, the feature vector is passed through a fully connected layer to produce the output logits, as shown in (26).

$$z_t=\sigma (W_z{x}_t+U_z{h}_{t-1}+b_z)$$

(19)

$$r_t=\sigma (W_r{x}_t+U_r{h}_{t-1}+b_r)$$

(20)

$${\tilde{h}}_t=tanh\left(W_h{x}_t+U_h(r_t\odot h_{t-1})+b_h\right)$$

(21)

$$h_t=(1-z_t)\odot h_{t-1}+z_t\odot {\tilde{h}}_t$$

(22)

$$\overrightarrow{h_t}={\mathrm{GRU}}_f(x_t,\overrightarrow{h_{t-1}})$$

(23)

$$\overleftarrow{h_t}={\mathrm{GRU}}_b(x_t,\overleftarrow{h_{t+1}})$$

(24)

$$h_t^{*}=\left[\overrightarrow{h_t}\oplus \overleftarrow{h_t}\right]\in {\mathbb{R}}^{2d}$$

(25)

$$z=W_y{h}_T^{*}+b_y$$

(26)

For all proposed models, the final logits $\mathbf{z}$ are converted into class probabilities using the softmax function, as shown in (27). The models are trained using the categorical cross-entropy loss, as shown in (28), where C denotes the number of classes, $y_i$ is the ground-truth label, and ${\widehat{y}}_i$ is the predicted probability for class i. To improve generalization and mitigate overfitting, a dropout layer is applied to the extracted feature representation before the final classification layer. In practice, the softmax operation is internally combined with the cross-entropy loss during training.

$${\widehat{y}}_i={\displaystyle \frac{exp\left(z_i\right)}{{\sum }_{j=1}^{C}exp\left(z_j\right)}}$$

(27)

$$\mathcal{L}=-\sum _{i=1}^C{y}_{i}log\left({\widehat{y}}_i\right)$$

(28)

3.5. Evaluation

To evaluate model performance across categories in the WISESIGHT dataset, we use three standard classification metrics. The metrics include accuracy as shown in (29), precision, recall, and F1-Score. For multi-class evaluation, precision, recall, and F1-Score are computed per class as shown in (30), (31), and (32), respectively. In addition, the macro-average scores are used to evaluate the average of precision, recall, and F1-score as shown in (33), (34), and (35), respectively. Macro-average F1-score is adopted as the primary evaluation metric because the WISESIGHT dataset is highly imbalanced. Unlike accuracy or micro-average F1-score, macro-average F1-score assigns equal importance to all classes, ensuring that performance on minority classes is not overlooked. This provides a more reliable and fair assessment of model effectiveness in real-world sentiment classification scenarios.

$$Acc={\displaystyle \frac{{\sum }_{c=1}^{4}T{P}_c}{N}}$$

(29)

$$P_c={\displaystyle \frac{T{P}_c}{T{P}_c+F{P}_c}}$$

(30)

$$R_c={\displaystyle \frac{T{P}_c}{T{P}_c+F{N}_c}}$$

(31)

$$F{1}_c=2\times {\displaystyle \frac{P_c\times R_c}{P_c+R_c}}$$

(32)

$$\mathit{Macroavg}.P={\displaystyle \frac{1}{4}}\sum _{c=1}^4{P}_c$$

(33)

$$\mathit{Macroavg}.R={\displaystyle \frac{1}{4}}\sum _{c=1}^4{R}_c$$

(34)

$$\mathit{Macroavg}.F1={\displaystyle \frac{1}{4}}\sum _{c=1}^{4}F{1}_c$$

(35)

From equations (29) - (35), c denotes each sentiment category. There are four categories: positive, neutral, negative, or question. $T{P}_c$ refers to the number of correctly predicted samples for class c. For example, $TP$ of the positive class means the number of correctly predicted samples as positive. N represents the total number of predictions. $F{P}_c$ indicates the number of samples that do not belong to class c but were classified as class c. For example, $FP$ for the positive class is the number of samples predicted as positive but belonging to the neutral, negative, or question class. $F{N}_c$ represents the number of samples that belong to class c but were predicted as other classes. For example, $FN$ for the positive class is the number of samples that belong to the positive class but were predicted as neutral, negative, or question. In addition, confusion matrices are utilized to analyze classification performance, and the number of model parameters is evaluated to verify the lightweight nature of the proposed models.

4. Results and Discussion

In the experimental setup, the input sequence length was limited to 128 tokens. The model was trained with a batch size of 16 and optimized using AdamW with a learning rate of $2e^{-5}$ , incorporating a weight decay of 0.01 to mitigate overfitting. A dropout rate of 0.2 was also applied to further improve generalization. Together, weight decay and dropout serve as key regularization techniques in the training process. All our models were trained for 20 epochs to identify the optimal training point by monitoring the smallest gap between the training and validation losses. The optimal point of each model was used for classification.

4.1. Performance on the WISESIGHT Dataset

First, the WISESIGHT benchmark (without data balancing) is used in the experiment to compare our proposed models with the state-of-the-art (the parallel hybrid model [13]), and BiLSTM-CNN [1]. The results are shown in Table 3.

For the first experiment, we found that WangchanBERTa-CNN with kernel sizes [2, 3, 4] yields higher accuracy than the parallel hybrid models WangchanBERTa-BiLSTM and WangchanBERTa-BiGRU. Therefore, we further investigated the impact of convolutional kernel sizes on model performance. Multiple WangchanBERTa-CNN configurations were evaluated using different kernel combinations, including [1,2,3], [2,3,4], [3,4,5], and [1,2,3,4]. Kernel size combinations such as [1,2] or single kernel sizes were not selected because they provide limited coverage of textual patterns. Using only small kernels (e.g., [1,2]) restricts the model to very short n-gram features, which may fail to capture meaningful phrase-level context. Similarly, using a single kernel size limits feature diversity, as the model can only learn patterns within a fixed receptive field [32]. Since the kernel size combinations [1,2,3] and [2,3,4] achieved high macro-average F1-scores. Therefore, we further extended this idea by evaluating the configuration [1,2,3,4], which integrates both smaller and slightly larger receptive fields. The results of kernel size configurations are also shown in Table 2. Overall, the WangchanBERTa-CNN model with kernel sizes [2,3,4] achieves the highest macro-average F1-score at 65.80%, outperforming both smaller [1,2,3], larger [3,4,5], and [1,2,3,4] kernel settings as well as WangchanBERTa-BiLSTM and WangchanBERTa-BiGRU. Its macro-average F1-Score is also slightly higher than that of all competing models. It outperforms the state-of-the-art parallel hybrid model [13] by 4.94%. Our findings suggest that medium-sized kernels capture local linguistic patterns in Thai social media text more effectively than either narrower or broader receptive fields.

Across sentiment categories, the negative class consistently shows the strongest performance for all models. The WangchanBERTa-CNN [2,3,4] model achieves the highest recall (83.89%) and F1-score (79.64%) for this class. The model is particularly effective in identifying negative sentiment, which often contains clearer lexical or emotional cues. The neutral class follows with stable performance across models, although the parallel hybrid model has the highest recall (83.49%). In contrast, performance on the positive and question classes remains comparatively low across all models. For the positive class, the best F1-score is achieved by the WangchanBERTa-CNN [2,3,4] model (53.18%), but it still outperforms the state-of-the-art model. The WangchanBERTa-CNN [1,2,3] model achieves the highest F1-score for the positive class. The question class is the most challenging category due to its very small size. The WangchanBERTa-CNN [2,3,4] model achieves the highest F1-score of 51.38%, and its overall performance remains high. However, it still suffers from limited training data and the inherent ambiguity of interrogative structures. These results reinforce the dataset’s imbalance and the linguistic complexity of Thai questions in social media contexts.

Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 compare the confusion matrices of all our evaluated models and reveal clear differences in classification performance across sentiment categories. From the confusion matrices, the WangchanBERTa-CNN [2, 3, 4] model achieves the most accurate and balanced predictions. It is the best model for accurately identifying negative sentiment. Moreover, the WangchanBERTa-CNN model performs better than other configurations on the question class. The smaller [1,2,3] and larger [3,4,5] kernel sizes produce more misclassifications, particularly between neutral and positive texts. The results indicate that medium-sized convolutional kernels can more effectively capture key linguistic cues in short, informal Thai social media texts.

In summary, Table 2 and Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 show that the combination of WangchanBERTa with CNN using medium kernel sizes [2, 3, 4] achieves the most robust performance across sentiment categories. Therefore, combining contextualized embeddings and local feature extraction is particularly effective for handling short, noisy, and linguistically diverse Thai social media texts.

Next, we further enhanced the best-performing model, WangchanBERTa-CNN [2,3,4], by incorporating several advanced techniques, including attention mechanisms, squeeze-and-excitation (SE) blocks, gate fusion, and a three-model hybrid architecture. Attention-based approaches, including single-head and multi-head attention, were applied to the concatenated CNN feature maps to dynamically assign importance weights to different time steps. A SE block is applied to each convolutional output. The SE block performs channel-wise recalibration by first applying global average pooling (squeeze operation) to capture channel-level statistics. For the gate fusion model, the parallel CNN branches with different kernel configurations ([1,2,3] and [2,3,4]) are utilized to capture multi-scale local patterns. To adaptively control feature importance, a gated fusion mechanism is applied. This mechanism enables the model to selectively emphasize or suppress features from different CNN branches. We also hybridize WangchanBERTa, BiLSTM, and CNN with kernel size [2, 3, 4] to enable complementary modeling of sequential dependencies and local n-gram features, while the attention mechanism further enhances the model by selectively focusing on the most informative contextual representations. Adding these techniques to WangchanBERTa-CNN [2,3,4] did not lead to performance improvements, as shown in Table 4. This may be due to several factors. First, the WangchanBERTa-CNN [2, 3, 4] model is already well-optimized, and additional components may introduce unnecessary complexity, leading to overfitting, especially given the limited and noisy nature of social media data. Second, techniques such as attention and fusion mechanisms may not effectively capture additional useful information beyond what the pretrained encoder already learns. Third, the class imbalance and subtle distinctions between sentiment categories (e.g., neutral vs. question) may limit the effectiveness of more complex architectures. Finally, increasing model complexity can make training more difficult and less stable, resulting in suboptimal generalization performance. In addition, adding some techniques to WangchanBERTa-CNN increases the number of parameters and makes the models more complex, as shown in Figure 11. Figure 11 illustrates the relationship between model performance and parameter size for only the proposed and extended models, as prior studies do not report parameter sizes. Figure 11 demonstrates that the proposed WangchanBERTa-CNN [2,3,4] model achieves the highest performance while maintaining a relatively lower parameter count compared to other models.

In summary, the results show that the simple combination of WangchanBERTa with CNN using medium kernel sizes, [2, 3, 4], achieves the most robust performance across sentiment categories while maintaining a relatively small parameter size. Therefore, combining contextualized embeddings and local feature extraction is particularly effective for handling short, noisy, and linguistically diverse Thai social media texts. The WangchanBERTa-CNN model achieved the 65.80% marco-average F1-score and improved the macro-average F1-score by approximately 4.94% when compared with the state-of-the-art, the parallel hybrid model [13].

4.2. Performance on the Balanced WISESIGHT Dataset

To ensure robust performance on the WISESIGHT dataset, models achieving a macro-average F1-score comparable to the best-performing model (i.e., greater than 64%) were selected for further analysis on the balanced WISESIGHT dataset. The WangchanBERTa-BiLSTM, WangchanBERTa-CNN [1,2,3], WangchanBERTa-CNN [2,3,4], and WangchanBERTa-CNN [3,4,5] models were experiment future and report the results as shown in Table 5.

The results in Table 5 clearly demonstrate that applying data balancing significantly improves model performance across all four architectures. Macro-average F1-score improvements exceeding 40% after balancing. The WangchanBERTa–CNN with kernel sizes [2,3,4] achieves the best overall performance. It achieve a macro-average F1-Score of 94.11%. The most pronounced improvements are observed in the “Question” and “Positive” classes. The balancing approach effectively addresses class imbalance and enhances minority-class recognition. Overall, these findings confirm that random oversampling plays a crucial role in boosting classification performance, especially when combined with an appropriately designed hybrid architecture.

5. Conclusions

This study presents lightweight hybrid models that integrate WangchanBERTa with deep learning architectures, including BiLSTM, BiGRU, and CNN, for Thai sentiment analysis on social media texts. The proposed approach effectively addresses key challenges in the WISESIGHT benchmark, including class imbalance, short text lengths, noisy user-generated content, and ambiguity between sentiment categories. Experimental results demonstrate that the WangchanBERTa–CNN model with optimal kernel size configuration ([2,3,4]) achieves the best performance on the WISESIGHT benchmark, with a macro-average F1-score of 65.80%, outperforming the state-of-the-art by 4.94% while maintaining a relatively low parameter size. Furthermore, the systematic analysis of CNN kernel sizes provides new insights into sentiment-specific feature extraction, highlighting the importance of combining contextualized embeddings with local feature learning. On the balanced WISESIGHT dataset, the WangchanBERTa–CNN model continues to achieve the best performance. In addition, dataset balancing significantly improves the performance of the proposed models, as random oversampling effectively enhances classification results. The WangchanBERTa–CNN model with kernel size configuration [2,3,4] achieves an excellent macro-average F1-Score of 94.11% on the balanced WISESIGHT dataset. These findings confirm that the proposed hybrid approach, the WangchanBERTa–CNN model with a kernel size configuration of [2,3,4], is effective for handling short, noisy, and linguistically diverse Thai social media texts.

While the final version of the proposed WangchanBERTa-CNN architecture with medium kernel sizes also shows strong performance, several avenues for future work remain. For instance, focal loss or class-balanced loss, SMOTE-style oversampling, and data augmentation strategies tailored to Thai social media can help address the under-representation of the positive and question classes. The incorporation of syntactic or semantic features, POS tags, dependency relations, and sentiment lexicons would improve the detection of more subtle sentiment signals, which CNNs alone fail to learn. If we extend this method to incorporate more general conversational context (threads, replies, or posts surrounding them), classification accuracy may improve, particularly for ambiguous sentiment. Testing the model on diverse Thai sentiment datasets or developing it in multi-domain settings (such as product reviews and political discourse) may enhance its robustness and generalization.