Methodology
This study employs an LLM-assisted hybrid approach, integrating text mining, sentiment analysis, and Partial Least Squares Structural Equation Modeling (PLS-SEM) to analyze consumer perceptions of creativity in MCCPs. The methodology consists of three main phases: (1) data collection and preprocessing, (2) topic modeling and clustering using LDA + SVM and BERT + K-Means with LLM-enhanced semantic analysis, and (3) validation of extracted MCCP dimensions via PLS-SEM.
Data Collection
The Palace Museum is widely recognized as the benchmark for MCCP innovation in China, setting standards that many other museums follow. As one of China’s most significant cultural heritage sites, it holds a central position in the country’s historical and artistic legacy, making it an ideal case for studying consumer perceptions of MCCPs. Notably, in 2024, the successful World Heritage inscription of Beijing’s Central Axis, where the Palace Museum serves as the core landmark, has further reinforced its cultural and symbolic importance. Given its unparalleled influence in heritage preservation and cultural commercialization, the Palace Museum provides a unique and representative context for analyzing how consumers engage with MCCPs in a market-driven heritage landscape. Furthermore, with China emerging as a global leader in digital commerce, an increasing number of museums have established official e-commerce platforms, such as Tmall flagship stores, to enhance accessibility to their MCCPs. These platforms, which synchronize product availability with physical museum stores and offer nationwide free shipping, have become a primary purchasing channel for MCCPs in China. Industry reports indicate that online transactions now constitute a substantial share of total MCCP sales, as virtual museum visits have surpassed physical attendance for the first time, with Tmall and Taobao museum flagship stores alone recording 1.6 billion visits in a single year—1.5 times the total nationwide museum foot traffic
1—underscoring the growing influence of e-commerce on consumer engagement with MCCPs.
This study does not rely on in-person museum visits to collect consumer feedback for two key reasons. First, direct interviews with consumers could introduce social desirability bias, where respondents may feel pressured to provide more favorable responses when interacting face-to-face. Second, on-site surveys present logistical challenges, requiring significant resources while yielding a relatively limited volume of textual data. In contrast, e-commerce platforms offer a vast and readily available repository of consumer-generated reviews, enabling large-scale text mining and quantitative analysis. Another advantage of leveraging e-commerce reviews over social media discussions is the authenticity of the feedback. Each review is tied to an actual purchase, ensuring that the dataset reflects real consumer experiences rather than speculative or promotional content.
Data Cleaning
To ensure that the dataset contains only meaningful consumer insights, a systematic data-cleaning process will be applied. This process will involve removing non-informative, redundant, or irrelevant reviews while retaining those that provide explicit evaluations of MCCP attributes. By implementing these cleaning criteria, the study will ensure that the final dataset is optimized for subsequent text mining and structural modeling. The detailed cleaning criteria and their impact on the dataset size will be reported in the Results section.
Themes Identification
This study will employ a two-stage hybrid approach, integrating LDA+SVM and BERT+K-Means, to systematically extract consumer-perceived dimensions of MCCPs. When integrating results from LDA+SVM and BERT+K-Means, themes identified by both methods will be merged, while those detected in only one method will be retained for empirical validation in PLS-SEM. Finally, to ensure the accuracy of dimensions and indicators, a text analysis of the comments will be conducted in SPSSAU by identifying high-frequency keywords.
LDA and SVM
In the first stage, Latent Dirichlet Allocation (LDA), a probabilistic generative model commonly used for topic modeling, will be employed to identify latent topics within the consumer review corpus in DIKW (7.83). LDA is particularly suitable for this study as it identifies underlying themes in large text datasets by analyzing word co-occurrence patterns, making it an effective tool for extracting consumer-perceived dimensions of MCCPs. To validate LDA-extracted topics, the topic coherence score will determine the optimal number of topics, selecting the model with the highest coherence score. Additionally, LDAvis will assess topic dispersion and overlap, further validating the separation and interpretability of themes. To determine the indicators for each theme, only keywords with topic weights greater than zero, rounded to three decimal places, will be retained. Lower-weighted words will be considered statistically insignificant in defining the thematic structure. This filtering process will ensure that only highly relevant keywords contribute to the final thematic framework. Using this approach, the study will identify dimensions (themes) along with their indicators (high-weight keywords), which will form the foundation for further classification.
To validate the robustness of the topics extracted by LDA, a Support Vector Machine (SVM) classifier will be employed to refine the topic assignments. Since the themes derived from LDA are probabilistic rather than deterministic, the use of SVM will ensure that each review is assigned a dominant, definitive theme. This approach will help verify the accuracy of the LDA results, confirming that the identified themes reliably represent the underlying patterns in the data. Moreover, the SVM model will be trained using the LDA-derived themes as target labels. The model’s performance will be evaluated using standard classification metrics, such as accuracy, precision, recall, and F1-score, which will provide evidence that the LDA-derived themes are reliable and consistent. For the process of SVM, the dataset will be split into training (80%), and test (20%) sets, and a grid search with five-fold cross-validation will be conducted to identify the optimal SVM hyperparameters, including kernel type (linear, RBF, or polynomial), regularization parameter (C), and gamma value. This optimization process ensures that the classifier is well-calibrated, preventing both overfitting and underfitting.
BERT and K-Means
In the second stage, Bidirectional Encoder Representations from Transformers (BERT), a deep learning-based natural language processing model, will be employed to refine and validate the creativity-related themes extracted from LDA+SVM. Unlike LDA, which identifies topics based on probabilistic word co-occurrence, BERT generates context-aware word embeddings that capture deeper semantic relationships between words and phrases. This ability makes BERT particularly suitable for analyzing consumer reviews, as it can identify subtle meanings in text that may not be evident in traditional topic modeling approaches. By integrating BERT embeddings with K-Means clustering, this study will enhance the robustness of theme identification by uncovering underlying patterns and improving the interpretability of creativity-related themes in MCCPs.
To generate BERT embeddings, each consumer review will first be tokenized using a pre-trained BERT tokenizer (bert-base-chinese), a model trained on a large corpus of Chinese texts, which will output high-dimensional vector representations encoding the semantic meaning of each review. This tokenizer segments Chinese text into meaningful subwords, ensuring that words are correctly mapped to their corresponding embeddings in the pre-trained BERT model. Instead of reducing dimensionality through PCA or other techniques, this study will retain the full-dimensional BERT feature vectors and apply K-Means clustering directly to group semantically similar reviews into distinct themes. The decision to preserve the full-dimensional embeddings is based on the need to retain as much contextual information as possible, ensuring that fine-grained semantic relationships within consumer reviews are not lost.
Following the extraction of BERT embeddings, the study will employ K-Means clustering to group semantically similar reviews into distinct themes. K-Means is a widely used unsupervised machine learning algorithm that partitions data into clusters based on similarity, making it particularly effective for grouping consumer reviews with shared perceptions. To determine the optimal number of clusters (K), two widely used validation techniques will be applied. First, the Silhouette Score will be calculated for different values of K, measuring how well each data point fits within its assigned cluster while maintaining a clear separation from other clusters. A higher silhouette score will indicate that the clustering structure is well-defined. Second, the Elbow Method will be used to analyze the sum of squared errors (SSE) for different values of K, identifying the point at which adding more clusters no longer significantly reduces variance. This combination of methods will ensure that the most meaningful and well-separated clusters are selected for analysis. Once the optimal number of clusters (K) is identified, the best clustering model will be applied to categorize the reviews, generating the theme clusters that represent consumer-perceived dimensions of MCCPs.
To summarize and extract key themes from the results of BERT + K-Means clustering, DeepSeek R1, and ChatGPT o3-mini-high will be used. DeepSeek R1 excels in deep contextual understanding, enabling it to capture complex themes within clustered reviews accurately. At the same time, ChatGPT o3-mini-high is highly effective in generating concise, contextually relevant summaries from large, unstructured text. The keyword summaries produced by both models will be compared, and their outputs are expected to be broadly consistent. These keywords will then be used to supplement and validate the results of the LDA + SVM analysis, offering further insights into the thematic structure of the consumer feedback.
Text Analysis
To ensure both the adequacy and accuracy of the identified dimensions and their corresponding indicators, a final step of text analysis will be conducted by extracting high-frequency keywords from consumer reviews. This process aims to refine and supplement the dimensions and indicators derived from the LDA+SVM and BERT+K-Means approaches. By systematically identifying frequently occurring terms associated with consumer perceptions, additional relevant indicators will be incorporated to enhance the comprehensiveness of the analytical framework. This step not only ensures that all key aspects of MCCP evaluations were adequately captured but also reinforces the robustness of the categorization process, minimizing the risk of omitting critical consumer insights.
By integrating insights from probabilistic topic modeling (LDA), supervised classification (SVM), and deep learning-based clustering (BERT+K-Means), as well as two advanced LLMs, this study will establish a rigorous and data-driven thematic framework that accurately reflects consumer perceptions of MCCPs. The final themes derived from this hybrid methodology will provide the foundation for PLS-SEM validation, enabling further investigation into how MCCP dimensions influence consumer satisfaction.
Data Structurization
Before proceeding with PLS-SEM, it is necessary to transform the unstructured textual data into a structured form suitable for quantitative analysis. Since consumer reviews are qualitative in nature, sentiment analysis will be applied to derive numerical scores that represent consumer attitudes toward different MCCP dimensions. This transformation process consists of two key steps: sentiment classification and matrix transformation, ensuring that the final dataset is appropriately structured for PLS-SEM modeling.
Sentiment Classification
To classify sentiment within consumer reviews, this study will employ two independent sentiment analysis tools: the sentiment analysis module of SPSSAU (an automated statistical analysis platform widely used for academic research in China) and the sentiment analysis API of Alibaba Cloud (i.e., Aliyun, a leading cloud computing service provider in China, specializing in AI-driven data processing and e-commerce analytics). SPSSAU’s tool categorizes text into four sentiment levels—positive, slightly positive, slightly negative, and negative—providing a nuanced understanding of sentiment variation. In contrast, Alibaba Cloud’s sentiment analysis API, a machine learning-based tool trained on extensive consumer data from Taobao’s e-commerce platform, classifies sentiment into three broader categories: positive, neutral, and hostile. The integration of these two tools will enable cross-validation, thereby improving the reliability of sentiment classification.
Since sentiment classification directly affects the validity of the final quantitative dataset, special attention must be given to the reliability of the sentiment results analyzed by the tools. Given their maturity and reliability, a direct reliability metric, such as Cohen’s Kappa or Krippendorff’s Alpha, was deemed unnecessary. However, for situations where one tool labels a review as positive while the other categorizes it as negative, manual verification will still be needed to determine the correct sentiment classification. Thus, we introduce a procedure of manual judgment, where an expert with more than 8 years of experience in the research field of MCCPs will distinguish discrepancies among the results, ensuring that positive and negative elements are assigned to the appropriate MCCP dimensions.
Given that SPSSAU provides four sentiment levels while Alibaba Cloud offers only three, this study will adopt SPSSAU’s sentiment classification as the primary scoring system, with Alibaba Cloud’s classification serving as a validation tool. To standardize sentiment ratings, the following scoring system will be applied: (1) Positive = 7; (2) Slightly Positive = 5; (3) Neutral =4; (4) Slightly Negative = 3; (5) Negative = 1. This scoring system is designed to enhance differentiation among sentiment levels, ensuring that subtle variations in consumer sentiment are effectively captured in the final dataset.
Missingness Handling
After structuring the dataset into an indicator-sentiment score matrix, missing values may still exist due to incomplete sentiment analysis, partial keyword matching, or cases where consumer reviews do not contain sufficient information to map specific indicators. Since PLS-SEM performs best with complete datasets for optimal estimation, to ensure model validity and robustness, missing values must be handled appropriately before proceeding with modeling.
To address this issue, this study will apply K-Nearest Neighbors (KNN) imputation, a widely used technique for handling missing data. KNN imputation identifies the k most similar observations (neighbors) in the dataset and estimates the missing value based on the weighted average or mode of these neighbors. This approach ensures that missing values are replaced based on patterns observed in the existing data rather than relying on arbitrary assumptions, thereby preserving the integrity of consumer sentiment information. The decision to use KNN imputation instead of Mean, Regression, and Multiple Imputation (MI) is based on methodological considerations related to the nature of the sentiment score dataset: Mean imputation oversimplifies missing data by replacing values with the average, reducing variance and distorting sentiment distributions. Regression imputation assumes fixed linear relationships, which do not capture the diverse linguistic expressions in consumer sentiment. MI generates probabilistic values that may fall outside the predefined sentiment scale, complicating interpretation. In contrast, KNN imputation preserves data structure, maintains scale consistency, and imputes missing values based on real observed patterns, ensuring methodological robustness.
The KNN imputation process will proceed as follows. First, the dataset will be examined for missing values in indicators’ scores. Suppose an indicator is missing from a given review. In that case, it suggests that either the sentiment classification failed to detect relevant content or the review did not contain an explicit sentiment toward that dimension. Next, the number of nearest neighbors (k) will be selected using cross-validation techniques to optimize imputation accuracy, with different k values (e.g., 3, 5, and 7) tested to determine the most effective configuration. Once the optimal k-value is identified, the algorithm will find similar consumer reviews based on the available sentiment scores in other MCCP dimensions. The weighted average of the k neighbors will then replace the missing value. Finally, statistical validation will be performed to compare the distribution of the original and imputed data, ensuring that the imputation process does not introduce bias or distort the dataset’s structure.
The missingness in sentiment scores primarily arises from cases where consumer reviews lacked explicit expressions related to specific dimensions rather than from random or systematic data loss (Kim & Im, 2018; Li et al., 2023; Mellinas & Leoni, 2024; Mitra et al., 2023; Zhou et al., 2024), addressing them through statistical reporting would provide limited additional insights beyond what is already captured by the text-mining framework. Given that missing values were distributed across various MCCP dimensions rather than concentrated in specific categories, KNN imputation was selected to leverage existing sentiment scores and maintain contextual consistency in consumer evaluations.
Reliability Testing
To ensure the reliability of the structured dataset before conducting PLS-SEM analysis, this study will evaluate internal consistency using McDonald’s Omega (ω). While Cronbach’s Alpha (α) has traditionally been the most widely used measure for reliability testing, recent research suggests that McDonald’s Omega provides a more robust and theoretically sound estimate of internal consistency, particularly in cases where the scale exhibits heterogeneity among its indicators (Njeri et al., 2024; Orçan, 2023; Trizano-Hermosilla & Alvarado, 2016). Given that our dataset is derived from consumer sentiment analysis of textual reviews rather than traditional survey-based Likert-scale responses, heterogeneity in item loadings is expected, making McDonald’s Omega a more appropriate reliability metric. Unlike Cronbach’s Alpha, which assumes tau-equivalence (i.e., all items contribute equally to the latent construct), McDonald’s Omega does not require this assumption. Instead, it accounts for the actual variance explained by each indicator, providing a more precise measurement of internal consistency. It is imperative in sentiment-based data, where different MCCP dimensions may have varying levels of contribution to consumer perception. For interpretation, a McDonald’s Omega value above 0.80 indicates high reliability, while values between 0.70 and 0.80 suggest good reliability; values between 0.60 and 0.70 are considered acceptable, and those below 0.60 indicate low reliability and potential measurement concerns. In this study, SPSSAU will be used to calculate the McDonald’s Omega of each dimension and the overall result.
PLS-SEM Model Specification
Given the nature of the extracted dimensions from consumer sentiment analysis, this study will adopt a formative measurement model rather than a reflective measurement model. The choice of a formative specification is based on the theoretical and methodological considerations regarding the nature of MCCP dimensions in consumer perception. A reflective measurement model assumes that latent constructs cause their indicators, meaning that all indicators should be highly correlated and interchangeable with one another. If one indicator is removed, the meaning of the construct remains unchanged. This approach is suitable for traditional psychological scales, where responses to different items reflect an underlying trait (e.g., satisfaction or brand trust). However, in our case, MCCP perception is not a single latent trait that generates consumer sentiment scores but rather an emergent construct composed of distinct dimensions. These dimensions are not necessarily correlated, but together, they define the overall perception of MCCPs. By contrast, a formative measurement model assumes that indicators collectively form the latent construct, meaning that each dimension uniquely contributes to the construct’s meaning. In the context of consumer MCCP perception, dimensions such as aesthetic appeal, craftsmanship, functionality, and emotional engagement independently shape the overall assessment of a product. The absence or variation in one dimension does not invalidate the construct; instead, it alters its overall composition and meaning. Given that our sentiment-based dataset captures diverse consumer expressions toward different aspects of MCCPs, a formative approach allows us to model these dimensions as independent yet complementary contributors to MCCPs perception. Thus, using a formative measurement model in PLS-SEM ensures that our analysis reflects the causal relationship between dimensions and overall perception rather than assuming that consumer sentiment toward one dimension is necessarily correlated with another. This approach provides a more accurate representation of how consumers evaluate MCCPs, allowing us to quantify the relative importance of each dimension in shaping consumer satisfaction.
In PLS-SEM with a formative measurement model, it is crucial to report appropriate indicators while avoiding internal consistency metrics, such as Composite Reliability (CR) and Average Variance Extracted (AVE), which are only applicable to reflective models. Key essential indicators include Variance Inflation Factor (VIF), ensuring multicollinearity is within acceptable limits (VIF < 5, ideally <3.3), and outer weights, which indicate each indicator’s contribution to the construct, with significance tested via bootstrapping (p < 0.05). Path coefficients also should be reported to assess the strength and direction of relationships (range: -1 to 1, p < 0.05). Additionally, R² will measure the variance explained by independent variables (≥0.25 weak, ≥0.50 moderate, ≥0.75 vigorous), while f² will quantify effect size (≥0.02 small, ≥0.15 medium, ≥0.35 large). Optional indicators include Q² (predictive relevance), which is used to assess out-of-sample prediction (Q² > 0 indicates relevance). To determine the primary indicator influencing the dimension of consumer satisfaction, Importance-Performance Map Analysis (IPMA) will provide managerial insights. Because PLS-SEM prioritizes prediction over overall model fit and formative constructs are defined by their indicators rather than inferred from them (Henseler et al., 2016), the model evaluation does not rely on traditional model fit indices (e.g., RMSEA, SRMR, CFI). Thus, the metrics of model fit are optional. At last, PLS-SEM researchers suggest that to detect a medium effect size (f² = 0.15) with 80% power, at least 150-200 cases are required (Cohen, 1992; Hair et al., 2021); thus, the threshold for data size in PLS-SEM is set at 200.
The PLS-SEM analysis will be conducted in SmartPLS 4.0.0.