Online Review Analysis from a Customer Behavior Observation Perspective for Product Development

1. Introduction

Competitive and innovative product design requires a deep understanding of the customer. Currently, companies are making efforts to understand customers and design products from their perspective. However, customer understanding is impossible without empathy for the customer [1]. To achieve this, companies use techniques such as customer observation and interviews to gain insight into their customers. However, due to constraints of time, space, and lack of diverse information, companies face difficulties in this endeavor. Online product review analysis can overcome these problems by allowing companies to observe customers in their natural environment. With the development of smartphones and the internet, companies can easily obtain customer feedback on their products. Online product review analysis has the potential to observe customer behavior without the constraints of time and space [2]. Furthermore, due to the diverse range of people evaluating products, there is a high potential for diversity in the information gathered. This paper proposes an online review analysis approach that can overcome the limitations of traditional customer observation methods. To accomplish this, the Customer Journey Map (CJM) tool was applied to product review analysis to facilitate the text analysis process.

In addition, to infer customer emotions and opinions from their product usage, text analysis techniques were comprehensively applied to derive customer behavior Voice of Customer (VOC). This technique was applied to the product data of true wireless earbuds to compare the differences between VOC derived from customer behavior observations and the results of traditional product review analyses. The validity and effectiveness of the approach was verified through expert interviews.

2. Related work

2.1. Customers Observation and Product Design

To identify customer needs, companies should examine the product from the customer’s perspective [3]. A product design that satisfies customers can be achieved by empathizing with customer behavior and thoughts [4,5,6]. To achieve this, customer observation and interview techniques can be used to determine customer characteristics and gather detailed information on product users and their preferences [7]. These customer observation techniques have evolved into ethnographic methodologies. This approach, which originated in anthropology, introduces a fundamental approach to closely observe and gain insight into customer behavior by entering their world for product design [8,9]. Through customer observation, companies make efforts to understand the customer’s thoughts and behaviors by accessing their world. As a result, companies can develop products that better fulfill customer needs, which is directly linked to the success of new product development [10].

However, there are several issues in applying this methodology to product development. First, there are constraints of time and space. Selecting customers for interviews and customer observation requires substantial effort and time [6,12,13]. However, product release cycles continue to get shorter, demanding short-term customer analysis [12,13,14]. Additionally, in the case of customer observation, spatial constraints may arise depending on the product. Spatial circumstances may make it difficult for companies to observe customers using portable products while on the move.

Lastly, there is a diversity issue in the information acquired. Existing customer observation methodologies have limitations in the number of participating customers, resulting in a lack of diversity in the information obtained, and a tendency towards heavy biases in the analysis results of the analysis subjects [12,14,15].

2.2. Customer Review Analysis and Product Design

The introduction of smartphones and the internet has led to an increase in customer reviews on online commerce and Social Network platforms. This has been highlighted as a new source of finding the VOC. Customer reviews are written with the purpose of providing information to potential buyers who are considering purchasing the product [16]. Reviews that describe detailed usage experiences of the product have sufficient value for companies [2].

Research identifying product features from customer reviews has persisted for an extended period. [17,18,19]. In line with this trend, studies analyzing quantitative VOC based on sentiment analysis of product features have been conducted [20,21,22,23,24,25]. Customer reviews provide a wealth of unstructured customer feedback information. Therefore, they not only include feedback on product features but also encompass various customer behaviors that can assist in customer observation. To explore the targeted data from such, it is important to define the direction of review analysis and design an appropriate method [26]. In this regard, efforts have been made to find useful information for product development [23,27].

If enough customer review data is available, valuable insights into customer behavior can be obtained in a short period of time with less effort compared to approaches such as customer interviews and observations. Additionally, the contribution of diverse customers to the review data helps to overcome the issue of information bias encountered in traditional customer observations [11].

To find insights into customer observation from customer reviews, Customer Journey Map (CJM) can be used as an analytical tool to track the customer behavior journey and analyze the interactions between service providers and customers. CJM is a tool that tracks the customer journey and analyzes interactions between service providers and customers to explore customer experiences [28]. CJM are usually created through direct observation or interviews with customers. It can also be derived from customer review data. This allows us to understand product usage behavior and discern the VOC, also known as behavior VOC. Unlike traditional product feature-based review analysis, behavior VOC captures the needs and feelings of customers as they interact with the product. With such an approach, we can obtain reviews that more vividly reflect customer behaviors, thereby acquiring information that can contribute to the development of new products.

2.3. Customer Journey and Mapping

Customer observation is the act of recording product usage space, customer behavior, events, time, goals, feelings, etc., for a company so that they can produce competitive products and services [29]. In line with this perspective, CJM observes customers’ service usage behavior and tracks their thoughts, evaluations, and emotions step-by-step, conveying them to the service providers. CJM is a two-dimensional matrix with two axes. The horizontal axis consists of the actions, situations, and procedures that customers encounter while using the service, arranged according to their temporal flow. These elements are the touchpoints through which customers interact with the service. CJM then analyzes the emotions, thoughts, and experiences they have while using the service [30]. The vertical axis is where the service provider selects the topics they want to obtain information about the customers at the touchpoints. The service provider can change the topic according to the analysis objectives and select the emotions, thoughts, opinions, etc. of the customers at the touchpoints to analyze.

In this study, we will use the behavior of customer product usage as the touchpoints and place them on the horizontal axis. We aim to extract the behavior VOC and product usage environment information related to the touchpoints from the review data. These elements will then be placed on the vertical axis. Finally, we will apply text data analysis techniques to find information that corresponds to the two axes set in the customer reviews.

To explore information related to customer behavior by applying text mining techniques to CJM, we need to solve one problem. In traditional review analysis, customer feedback was identified by focusing on frequently used noun-centered keywords. These keywords were then used to identify critical product features, followed by each customer’s qualitative and quantitative analysis. We will explore the background and customer product usage behavior of CJM using text mining techniques. Nouns related to product features are typically expressed in one or two words and are used as keywords for analysis. However, analyzing behaviors requires considering various verbs with the same meaning. We plan to group these verbs for our analysis. By focusing more on customer behaviors and verbs than product features and nouns, we expect to find new insights from product reviews. Further details of the analysis method will be discussed in the next chapter.

3. Research Model

3.1. Creating a Customer Journey Map from the Perspective of Product Usage

We aim to apply the CJM used in services towards products to derive behavior VOC through customer product usage observation. To achieve this, we will examine the key elements of CJM in previous studies and apply them to customer product usage behavior.

The core elements of CJM defined the service usage process and the customer’s needs and emotions as observed during interactions between service providers and customers [31]. In the case of a product CJM, the service usage process can be defined as the product usage process, while the customer’s needs and emotions can be defined as behavior VOC. From a product perspective, the interaction in CJM can be divided into stages of customer product usage behavior, which can be analyzed in a step-by-step manner. These steps can be divided into three stages: the installation and maintenance stage before the customer uses the product, the stage where the customer uses the product, and the last stage where the product malfunctions after the customer uses it. This classification is based on previous research [32,33].

Once all the CJM elements from the product development perspective are defined, they are classified in detail using text data analysis techniques and judged as touchpoints of CJM. The CJM overall map is then created with touchpoints on the horizontal axis and background, behavior VOC on the vertical axis. The creation of the entire map is carried out in two stages. First, customer product usage behavior is explored from review data to identify CJM touchpoints which is then segmented into stages (Process). Next, VOC and environment information are derived from the touchpoints. The completed CJM map is used as a guide for review data analysis and is also used to summarize analysis results in a table, as shown in Figure 1.

3.2. Model for Customer Review Analysis through Product CJM

3.2.1. Data Acquisition

Customer review data can be collected from channels such as SNS and online commerce. The review text data is collected using a web parser.

3.2.2. Data Preparation

To facilitate review analysis, data preprocessing is performed. In this study, three preprocessing steps were conducted: fake review filtering, text preprocessing, and part of speech (POS) tagging.

Fake review filtering is a process of removing intentionally biased or exaggerated reviews, which can have a negative impact on product purchasing decisions, from the collected reviews. In this process, we filtered fake reviews using a word arrangement analysis-based machine learning classifier [34]. Then, we conducted text preprocessing on the filtered review data to remove unnecessary special characters and convert all characters to lowercase. Finally, we reduced the text data volume by selecting only the parts of speech that match the analysis objective using the POS tagging method [35]. In this study, we included verbs in addition to commonly used nouns and adjectives because the analysis focused on customer behavior.

3.2.3. Touchpoint Exploration

In the preprocessed data, we identify touchpoints by analyzing customers’ product usage behavior in a step-by-step process. In this process we 1) explore touchpoints in the CJM from the perspective of customer product usage and 2) break down the product usage process at the touchpoints. When determining touchpoints from customer behavior, various verbs that convey the same meaning must be taken into consideration. Hence, to include more reviews on customer behavior in the analysis, it is essential to consider not just individual words but also groups of words with the same meaning. This approach sets our method apart from previous Latent Dirichlet Allocation (LDA) based review analysis related studies [36,37]. Therefore, we group similar verbs together to determine touchpoints. To group words, we use word embedding to calculate the vector value of the position of words in a sentence and use K-Means clustering based on this [38,39]. After the word grouping is completed, we evaluate the meaning of each group to identify touchpoints and processes. We determine whether each word group belongs to the stage of product use before, during, or after the touchpoint. Then we examine whether it can be further separated into a product use process.

3.2.4. Behavior VOC Exploration from Touchpoint

To identifying the touchpoints of the CJM and product usage stages, the behavior VOC and product usage context are explored at each stage. To facilitate exploration, keyword groups matched to each stage are employed to select related reviews from the entire review dataset. The selected reviews are then analyzed for keywords-related behavior VOC using the LDA technique [40].

Prior to selecting the reviews, they were segmented into sentence-level units. This approach was taken to prevent the LDA results from becoming overly concentrated on dominant words frequently used when analyzing entire review paragraphs. The simplicity of this CJM mapping process is illustrated in Figure 2.

4. Empirical Study

4.1. Data Acquisition from Online retail site

A web parser was developed in Python to collect reviews of the TWS (True Wireless Stereo) Earbuds product category from the Online retail website, Amazon (www.amazon.com). Data were collected for 93 TWS Earbuds products with more than 300 reviews each. The data collection period spanned from August to September 2020, yielding a total of 225,241 reviews. The data categories included release year, actual product purchase verification, product classification, and review content.

4.2. Data Preparation for the Acquired Data

To eliminate deliberately misleading fake reviews that could cause confusion, the filtering technique of Support Vector Machine (SVM) was applied [41]. Pre-determined Amazon review dataset was used as the training data [42]. Out of the total 225,241 reviews, fake reviews were removed, leaving 179,165 reviews for analysis.

To alleviate the computational burden of review analysis and to analyze customer behavior centered on verbs as parts of speech, each word was tagged and selected according to its part of speech. This process employed the Penn Treebank tag set from the NLTK library [43]. Since the customer behavior is the key for review analysis, we selected the following parts of speech: verbs, adjectives, and nouns.

4.3. Touchpoint Exploration in the use of TWS Earbuds

To assign vector coordinate values to words based on their position within a sentence and group words with similar meanings, we employed The Continuous Bag of Words (CBOW) algorithm of Word2Vec [44]. During the implementation of the algorithm, words with a frequency of less than 100 were excluded. And the vector size was set to 100, with a window size of 5. K-Means clustering was then employed to group words with similar coordinate values. Due to the configuration of the window size in Word2Vec being set to 5, the number of fitting words within each cluster was accordingly determined to be the same. From the word groups, the touchpoints of the CJM are determined and the product usage stage—before use, product use, or after use—is identified. For example, the words [‘charge’, ‘charging’, ‘charged’, ‘charger’, ‘power’, ‘charges’, ‘recharge’, ‘plugged’, ‘recharging’, ‘recharges’] can be considered as part of the before use stage, representing the customer activity of charging the product. Using this approach, 9 groups related to customer behavior were selected from 527-word groups, with the results shown in Table 1. Detailed information about the word groups and their constituent words can be found in Appendix Table A1. Two stages were identified in the before use stage: customers installing the product and charging it. During the product use stage, six stages were discovered based on various situations in which customers used the product. In the after-use stage, behavior associated with product malfunction was identified. A closer examination of the product usage stages based on the frequency of word occurrences in each group revealed that listening to music had the highest usage ratio for earbuds. Additionally, it was confirmed that mentions of situations such as making phone calls or exercising were more frequent than expected.

4.4. Behavior VOC Exploration from TWS Earbuds Touchpoint

The 179,165 reviews were divided into 647,015 sentences, and sentences were selected using the constituent words of the behavior focused word groups as keywords. By applying the LDA to the selected sentences, the main topics mentioned in the sentences can be identified. These then become the behavior VOC for those keywords. In the LDA analysis process, the optimal number of topics was determined for each group of selected texts, where the perplexity was low, and the coherence was at its maximum. The number of words output was set to five in order of contribution. Table 2 below shows the main behavior VOC of customers as recorded by applying LDA analysis to review sentences related to music listening behavior in the CJM.

Each word group can be further analyzed for behavior VOC by applying frequency analysis and Word2Vec. For example, in the music listening stage, customers evaluate items related to bass and treble. To determine whether customers were more interested in the bass or treble, the importance of ‘highs,’ which is used synonymously with ‘treble,’ was evaluated using Word2Vec. Mentions of ‘bass’ were found 15,904 times, and those of treble frequencies ‘treble’ and ‘highs’ totaled 3,854 occurrences (1,154 and 1,700, respectively). Based on these results, it can be inferred that customers consider bass the most important, followed by treble.

By repeating this process, behavior VOC based on the CJM framework can be mapped. First, touchpoints belonging to before-use, product-use, and after-use are arranged in the horizontal rows. Here, the touchpoints include Setup and Charge in the before-use stage, and Music, Video, Game, Phone Call, Sports, and Move in the product-use stage, as well as Failure in the after-use stage. Next, information related to the product usage environment associated with the touchpoints is placed at the top of the vertical columns. Below that, the thoughts and feelings customers have while using the product are described as VOC based on their behavior. The completed CJM framework-based behavior VOC is shown in Figure 3 in next page, and a larger image is included in Appendix Figure A1.

Furthermore, the product usage environment was clarified for each customer behavior stage. The environment includes the purpose of use, related smartphone apps, and additional descriptions of usage behaviors. This information helps product developers better understand customer behavior. In the VOC items, emotions and thoughts experienced by customers at each stage are expressed as VOC. Online reviews contain customer product evaluations and judgments based on their product usage behaviors. The previously mentioned behavior VOC along with background information allow for indirect observation of customer actions.

It was found that collecting and analyzing data took approximately one week, excluding the development of the analytical algorithm. Although the time required may vary depending on the situation and objectives, direct customer observation in this case required 3 months [45]. The customer interview method through surveys took at least 1 month to collect questionnaires [46]. In comparison to these methods, this project was able to explore customer observation information within a shorter period. Furthermore, less personnel were involved in understanding the meaning of word groups and identifying behavior VOC from the LDA analysis results. This demonstrated that fewer human resources were utilized in comparison to traditional customer observation methods, where long-term customer behavior must be closely monitored and analyzed in real time as indicated [47].

5. Discussion

In the previous chapter, we demonstrated that customer reviews of TWS Earbuds allow us to identify stages and touchpoints of customer product usage behavior, enabling customer observation with less time and effort. We aim to examine the uniqueness of the behavior VOC resulting from customer observation and validate its contribution to product development.

5.1. Uniqueness and Contribution

In this section, we examine the uniqueness and contribution of the online review analysis results. The unique feature of this study’s online review analysis is the examination of customer behavior. By analyzing customers’ product usage behaviors, environments, and opinions, we acquire knowledge about the causes and processes leading to product evaluations. This represents the unique information of this online review analysis study from a customer observation perspective.

In contrast, most of the previous research on online product review have primarily focused on identifying and analyzing the core product features evaluated by customers [17,20,21,48,49,50,51,52]. However, an analysis skewed towards product features is not suitable for gaining insights into the causes and processes that lead customers to evaluate products. In contrast, this study analyzes customer product usage behavior to derive the causes and processes behind product evaluations.

To verify this, the study first compares the qualitative VOC derived from traditional product feature analysis with the behavioral VOC from this study. As a result, the unique aspects of this study in relation to product feature analysis are identified and its contribution to product development is considered. To identify differences, the same data set used in the research is employed to locate product feature elements and compare them with the results of this study. Previous research focused on identifying core product features by concentrating on noun, whereas this study emphasizes understanding product usage behavior through verb. Therefore, while previous research focus on understanding customers’ direct needs through evaluations of the product’s core features, this research centers on analyzing customer product usage behavior to derive the reasons and processes behind their product evaluations.

The analysis revealed that previous research identified core product features evaluated by customers, such as Sound Quality, Battery, Ear Fits, Case, Charging, Noise Cancellation, Waterproof, Connection, Phone Call, and Equalizer. In contrast, this study derived elements of customer product usage behavior, such as Setup, Charging, Listening to Music, Watching Videos, Gaming, Making Calls, Engaging in Sports, and Using While Moving. The difference becomes clear in the qualitative VOC identified through LDA analysis of online reviews using each word group as keywords.

Comparing the VOC from the two analyses, the difference in the knowledge gained through online review analysis becomes clear. While previous research focuses on understanding customers’ direct needs through evaluations of the product’s core features, this study centers on analyzing customer product usage behavior to derive the reasons and processes behind their product evaluations. For instance, in the product feature analysis the Connection VOC reveals that customers evaluate connection stability, but it is difficult to not clear why they consider it important. However, the results of this study show that customers are sensitive to sound delays while watching videos and playing games. As another example, customers in the product feature analysis evaluated the importance of battery capacity. However, this study reveals that customers assess battery capacity while listening to music, making phone calls, and engaging in sports. The specific differences in VOC can be seen in Figure 3 on the previous page and in the product feature analysis Table 3, with unique behavioral VOC highlighted.

The VOC from a behavioral perspective provides unique VOC not observable in product feature analysis, aiding in understanding the reasons and processes behind customer product evaluations. This understanding can offer product developers insights into customers’ indirect experiences and potential needs, contributing to the development of new products.

5.2. Validation

In this section, we aim to verify the practicality of the analysis results for product development. To verify the effectiveness, the same data was analyzed as a quantitative VOC for sentiment analysis based on existing product features. Then TWS Earbuds developers evaluated the results. Sentiment analysis enables researchers to quantitatively understand customer evaluations of product features. Various studies have been conducted in this area. Some studies focused on finding quantitative customer evaluations based on product features [20,45,53]. Other studies aimed to identify product weaknesses by comparing analysis results with competing products [54]. Some studies suggested a method to automatically identify positive and negative sentiments by finding similar words using Word2Vec and conducting sentiment analysis [55]. Each study shares a common goal of quantifying customers’ positive and negative emotions toward products and determining their relative importance.

Sentiment analysis was performed using the unsupervised learning based SentiWordNet [56]. The results from this analysis are tabulated in the Table 4 on the following page. The categories resemble those of the previously mentioned qualitative VOC for product features. However, the results are presented as quantitative VOC by ranking positive sentiments and frequency.

To compare differences between analysis results, we interviewed TWS Earbuds product development experts. Participants were three developers from LG Electronics’ TWS Earbuds. Interviews were conducted via an online survey, with questions from the Technology Acceptance Model perspective. Focus was on the usefulness and ease of use when developers utilize analysis results in product development [57].In comparing the VOC of this study’s customer behavior observation (Figure 3.) with the quantitative feature analysis (Table 4.), experts positively evaluated the results of this study in terms of usefulness, as they helped to acquire information and develop product concepts for new product development. They found it easy to understand the purpose of customers using the product and to comprehend detailed customer thoughts at each stage of use. In addition, they positively evaluated the ability to confirm TWS Earbuds product usage behavior and context. In particular, the use of the call function for conferences or meetings and the VOC in the sports category were considered useful information obtained from customer behavior observation analysis.

Regarding ease of use, the study received good evaluations for understanding customer product usage behavior at each stage. However, in comparison with previous studies, it was considered insufficient for applying analysis results to product development from a product improvement perspective. This is because the existing product feature-centered quantitative VOC was much more intuitive in verifying customer evaluation results, receiving relatively higher ratings for ease of use.

Lastly, the experts emphasized that while quantitative VOC is important for determining Technology Priority and the detailed product specifications of required technologies, this study’s results are more useful for the refinement of new product concepts and technologies. Ultimately, they mentioned that despite the differences in results from evaluating the same data from different perspectives, it should primarily be utilized in product development.

Through the interview results, we confirmed that this study provides useful information that can be utilized complementarily with existing sentiment analysis research. Furthermore, it is significant that companies can both understand and empathize with customers which is essential to the development of new products, by mapping customer behavior CJM from review data. The CJM in this study is specialized for observing and empathizing with customers in a phased manner. This approach is essential for developing new product development concepts. The CJM used in this review analysis study serves as a tool for exploring latent customer requirements and solutions [58]. This is why it receives positive evaluations from product developers for its contribution to new product development concept.

We have also discovered that review data can yield different information contributing to product development, depending on the analytical perspective and the selection of parts of speech for analysis and changes in visualization. This suggests that, going forward, varying the analytical approach based on the perspective can uncover different types of information.

6. Conclusion

In this study, we aimed to define customer behavior inherent in review data, specifically focusing on customer product usage and consequently exploring the VOC through this observation. We utilized the CJM as a guide for investigating interactions between customers and products, enabling us to identify behavior VOC. To accomplish this, we included verbs uncommonly used in conventional review analysis studies and employed techniques such as Word2Vec and clustering to group different verbs with similar meanings. Subsequently, we applied LDA around the keywords associated with these groups to identify VOC related to customer behavior.

We have confirmed that our proposed method, when applied to TWS Earbuds review data, allows for the identification of customer product usage behavior and VOC in a shorter time span. This approach requires fewer resources compared to traditional customer observation methods such as interviews, filming, and surveys. This review-centric customer behavior observation approach can accommodate rapid product development cycles and is expected to be beneficial to individuals and developers in small-to-medium enterprises (SMEs) who may struggle with traditional customer observation.

From the perspective of customer behavior review analysis uncovers information about the reasons and processes behind customers’ product evaluations, which was difficult to find in traditional product feature-based review analysis. Through expert interviews, we validated the utility of review information from a behavior observation standpoint, confirming it as a valuable resource for product developers to understand customers, generate new product development concepts, and prioritize technologies.

We acknowledge that review and text data are vast and, depending on the selection of keywords and analysis methodology, can yield diverse customer VOC. While this study utilized CJM as an analysis guide to clarify customer behavior and its associated VOC, there remains potential for unearthing new insights. Future research may explore the feasibility of extracting useful information which can be used in fields other than product development.