Nonverbal Behavior of Service Robots in Social Interactions – A Survey on Recent Studies

1. Introduction

A growing number of service robots are emerging from factories and research laboratories into people’s daily lives. In terms of financial numbers, the whole service robotics market is forecasted to grow from 41.5 billion USD in 2023 to 84.8 billion USD in 2028 [1]. A service robot, according to one of its earliest definitions, is “a freely programmable kinematic device that performs services semi-or fully automatically,” whereas services here are understood as “useful work for humans and equipment” [2], excluding industrial automation applications (see also [3,4]). More recently, service robots are increasingly expected to make use of AI and natural communication [5] to be able to operate in an uncontrolled environment [6,7], enabling them to act as socially-capable assistants [8]. Such social service robots are becoming increasingly important in various fields of human life, enhancing efficiency in production or alleviating the lack of workforce in other areas such as education or healthcare [1,9,10]. In everyday life scenarios, people can increasingly encounter these robots in public spaces without warning. To preclude minor conflicts, robots navigating shared physical spaces with humans will likely need to conform to the same general interaction norms applicable to people. As the assimilation of robots into diverse aspects of human existence continues to expand, a nuanced comprehension and effective utilization of non-verbal cues emerge as imperative for achieving harmonious and efficient human-robot interactions [11]. For instance, when a person encounters a delivery robot on a narrow sidewalk, mutual understanding of intentions is crucial for a safe and efficient passage, allowing both parties to proceed along their respective paths seamlessly. Consequently, service robots operating amidst ordinary individuals must be ready for social interaction [12]. These robots should have the capacity to interact with people, move alongside them, while with some of them, their control can be taken over remotely by a human operator or they can be used as telepresence robots for direct interaction between physically present and remote participants. In the context of this paper, we focus on service robots that have significant social capabilities (social service robots) being able to participate as independent social entities in communication situations. The subcategories of such social service robots that fit this description are social robots and robot-assistants. According to [13], a social robot is an intelligent machine characterized by its ability to engage in interactive and conversational exchanges, embodying features such as automated personal assistance, ambient assistive living technologies, and computational intelligence in the realms of games, storytelling, health care, and education services, reflecting a shift from traditional notions of purpose-built machines. Whereas a robot-assistant, as stated by [14], is an autonomous physical device capable of providing services independently, offering benefits such as reduced service time, accessible information, personalized service, and consistent, albeit limited, service quality.

Modern social service robots are state-of-the-art machines that can be quite expensive. For example, the 5G capable TEMI robot assistant costs $7500, and to use all of its features, an additional subscription is needed with the yearly price of $1500 [15]. Another robot assistant by LG, CLOi, costs $39,999 [16]. In order to justify the implementation costs and successfully capitalize these robots, their human users need to be skilled and motivated to accept them [9]. The efficient use of social service robots seems to depend on their users’ self-efficacy and is strongly influenced by the robot’s natural communication skills, especially its nonverbal communication abilities [5,9,17]. The discernment of human emotions stands as a pivotal aspect of HRI facilitated by non-verbal cues. The tacit language of facial expressions, gestures, and body language emerges as a conduit for deciphering and responding to human affective states. This interpretative acumen engenders a more empathetic and adaptive robotic interaction, transcending the confines of explicit verbal communication.

In the establishment of trust and rapport, non-verbal cues assume a critical role. In collaborative work settings where humans and robots converge, the ability of robots to convey attentiveness through judicious employment of eye contact, nodding, and other non-verbal manifestations contributes substantively to the perception of reliability and mutual cooperation [12,18]. This, in turn enhances the efficacy of human-robot collaborative endeavors. Moreover, the integration of non-verbal cues substantially contributes to the enhancement of the overall user experience, rendering interactions more organic and intuitive [19]. In domestic environments, for instance, a robot adept at discerning and responding to non-verbal prompts, such as gestures or pointing, seamlessly integrates into the fabric of daily life. This adaptive responsiveness fosters a more fluid and humanized exchange.

Due to their relatively high acquisition and implementation costs [20], it has great economic importance to understand the influence of the social service robots’ nonverbal skills on their efficient implementation. For this end we conducted the literature review, where we explored the general statistics of available papers (i.e., the number of available papers, their publishing years, the main contributing countries and authors), and also studied the more specific aspects of the literature, such as the robots used in the experiments, the main research aims and the nonverbal cues examined, the methods used for obtaining data, and the main results of the studies.

Given the aforementioned context, the question of providing social service robots with adequate nonverbal communication abilities becomes increasingly important as it holds the keys to efficient use of this promising although relatively expensive emerging technology. Our aim is to conduct a literature review to examine the state of the art of the research conducted on the real-life use of nonverbal behavior by social service robots in HRI. We will focus on the use of nonverbal characteristics such as distance, personal space, touch, gaze, and others as expressed or observed and interpreted by physical (not simulated) social service robots in their communication with physically present humans in a social environment. We have formed the following research questions to guide our study:

• What is the state of the art of literature about nonverbal interaction in HRI with social service robots? We are interested in the following general metrics:

  a)

  the number of available articles;

  b)

  the publishing years of the papers;

  c)

  the main contributing countries; and

  d)

  the main contributing authors.

• In addition our interests lay in the following specific aspects of the studies:

  a)

  What robots were mainly used in the studies?

  b)

  What were the main research aims?

  c)

  What types of nonverbal communication were mostly examined?

  d)

  What methods were used for research?

  e)

  What are the main results of the studies?

2. Materials and Methods

The goal of the study was to understand which nonverbal behavior cues (e.g., distance, personal space, touch and gaze) were considered relevant in the literature about HRI between people and social service robots (i.e., robot assistants or social robots) in higher education. For these purposes, we conducted a literature review of open access papers, basing our search on three well-known, academically respectable and most relevant search engines [21]: Web of Science [22], Scopus [23] and EBSCO Discovery Service [24]. We formed our search formula as follows, using the terms “robot assistant” and “social robot” in order to narrow down the search results to match better our focus on service robots with social abilities:

(HRI) AND ((robot assistant) OR (social robot)) AND ((non-verbal) OR (nonverbal))

We used the terms “robot assistant” and “social robot” because these are service robots with dedicated social capabilities, and also because the search phrase “(HRI) AND (SERVICE ROBOT) AND ((NONVERBAL) OR (NON-VERBAL))” gave significantly less matches (e.g., only two papers with full text available in EBSCO Discovery).

Next, we refined our search parameters by restricting the retrieval of articles to those classified as "Full text available" within EBSCO Discovery and those falling under the category of "Open access papers" in both Scopus and Web of Science. This selection process, as illustrated in Figure 1, was executed on the 28 October 2023, and it yielded 150 articles across these platforms: EBSCO Discovery yielded 36 papers, Scopus contributed 57, and Web of Science presented 57 papers. Subsequently, to ensure the quality of our dataset, we identified and eliminated any duplicate entries. This curation process left us with 88 distinctive research papers. Next, we examined the selected papers, excluding any that did not align with the criteria of substantial academic content, such as posters, one-pagers or documents that were not peer reviewed. After this thorough examination, we were left with a refined set of 81 academic papers.

Our primary objective was to chart research on real HRI with service robots. We excluded purely theoretical studies due to their inability to incorporate modern technological advancements, emphasizing the importance of examining the human element in real-world HRI experiments while considering the surrounding context [25]. In addition, we excluded studies where robots were presented through mediating tools, such as computer screens, virtual, or augmented reality. After this exclusion, 39 papers remained to be studied (Figure 1 and Appendix A).

Figure 1. Literature selection process flowchart.

Figure 1. Literature selection process flowchart.

In the analytical phase, the chosen papers underwent a meticulous process of open coding, conducted by two researchers. Their objective was to discern and delineate clusters of meaning that encapsulated the nuances of nonverbal interaction cues within Human-Robot Interaction (HRI). This intricate coding procedure involved thorough scrutiny and interpretation of the content. Any disparities in coding were addressed through thoughtful discussions, ensuring a harmonized and comprehensive extraction of themes.

3. Results

Our final selection of papers concerning nonverbal communication in HRI involving service robots with social capabilities, referred to as social robots or robot assistants, was based on a search conducted on October 28, 2023, resulting in the inclusion of 39 works. This collection of papers can be characterized using the following key metrics:

Firstly, these papers emanate from 20 distinct countries. Among these, the most prominent contributors were the United States and Japan, each with 8 papers, followed by the United Kingdom and Italy, both contributing 7 papers. Additionally, the Netherlands and France each had 5 papers, while Germany, Sweden, and Belgium all contributed 3 papers. Next, there were in total 167 individual authors associated with these papers, with only 16 of them having authorship in more than one paper. Notably, Francesco Rea emerged as the most prolific author with 5 papers, followed closely by Alessandra Sciutti, who authored 4 papers. Finally, the publication dates of these articles span from 2006 to 2023 (Figure 2). An analysis of this data suggests that the popularity of the subject matter concerning the nonverbal communication skills of service robots experienced a decline during the years 2015-2018. However, it is worth noting that the unique circumstances of the recent COVID-19 pandemic appear to have rekindled interest in this topic. Beginning in 2018, the number of papers addressing this subject has displayed a consistent upward trend. The relatively lower number of papers available for the year 2023 can be attributed to the fact that our search was conducted in October 2023, and many papers for that year were still in the process of being published.

Next, we will examine the results for the following specific aspects of the selected studies: the robots used; the main research aims; the types of nonverbal communication mostly examined; methods used; and the main results.

3.1. Robots Used in the Examined Studies

In the empirical parts of the examined papers, various robot platforms were used to investigate the dynamics of communication and engagement between humans and robots. Next, we will provide an overview of the most prevalent robot designs and types employed for experiments in these studies.

Humanoid robots are a dominant category, representing 27 instances across the examined studies. Among humanoid robots, the NAO robot stands out as the most popular choice, with mentions in 10 articles. The NAO robot's humanoid appearance, coupled with its programmable capabilities, seems to make it a versatile platform for investigating nonverbal communication in HRI. iCub, another humanoid robot, is featured in 6 studies, and Pepper, together with Robovie II, (both with 2 mentions) also contribute to the diversity of humanoid robots in the studies.

Bust and face robots provide a unique dimension to HRI research, with 4 instances identified in the examined papers. Furhat [26] and SociBot [27,28] are notable representatives of this category. These robots, designed to focus on facial expressions and interactions, offer insights into nonverbal cues and facial communication between humans and robots.

AIBO, the robot dog [29], and Pleo, the robot dinosaur [30], are featured in 2 studies, emphasizing the interest in exploring how animals' behavior and interactions with humans can inform robot design and nonverbal communication, but also the potential for unconventional robot designs to engage with humans.

The Roomba iRobot Create 2, a robot vacuum, appears in 1 study [31]. Although not a typical HRI robot, its inclusion suggests the exploration of nonverbal cues and interactions with non-humanoid robots in domestic settings.

Five studies [32,33,34,35,36] highlight robots with product-oriented appearances designed to maximize dedicated functions (see [37]). These robots represent a different approach to nonverbal communication and human-robot engagement, emphasizing specific functions and purposes.

The selection of robots in academic studies on nonverbal communication in HRI reflects a broad variety of robot types and designs. Humanoid robots, such as NAO and iCub, dominate the field, highlighting their adaptability for studying nonverbal communication. In addition, bust and face robots, a robot dog, a robot vacuum, and even a robot dinosaur provide unique approaches on nonverbal interactions between humans and robots. The inclusion of product-oriented robot appearances further diversifies the robot platforms used in these studies, highlighting the importance of nonverbal communication in various contexts (e.g., in production or warehouses).

3.2. Main Research Aims

The predominant research aim among the selected academic papers is to assess various facets of HRI. This overarching objective encompasses studies that scrutinize the technical proficiency of robots, gauge trust levels, probe connections between familiarity and engagement, and explore the consequences of robots on human behavior and emotions. We have categorized this wide-ranging array of research objectives into five discernible categories.

Understanding Human-Robot Communication and Interaction. Researchers have embarked on comprehensive investigations to unravel the intricacies of human-robot interaction from diverse vantage points. For instance, one study delved into the acceptance and utilization of robots, with a particular emphasis on their roles in interactions with children diagnosed with autism [38]. Another scholarly work sought to establish correlations between human verbal behavior and robot behavior predicated on the personality traits of the interacting humans, distinguishing them as introverted or extraverted [39]. Trust and the impact of nonverbal communication were paramount in a different study [40], while another explored the influence of robot nonverbal cues on individuals' perceptions of the robot's social agency [27]. Cross-cultural interactions and the efficacy of robot greetings and gestures were examined in a particular context [41,42]. Researchers aspired to devise empathetic and multi-modal interactions grounded in the emotional-cognitive profiles of users [43].

Emotional Interaction. The study of the emotional dimensions of human-robot interaction has constituted a point of high interest for some researchers. One study probed the impact of a humanoid robot on human emotional responses, particularly within cognitively demanding tasks [25]. Another investigation delved into the realm of personality in human-robot interaction, utilizing AIBO, a social robotic pet developed by Sony [29]. Furthermore, researchers delved into emotional expression and mood recognition in robot-human interactions, seeking to comprehend their effects on participants [44,45].

Robot Learning and Teaching. The nexus of human-robot interaction extends into the domain of learning and teaching. Researchers evaluated the performance of machine learners when instructed by human teachers [46]. Simultaneously, they probed the manner in which humans construed and integrated robot actions within their own plans [47]. A distinct study grappled with the challenge of evaluating engagement in human-robot interaction by amalgamating verbal and nonverbal behaviors [48].

Cognitive and Behavioral Aspects. Cognitive facets of interaction have assumed a central role for specific researchers. They endeavored to devise cognitive architectures for robots, facilitating close collaboration with humans in cooperative tasks [49]. The examination of the role of interaction dynamics and gestures within human-robot interaction was a pivotal focus within a study, particularly in joint tasks [50]. In addition, the perception of a robot's speech and its influence on language acquisition during interactions were subjects of investigation [51]. The inner workings of engagement and the formulation of a model delineating its components underwent scrutiny as well [30]. Other studies scrutinized the effectiveness of gaze-control techniques and control interfaces in shaping robot behavior [52] and monitored visual attention and head pose within human-robot interaction [53]. Implementing gaze behavior in functional robots to assist humans in deciphering their intent assumed primacy for another cohort of researchers [31].

Robot Behavior and Communication. The formulation of methods for inducing and detecting user affect within human-robot interactions has emerged as a noteworthy area of exploration [54]. Additionally, the impact of non-verbal communication on the efficacy and teamwork of human-robot partnerships has been a pivotal focus [36].

These diverse thematic clusters encompass a comprehensive array of research aims, reflecting the depth and breadth of scholarship within the field of human-robot interaction. From deciphering the psychological underpinnings of these interactions to enhancing robot behavior and communication, the research continues to advance our comprehension of this evolving domain.

3.3. Types of Nonverbal Communication Examined

The focus of our study encompasses the nonverbal communication cues investigated in the selected papers. These cues play a pivotal role in understanding and enhancing the quality of interactions between humans and robots. We categorized the nonverbal communication cues found in the selected papers as follows:

Gesture and Facial Expression. A substantial body of research has explored the use of gestures and facial expressions in HRI. Non-verbal gestures, including hand movements, head poses, and bodily movements, have been a common focus in HRI research. These gestures serve as crucial elements in conveying information, emotions, and intentions in human-robot interactions [31,49,50,53]. Researchers have also delved into conveying emotions such as sadness, boredom, and happiness through gestures and expressions. This includes the study of expressive motions and the mimicry of human gestures by robots. Additionally, facial expressions like happiness, sadness, surprise, anger, and laughter have been scrutinized as essential nonverbal communication elements in [29,38,55,56] and others.

Affective States and Emotions. Researchers have explored affective states and emotions in HRI, examining how robots can detect and respond to emotional cues from humans. This includes the investigation of emotional expressions, such as arousal-valence and action units [25,55,57,58].

Eye Gaze. The direction of eye gaze, as well as the dynamics of gaze via head movement, has been a focal point of several studies. Eye gaze serves as a crucial means of nonverbal communication, influencing the understanding and interpretation of social interactions in HRI [25,28,42,47, and others].

Voice and Speech Cues. Aspects related to speech, including speech volume, speed, pitch, style, vocabulary, topic selection, feedback, syntax, and backchannel, have been examined. These factors contribute significantly to nonverbal communication, impacting the quality of human-robot conversations by influencing mutual engagement and attentiveness [26,48,59].

Human-Robot Proximity and Movement. Studies have explored physical and psychological distances between humans and robots in various contexts, such as movement within a room. The extent and speed of movements, as well as distances during interactions, have been scrutinized to understand how these factors affect communication and engagement [42,60].

Cultural Aspects and Greetings. Cultural differences in nonverbal communication, particularly in greetings and gestures, have been a point of interest. Understanding how cultural norms influence nonverbal behavior in human-robot interactions has been a key theme [41,42].

These categories of the wide range of nonverbal communication cues play a significant role in shaping the effectiveness of service robots during human-robot interactions, enhancing engagement, trust, and the overall user experience.

3.4. Sample and Methods Used

3.4.1. Sample Size and Characteristics

All reviewed papers reported the size of their human samples. The average sample size across these studies is 27, with the largest sample consisting of 120 participants and the smallest encompassing 5 individuals. Notably, a subset of 5 papers specifically focused on children as their sample. Among these papers, the sample sizes varied, with the largest involving 67 child participants and the smallest comprising 7. In a distinct context, university students were the focal group in 16 of the examined papers. Within this category, the average sample size stood at 34 participants, with the largest encompassing 120 individuals and the smallest comprising 12.

Regarding the gender distribution within the examined papers, a notable equilibrium was observed. On average, the samples consisted of 46% female participants and 54% male participants. This balanced gender representation within the studies highlights the diversity and inclusivity of human participants in human-robot interaction research.

Age data was provided in 27 of the reviewed papers. The average age of participants across these studies was approximately 27 years. The range of ages within the samples varied, with the eldest participant reported as 84 years old and the youngest participant as young as 3. This comprehensive age range underscores the inclusivity of participants across different age groups in the realm of human-robot interaction research.

3.4.2. Data Collection

The reviewed studies employed a wide array of data collection and analysis techniques. Based on the sources of the data, we categorized them into three primary groups: (a) data generated by participants, such as interviews and questionnaires; (b) data measured during the experiments, including distances and ranges of hand gestures; and (c) alternative approaches for data collection. These categorized methods played a pivotal role in achieving a comprehensive understanding of human-robot interactions, encompassing both objective and subjective data, as well as diverse and innovative approaches.

To begin with, the direct collection of data from participants heavily relied on questionnaires. For instance, [32,39,56,61] utilized Likert scale questionnaires to assess various aspects of human-robot interaction. [45] and [51] employed pre-test and post-test questionnaires. In [43] the participants were requested to complete the Chat-bot Usability Questionnaire. In [28] the length of human speech was measured through questionnaires, while in [58] data was gathered using TIPI and RoSAS questionnaires. In [54] non-verbal emotional behaviors were measured through EEG signals. In addition, structured interviews were conducted by [46], and participant feedback was solicited by [40].

Moving on, the studies also collected objective data through various methods. Video analysis was a prevalent technique in the studies of [38] and [62], where it was employed to scrutinize social interactions and initial reactions of children with autism. In [61] it was used to assess the adequacy and intensity of human responses, while [55] applied it for automatic personality prediction. In [25] the impact of humanoid robots on human emotional responses was explored through video analysis. In [30] this method was also employed to test a model specifying the components of engagement. In [58] video analysis was utilized to investigate the extent to which a humanoid robot can influence someone's comfort. In [53] visual attention was monitored using this technique, and in [44] video analysis was used to recognize valence and arousal. In addition, the authors also used techniques such as collecting Inertial Measurement Unit sensor data about micro-movements [61] or measuring physiological signals and audio-visual behavior to collect physiological data [55] or motion sensors for gathering synchronized information streams [53].

Finally, the researchers of the selected studies also employed some unique or diverse data collection methods. For example, [54] used non-verbal emotional behaviors designed to elicit user affect, which was measured through EEG signals; in [34] a device called the "Interruptedness Metre" was used to collect ranked data from participants. Additional data collection methods included reflection and observation. For instance, in [38], the collection of qualitative data involved reflecting on parents' and children's views. In [40] free-form observation notes were used to capture body language and speech, while in [32] the graffiti wall method was employed to understand feelings and perceptions.

3.5. Main Empirical Results

To articulate the empirical study directions presented in the analyzed papers, we synthesized the principal findings extracted from these documents. Considering the frequency of occurrences in the studied works, we categorized the findings into seven distinct general themes as follows.

Nonverbal Communication and Robot Self-Efficacy. Nonverbal communication, encompassing facial expressions, gaze, and gestures, is a recurring theme across several academic papers. In [27] the powerful impact of facial expressions on the perceived anthropomorphism of a robot was emphasized. In addition, in [34] it was discussed how nonverbal cues such as speed of motion and proximity contribute to conveying interruption urgency and affect the perception of disruption by people. Nonverbal cues play a pivotal role in shaping the robot's self-efficacy, influencing how participants perceive it.

Robot Personality and Engagement. The exploration of robot personality as a key factor in engaging participants during interactions is a common thread in various papers. In [39] the importance of a robot's personality in influencing participants' perceptions and preferences was underscored. Combining gestures and speech, as suggested in [39], enhances the robot's engagement and naturalness, making it more appealing and effective. In [60] the potential for improving human-robot interaction by modulating robot behavior to express different moods was also indicated.

Trust and Trustworthiness. The topic of trust is addressed in several papers. In [40] the correlation between self-assessed trust and trust measured during a game was highlighted, emphasizing the positive impact of nonverbal communication on trust levels. Trust is a critical component in human-robot interactions, and nonverbal cues contribute to building and maintaining trust with robots. In [49] it was demonstrated how nonverbal cues, such as gaze, significantly improve cooperation between the robot and the human participant, underlining the role of trust in collaboration.

Sociability and Familiarity. Participants' perceptions of a robot's sociability and familiarity are examined in [61], where more sociable robots lead to a greater sense of familiarity. This highlights the significance of nonverbal cues in creating a sense of comfort during interactions. In [42] it was also discussed how different robot appearances affect nonverbal behaviors, such as the distance and delay of response, further illustrating the role of sociability in human-robot interaction.

Transparency and Active Learning. Transparency in robot behavior is a central theme in [46]. It is noted that active learning may not always be the best approach, as some participants prefer more control over the interaction. Transparency is vital for ensuring that the robot's behavior aligns with participants' expectations. In [43] the importance of balancing transparency and legibility of the robot's goals were highlighted, emphasizing that a clear understanding of the robot's intentions contributes to successful interactions.

Joint Tasks and Adaptation. Papers such as [47] and [25] delve into how participants respond to joint tasks with robots. Participants adapt their gaze patterns and engage in joint actions, but there can be challenges, such as finding robot movements less humanlike. This topic underscores the need for robots to adapt effectively in collaborative settings. In [49] and [55] it was emphasized the role of adaptability in human-robot interactions and how different expression levels and behaviors influence participants' emotional states and the analysis of the robot's role.

Cultural Differences and Shared Perception. The influence of different robot appearances and cultural differences is explored in various papers. For instance, in [63] it is discussed how shared perception in interactions improves cooperation and performance. Cultural nuances, such as bowing and handshakes, play a role in shaping the interaction's dynamics. In [41] and [58] insights are provided into how cultural differences affect the robot's learning progress and the robot's behavior in response to different cultural contexts.

These outlined themes underscore the role of nonverbal cues in shaping robot self-efficacy, trust, engagement, transparency, adaptation, and the impact of cultural differences in human-robot interactions.

3.6. Theoretical Approaches Proposed

In order to put the results into proper context, we need also to review the theoretical approaches proposed in literature, starting from the acceptance of service robots by their users. An important aspect of technology acceptance is self efficacy that refers to the person’s confidence and motivation in using the technology [9]. According to [64], general self-efficacy is “defined as people's beliefs about their capabilities to produce designated levels of performance that exercise influence over events that affect their lives.” People's self-efficacy in using technology, often referred to as computer self-efficacy, is associated with their attitudes toward technology, perceived usefulness, ease of use, and reduced anxiety [65]. The same principles apply to service robots, as robotics and self-efficacy are closely linked, impacting initiation, retention, and overcoming challenges. Robot self-efficacy is linked to general self-efficacy, tech interest, and willingness to use. Although self-efficacy cannot be used as a direct indicator for technology acceptance, due to reliance on others and technical issues, it is still usable for predicting the individual willingness to use the technology [65].

In the case of service robots as social entities, self-efficacy depends on the robot’s positive social influence on people [66], i.e., its ability to change people’s “motives and emotions, cognitions and beliefs, values and behavior that occur in an individual” [67]. In other words, self-efficacy depends on the robot's ability to be perceived as a social entity that has a potential for establishing and maintaining a social relationship [68]. As suggested by [66,69,70], a robot’s social influence and its user’s self-efficacy hinges on the effective utilization of nonverbal cues, a crucial element in human communication, i.e., the robot’s ability to have 2-way natural communication with humans [5]. In essence, the robot's capacity to emulate natural human interaction plays a pivotal role in determining self-efficacy, highlighting the interconnectedness of both direct verbal and indirect nonverbal communication [71]. The importance of the robot’s natural communication skills is stressed by [17] by proposing that robots with only verbal interaction skills are more likely ignored when asking help, compared to the robots that also use nonverbal cues in HRI. This suggests that effective collaboration between a human and a robot would be at least partly based on using both verbal and nonverbal cues in their interaction.

In verbal interaction, mostly language-based information is shared while nonverbal interaction carries various cues that help people to share interaction roles and to put relayed information into proper context [72]. In addition, nonverbal cues help interaction parties predict the next appropriate action. For example, the end of someone’s speech can be understood even before the speaker tells it [73]. Nonverbal cues involve physical appearance, gestures and posture, face and eye behavior, vocal behavior, space and environment [72]. However, as robotic bodies still do not have full capacities to relay immaculately human nonverbal cues, these cues could become inadequately presented or go missing in HRI. Inadequate nonverbal cues could determine robot use self-efficacy and acceptance in scenarios where people are interacting in social environments with robots or other persons mediated via robots, such as in higher education teaching and learning, in elderly care or in health care in general [9,10].

However, there is limited information regarding the real-world utilization of nonverbal behaviors in service robots. To address this gap, we delve into three case studies [17,65,69] related to robot self-efficacy and the significance of nonverbal communication skills in service robots. These case studies served as a foundation for shaping our research inquiries.

The authors of [65] investigate the relationship between self-efficacy and the use of robots. The paper highlights the importance of self-efficacy in the context of technology adoption, specifically robots. The authors suggest that robot self-efficacy in healthcare is correlated with general self-efficacy, interest in technology, and willingness to use robots. Their study emphasizes the role of successful experiences in enhancing self-efficacy, such as interacting with a social robot on household-related tasks. This kind of interaction can lead to more positive evaluations of the robot. The authors describe their experimental design, including measures for participant data, the Robot Self-Efficacy Scale, likability, and willingness to use the robot, in their case – the Pepper robot. The results indicate that brief interactions with the robot and its likeability play a significant role in changes in people’s robot self-efficacy. The study demonstrates the importance of self-efficacy in HRI.

The authors of [69] presented a study about the impact of a social robot's expressions, both verbal and non-verbal, on self-efficacy during cognitive tasks. The robot, Kebbi, showcased four different expressions: default voice and motion, default voice and designed motion, designed voice and default motion, and designed voice and designed motion. Seventeen university students with technical backgrounds engaged in the Wisconsin Card-Sorting Task, a test involving rule changes leading to mistakes. The study monitored participants' heart rates and brainwave data to gauge arousal and stress levels and recorded the experiment for comprehensive analysis. The study revealed that robot expressions influenced participants' perceptions and stress levels. Designed motion, combined with voice feedback, made the robot more likable and reduced frustration and stress. Individual personality traits also played a significant role in participant responses. The research underscores the importance of considering both verbal and non-verbal robot actions in educational support robots. It highlights the role of personal traits and individual differences in human-robot interactions, especially when enhancing self-efficacy in educational contexts.

The authors of [17] explored the impact of conversational interactions, both nonverbal and verbal, between robots and humans on relationship building. Their aim was to determine if such interactions could create a partnership between humans and service robots, potentially increasing helpful behavior from humans towards the robots. The authors suggest that as humans can form emotional connections with non-living entities, direct, personal, and spontaneous interactions are important in building relationships. In addition, for successful interaction, humans must perceive the robot as a genuine social actor. In their study, twenty-five students interacted with both an interactive and a non-interactive robot. Their results showed that the interactive robot was perceived more positively, resulting in higher social presence and a stronger human-robot relationship. The authors suggest that conversational interactions can enhance social presence and relationships in human-robot interactions, but understanding and predicting helping behavior remains a complex challenge. The authors propose that to foster effective cooperation, robots need to establish partnerships with humans, akin to how humans naturally collaborate using both verbal and non-verbal communication.

4. Discussion and Conclusions

In this study, we explored nonverbal communication in human-robot interaction (HRI), particularly focusing on service robots with social capabilities, often also referred to as social robots or robot assistants. The comprehension of how individuals perceive robots stands as a pivotal concern in gaining deeper insights into the realm of HRI [74]. As social robots become more prevalent within social landscapes, then they need to be perceived positively in order to become seamlessly integrated into our daily lives. The study of social perceptions regarding robots is an emerging field of research, one that is poised to gain increasing prominence in the years to come [75]. Our investigation has provided some insights into the state of research in this domain and unveiled findings that shed light on various aspects of this dynamic field.

Robots Used in the Examined Studies. The studies used a relatively diverse selection of robots in their experiments on nonverbal communication in HRI, reflecting the dynamics of the rapidly evolving landscape of HRI. Humanoid robots, particularly the NAO and iCub, have emerged as dominant platforms for studying nonverbal communication. Their humanoid appearance and programmable capabilities make them versatile choices for investigating the intricacies of HRI. The inclusion of bust and face robots, a robot dog (AIBO), a robot dinosaur (Pleo), and even non-humanoid robots (e.g., Roomba iRobot Create 2) provides further perspectives on nonverbal interactions between humans and robots. These diverse robot platforms enrich our understanding of nonverbal communication in various contexts, from social robots that mimic human features to product-oriented robots designed for specific functions. As evidenced by [21] and [76], engaging with robots can create a robust social context capable of influencing psychological needs, even in the absence of humanoid features.

Main Research Aims. Our analysis has revealed a wide range of research objectives within the selected academic papers. Understanding HRI remains a predominant focus, encompassing investigations into acceptance and utilization of robots, correlations between human verbal behavior and robot behavior, and the influence of nonverbal cues on individuals' perceptions of the robot's social agency. Emotional interaction has also been a central theme, with studies probing the impact of humanoid robots on human emotional responses and exploring personality in HRI. Learning and teaching in the context of HRI have also been addressed, examining the performance of machine learners when instructed by human teachers and the manner in which humans integrate robot actions into their plans. Cognitive and behavioral aspects have assumed a critical role in specific research, from devising cognitive architectures for robots to studying the role of gestures in joint tasks. The impact of robot behavior and communication on user affect and teamwork has been another notable focus. These diverse thematic clusters reflect the depth and breadth of scholarship within the field of HRI, encompassing a wide array of research aims that continue to advance our comprehension of this evolving domain.

Types of Nonverbal Communication Examined. The types of nonverbal communication cues examined in the selected papers are integral to understanding and enhancing the quality of interactions between humans and robots. We refrained from differentiating between studies that concentrated on robots interpreting nonverbal cues from people and those that centered on people interpreting the nonverbal cues of robots. This lack of distinction arises from the fact that these particular directions of communication are still in their rudimentary stage. Ideally, both communication parties should possess the capability to reciprocally indicate and comprehend intentions, even in scenarios where the robot lacks humanoid features, as demonstrated by [76]. Gesture and facial expression have been significant areas of exploration, with research focusing on hand movements, head poses, bodily movements, and facial expressions. These cues play a vital role in conveying information, emotions, and intentions in human-robot interactions. Affective states and emotions have also been extensively examined, with a focus on emotional expressions and mood recognition in robot-human interactions. Eye gaze, an essential means of nonverbal communication, has been scrutinized in terms of its impact on social interactions. Voice and speech cues have been explored, encompassing various aspects of speech, such as volume, speed, pitch, and style. Human-robot proximity and movement have been investigated to understand how physical and psychological distances influence communication and engagement. Cultural aspects and greetings have provided insights into the influence of cultural norms on nonverbal behavior in human-robot interactions. These categories of nonverbal communication cues play a pivotal role in shaping the effectiveness of service robots during human-robot interactions, enhancing engagement, trust, and the overall user experience.

Sample and Methods Used. The diversity in sample sizes and characteristics across the reviewed papers reflects the inclusivity and breadth of human participants in human-robot interaction research. Notably, a balanced gender distribution within the studies highlights the importance of diverse gender representation in these interactions. The wide range of participant ages underscores the inclusivity of different age groups in human-robot interaction research. The varied data collection methods employed in the selected studies encompassed a mix of quantitative and qualitative approaches, providing a comprehensive understanding of human-robot interactions.

Main Empirical Results. The main findings extracted from the analyzed papers can be categorized into several themes. Nonverbal communication has been identified as a key factor influencing robot self-efficacy, influencing how participants perceive and interact with robots. Robot personality and engagement are central themes, underscoring the significance of a robot's personality in shaping participants' perceptions and preferences. Trust and trustworthiness have been addressed, highlighting the positive impact of nonverbal communication on trust levels. Sociability and familiarity have also been explored, demonstrating the role of nonverbal cues in creating comfort during interactions. Transparency in robot behavior is emphasized, as it is vital for ensuring that the robot's behavior aligns with participants' expectations. Joint tasks and adaptation have been scrutinized, revealing the need for robots to adapt effectively in collaborative settings. Cultural differences and shared perception have provided insights into how different robot appearances and cultural norms influence the interaction's dynamics.

In the HRI, our examination of various studies leads us to a general observation: the predominant focus is on leveraging nonverbal communication to enhance the perceived seriousness of robots in their interactions with humans and make them more trustworthy. This approach aims to increase people’s robot efficacy in various settings, be it at home, in the office, or other contexts. Moreover, our research suggests a potential avenue for granting robots distinct personalities, allowing people to view them as genuine social entities that are able to make decisions about which they need to assist first [77]. This opens the door to meaningful interaction, relationships, and even negotiations with these robotic companions, with opportunities for assistance and support.

One intriguing aspect that could propel robots toward developing unique personalities while adhering to social norms is rather contentious. Often, robots offering services aren't owned by the individuals they assist. This can be attributed to factors such as their relatively high cost, making it inappropriate for individuals to engage with these robots beyond the scope of their designated services. Additionally, any interference with a robot's prescribed activities could lead to financial or health-related repercussions. Consequently, it becomes evident that maintaining an appropriate social distance [78] emerges as a fundamental social norm in HRI.

However, as we contemplate enabling robots to acquire distinct and evolving personalities, it raises several challenging questions. Philosophers find themselves grappling with issues like the concept of robot slavery or their potential for free will in the coming decades. On a more immediate note, regular individuals must ponder the issue of privacy. Personality-enabled robots naturally require data about the people they interact with to fine-tune their behavior (as discussed in [79]). Ideally, this data should remain localized, avoiding any cloud-based uploads, much like how we, as humans, collect and utilize data to inform our own actions.

The increasing number of articles, as indicated in Figure 2, suggests a growing interest in this field, likely influenced by the COVID-19 pandemic and rapid technological advancements. In recent years, there has been substantial development in both hardware and AI solutions for shaping robot personalities. It's crucial to explore not only how to make robots more acceptable to humans during initial encounters but also to delve into the psychology of developing personalities for robots that evolve in harmony with the context and the individuals they interact with in their work. This journey must ensure that the development of robot personalities does not lead to mental health-like impairments or issues.

The studies we have reviewed predominantly focus on the tangible hardware aspects of robot design. Nonverbal cues and behavior patterns are vital elements of robot personalities, which are essentially software-driven. While imparting nonverbal characteristics contributes to building trust with humans, creating an effective robot personality involves a collaborative process with people, their surroundings, and the specific context. This entails software memory and algorithms that consider other individuals, robots, and the environment in which the robot operates. For example, a household robot remembers those it encounters regularly and can differentiate them from newcomers, but this gives rise to questions regarding GDPR compliance.

Theoretical Approaches Proposed. Our results also underscore the significance of theoretical approaches in contextualizing the findings. Key concepts include technology acceptance and self-efficacy, emphasizing the essential role of self-efficacy in using service robots [9,65]. Self-efficacy, a critical aspect of technology acceptance, is linked to users' confidence, motivation, and their perception of a robot's social influence [64,66]. We draw attention to the interconnectedness of self-efficacy, robot social influence, and the effective utilization of nonverbal cues, pointing out their impact on users' willingness to engage with service robots [5,17,66]. Nonverbal communication, essential in human interaction, becomes a pivotal component for effective human-robot interaction, particularly in scenarios such as education, elderly care, and healthcare [9,72]. We highlighted three case studies on robot self-efficacy and the role of nonverbal communication in service robots [17,65,69]. These studies emphasize the influence of self-efficacy on technology adoption and the impact of robot expressions, both verbal and nonverbal, on user perceptions and stress levels. They highlight the significance of considering nonverbal cues in educational support robots and the importance of creating genuine, interactive relationships between humans and robots. Overall, we imply that self-efficacy, nonverbal communication, and the interplay between human perception and robot interaction play critical roles, shedding light on the complexities and importance of these dynamics in human-robot relationships.

In conclusion, our aim was to contribute to the growing body of knowledge in the field of HRI, with a specific focus on non-verbal communication in service robots endowed with social capabilities. The results underscore the potential of enhancing the quality of communication between humans and service robots. Nevertheless, there are aspects that have not been thoroughly explored thus far. To date, assessments have predominantly measured technology acceptance by humans or their perceived efficacy in using the robot within a simulated environment, specifically tailored for a particular scenario. In forthcoming studies, we emphasize the importance of conducting experiments in real public spaces, given that service robots find significant applications in such environments. Additionally, scenarios involving collaborative actions between humans and robots were limited, despite articles emphasizing the creation of an effective robot personality through a collaborative process with humans. The interaction between humans and robots in public spaces encompasses yet another crucial aspect that the analyzed studies failed to reflect. Little attention has been given to understanding how individuals feel in communication or collaborative situations with robots, particularly in public spaces where others are present. As a final point, despite studies highlighting the impact of self-efficacy on technology adoption, there has been no measurement of whether individuals' self-efficacy was higher when they achieved their goals in collaboration with the robot.

As technology continues to advance, further research in this field is paramount to ensuring that robots can effectively communicate with humans in diverse contexts, ultimately enhancing the overall HRI experience. This article provides a comprehensive overview of the current state of research and lays the foundation for future investigations in this dynamic field.