Submitted:
01 October 2023
Posted:
02 October 2023
You are already at the latest version
Abstract
Keywords:
1. Introduction
2. History of Body Language and Non-Verbal Communication
2.1. Antiquity and the Genesis of Non-Verbal Communication
2.2. Middle Ages and the Renaissance: The Codified Meaning of Gestures
2.3. The Evolution of Non-Verbal Communication in the Modern Era
2.4. The 20th Century: Interdisciplinary Advances and the Emergence of Microexpressions
2.5. The Digital Age and the Transformation of Non-Verbal Communication
- Evolution and Diverse Usage: Initially a set of basic smileys, emojis have rapidly evolved into a diverse lexicon, encapsulating an array of emotions, expressions, objects, activities, and even cultural symbols. From a simple smiling face to intricate depictions of laughter, love, frustration, or even tacos and flags, emojis enrich digital conversations, allowing for a more nuanced expression of feelings and ideas.
- Emojis as Emotional Enhancers: Research indicates that emojis not only convey emotions but also play a vital role in contextualizing the tone of a message [Wolf et al., 2018]. For instance, a simple "thank you" may convey different sentiments depending on whether it is accompanied by a smiley face or a thumbs-up emoji. Emojis, thus, act as emotional enhancers, adding layers of meaning and intent to textual communication.
- Cross-Cultural Universality: While languages differ globally, emojis have achieved a remarkable feat by offering a form of universal communication. A smiley face conveys happiness or positivity in various cultures, diminishing the potential for misinterpretation that often occurs with plain text. Emojis, in this sense, serve as a universal language, allowing people from diverse linguistic backgrounds to connect and comprehend sentiments effectively [Miller et al., 2016].
- Impact on Communication Dynamics: The proliferation of emojis has brought about a shift in how we communicate. Conversations are now infused with visual cues, making them more engaging and vibrant. Emojis add a personal touch, humanizing digital exchanges and bridging the emotional gap that exists in online communication. As a result, they have become an integral tool in enhancing user engagement and interaction on social media platforms.
- Emojis in Business and Marketing: Emojis are now recognized as powerful tools in the business realm. Companies leverage emojis in their marketing strategies to make their content more relatable and appealing to a broad audience. Emojis can convey brand identity, evoke emotional responses, and create memorable associations, making them a valuable asset in digital marketing campaigns.
- Avatars as Non-Verbal Expressions: Avatars, virtual representations of individuals in VR and AR environments, play a pivotal role in enabling non-verbal communication. Avatars mimic real-world movements and expressions, allowing users to convey emotions and intentions through virtual body language, gestures, and facial expressions. These nuanced non-verbal cues enhance the depth and authenticity of interactions within digital spaces [Bailenson et al., 2001].
- Impact on Interpersonal Dynamics: Research by Bailenson and Yee (2005) demonstrates how avatars and their non-verbal cues influence interpersonal dynamics in virtual environments. Even subtle alterations in avatar appearance and behavior significantly impact how individuals perceive and interact with each other. Avatars provide a medium for non-verbal expressions that, despite being digital, evoke genuine emotional responses, transforming the nature of virtual interactions [Bailenson et al., 2001].
- Integration of Multimodal Cues: VR and AR enable the integration of multiple non-verbal cues, such as gaze, gestures, proximity, and postures, into digital interactions. These cues enrich communication by mimicking face-to-face conversations, where a combination of verbal and non-verbal elements conveys a holistic message. For example, gaze direction in VR can be utilized to establish virtual eye contact, influencing the perception of attentiveness and engagement in a conversation [Bailenson et al., 2001].
- Applications in Various Sectors: The implications of non-verbal communication in VR and AR extend to diverse domains. In education, immersive environments facilitate interactive learning experiences where non-verbal cues aid in teacher-student and student-student interactions. In healthcare, AR overlays non-verbal cues, aiding medical professionals in conveying information during surgeries or consultations. These applications underscore the transformative potential of integrating non-verbal communication into virtual spaces.
- Future Prospects and Challenges: The future of non-verbal communication in VR and AR holds exciting prospects. As technology advances, more sophisticated avatars and immersive experiences will emerge, further blurring the line between physical and digital interactions. However, challenges such as the "uncanny valley" effect, where highly realistic avatars may evoke discomfort due to their almost-real yet not-quite-real appearance, need to be addressed to enhance user acceptance and experience.
- Cultural Emoji Interpretations: Emojis, being a universal form of non-verbal communication, are often used to express emotions, actions, and ideas. However, cultural backgrounds can influence the interpretation of emojis. A study by Dehghani et al. (2017) explored how individuals from different cultures perceive and use emojis. The research revealed that while some emojis have universal meanings, others can be interpreted differently based on cultural contexts. For instance, a thumbs-up emoji might be positive in some cultures but offensive in others, highlighting the importance of cultural understanding in digital communication [Dehghani et al., 2017].
- Contextualizing Non-Verbal Cues: Culture significantly influences how non-verbal cues are contextualized within digital conversations. Certain gestures or expressions may hold specific meanings or connotations in one culture but could be entirely different in another. Understanding these cultural nuances is critical in avoiding miscommunication or misunderstanding, ensuring effective and respectful communication across diverse cultural groups.
- Cultural Emoticon Variations: Emoticons, a precursor to emojis, are often used in digital communication to convey emotions or facial expressions using keyboard characters. Different cultures may employ distinct styles of emoticons or use them in varying frequencies. For example, the style and usage of emoticons in Asian cultures, such as Japan's extensive use of kaomoji, differ from those in Western cultures. Recognizing and adapting to these cultural differences is essential for effective cross-cultural digital communication.
- Non-Verbal Norms in Social Media: Social media platforms have become hubs for global interactions. Different cultures may have varying norms regarding the use of non-verbal cues on these platforms. Understanding these norms helps individuals navigate these spaces effectively, engage appropriately, and convey messages in a manner congruent with the expectations of specific cultural audiences.
- Localization of Non-Verbal Features: In the digital age, platforms often localize non-verbal features to cater to specific regions or cultures. For instance, certain social media platforms may introduce culturally relevant stickers or emojis that resonate with the users of a particular country or region. This localization acknowledges the importance of cultural context in digital non-verbal communication.
4. The Neuroscience of Non-Verbal Communication
4.1. The Neurobiology of Non-Verbal Communication: Decoding the Language of the Brain
- Sensory Input: Sensory information related to non-verbal cues, such as facial expressions, vocal tones, and body language, is conveyed to the amygdala via these sensory pathways. The amygdala integrates and processes this information to extract emotional content.
- Connections with Other Brain Regions: The amygdala has extensive connections with regions involved in memory, decision-making, and emotional regulation. This connectivity allows it to influence cognitive processes based on the emotional significance of the stimuli.
- Rapid Emotional Evaluation: Through its connections with sensory pathways, the amygdala can rapidly evaluate the emotional content of non-verbal cues. This rapid assessment is essential for immediate reactions and adaptive behaviors, particularly in potentially threatening or emotionally charged situations.
- Emotional Memory Formation: The amygdala is also deeply involved in the formation and consolidation of emotional memories. Emotional experiences linked to non-verbal cues are stored in the amygdala, influencing our future responses to similar stimuli.
- Automatic Emotional Responses: The amygdala generates automatic emotional responses, influencing facial expressions, gestures, and other non-verbal cues. For instance, a fear-inducing non-verbal cue may trigger a reflexive expression of fear, indicating the amygdala's role in shaping our immediate non-verbal reactions.
- Interpretation of Emotions: The amygdala helps interpret the emotional states of others based on their non-verbal expressions. This interpretation forms the basis of our empathy and understanding of the feelings and intentions conveyed through non-verbal communication.
- Cultural and Individual Variations: Research suggests that cultural and individual differences influence how the amygdala processes and responds to non-verbal cues. Understanding these variations is crucial in comprehending the intricate interplay between the amygdala and socio-cultural factors in non-verbal communication.
- Imitation and Empathy: The presence of mirror neurons suggests that the brain has a specialized mechanism for imitation, allowing individuals to replicate observed actions. This mechanism is fundamental for learning and socialization. Moreover, mirror neurons are thought to be a neural basis for empathy, enabling us to understand and share the feelings and experiences of others.
- Understanding Intentions: Mirror neurons not only fire when an action is performed but also when it is observed, providing insight into the intentions behind the action. This understanding of intentions is crucial for effective non-verbal communication, allowing us to interpret and respond to the unspoken motives of others.
- Motor Resonance: This simulation process, often referred to as "motor resonance," is at the core of mirror neuron functionality. It enables us to understand and imitate actions, facilitating our ability to learn by observing others.
- Empathy and Emotional Contagion: Mirror neurons are thought to be central to the human capacity for empathy and emotional contagion. When we see someone experiencing an emotion, the mirror neuron system fires, simulating that emotion within ourselves. This mechanism allows us to emotionally connect with others.
- Mimicry and Synchronization: Mirror neurons facilitate mimicry, where individuals subconsciously imitate the non-verbal behaviors of others. This mimicry fosters social bonds and rapport. Moreover, they aid in synchronization, enabling individuals in a group to coordinate their non-verbal cues, leading to a sense of unity and cohesion.
- Understanding Emotional Expressions: Mirror neurons help us understand and empathize with the emotions expressed by others through facial expressions and gestures. When we observe a particular emotion, our mirror neuron system simulates that emotion within ourselves, providing a basis for understanding the emotional states of others.
- Social Learning: By allowing us to imitate observed actions, mirror neurons facilitate social learning. This is particularly important in acquiring social norms, cultural practices, and even language through observation and imitation.
- Interoception and Emotional Awareness: The insula receives and integrates signals from the body's internal organs, enabling us to perceive and interpret our physiological states. This awareness of our own bodily sensations is foundational for emotional experiences and, consequently, how we express these emotions non-verbally.
- Connectivity with Emotional Centers: The insula is densely connected with various emotional processing centers in the brain, including the amygdala and the prefrontal cortex. These connections enable the insula to influence emotional regulation, empathy, and the interpretation of emotional states in ourselves and others.
- Mapping Visceral Sensations to Emotions: The insula maps bodily sensations to specific emotional states, allowing us to experience and label emotions. This mapping is crucial for our ability to express emotions non-verbally, including facial expressions and other gestures.
- Empathy and Emotional Contagion: The insula is engaged during empathetic responses, where we perceive and share the emotional experiences of others. This process is fundamental for social bonding and understanding non-verbal cues related to others' emotions.
- Facial Expressions and Emotional Mimicry: Research indicates that the insula is involved in mirroring facial expressions, allowing us to imitate the emotions we observe in others. This process of emotional mimicry is a fundamental aspect of non-verbal communication.
- Gut Feelings and Social Decision-Making: The insula, linked to interoception, provides us with "gut feelings" or intuitive responses. These visceral sensations often guide our social decisions and behaviors, influencing our non-verbal communication in various social contexts.
- Integration of Information: The prefrontal cortex integrates information from sensory, emotional, and memory-related brain areas. This integration is crucial for processing non-verbal cues, understanding emotional states, and generating appropriate responses in social interactions.
- Regulation and Control: The prefrontal cortex plays a pivotal role in regulating emotions and controlling impulsive behaviors. It allows us to modulate our non-verbal expressions to suit social norms and specific social contexts.
- Emotional Regulation: The vmPFC is involved in emotional regulation by assessing emotional significance and assigning appropriate responses. This regulation extends to non-verbal emotional expressions, aiding in the modulation of our facial expressions and gestures.
- Theory of Mind: The prefrontal cortex, especially the medial prefrontal regions, is essential for theory of mind—the ability to understand and infer the mental states, thoughts, and intentions of others. Theory of mind is fundamental for perceiving non-verbal cues accurately and responding empathetically.
- Social Decision-Making: The prefrontal cortex, including the OFC and ACC, guides social decision-making by weighing social cues and integrating them with internal states and contextual information. These decisions influence our non-verbal behaviors in social situations.
- Facilitating Empathy: Through its involvement in theory of mind and emotional regulation, the prefrontal cortex facilitates empathy, allowing us to understand and resonate with others' emotions. This understanding is conveyed through non-verbal cues, such as facial expressions mirroring the emotions we perceive.
4.1. Social Neuroscience and Non-Verbal Communication: Decoding the Interplay
5. Neuroscience and non-verbal communication – Multidisciplinary Applications
5.1. Mental Health and Therapy
5.2. Education and Learning Enhancement
5.3. Human-Computer Interaction (HCI)
5.4. Marketing and Consumer Behavior
5.5. Law Enforcement and Security
6. Conclusion
References
- Adolphs, R. Recognizing emotion from facial expressions: Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews 2002, 1(1), 21–62. [Google Scholar] [CrossRef] [PubMed]
- Bailenson, J.N.; Yee, N. Digital chameleons: Automatic assimilation of nonverbal gestures in immersive virtual environments. Psychological Science 2005, 16(10), 814–819. [Google Scholar] [CrossRef] [PubMed]
- Bailenson, J.N.; Beall, A.C.; Loomis, J.; Blascovich, J.; Turk, M. Transformed social interaction: Decoupling representation from behavior and form in collaborative virtual environments. Presence: Teleoperators and Virtual Environments 2001, 10(4), 412–427. [Google Scholar] [CrossRef]
- Baxandall, M. Patterns of Intention: On the Historical Explanation of Pictures. Yale University Press 1985.
- Brunner, P.; Joshi, S.; Briskin, S.; Wolpaw, J.R.; Bischof, H.; Schalk, G. Does the "P300" speller depend on eye gaze? Journal of Neural Engineering 2011, 8(5), 056013. [Google Scholar] [CrossRef] [PubMed]
- Calvo-Merino, B.; Glaser, D.E.; Grèzes, J.; Passingham, R.E.; Haggard, P. Action Observation and Acquired Motor Skills: An fMRI Study with Expert Dancers. Cerebral Cortex 2005, 15(8), 1243–1249. [Google Scholar] [CrossRef]
- Carney, D.R.; Cuddy, A.J.C.; Yap, A.J. Power Posing: Brief Nonverbal Displays Affect Neuroendocrine Levels and Risk Tolerance. Psychological Science 2010, 21(10), 1363–1368. [Google Scholar] [CrossRef]
- Castelli, L.; Carraro, L.; Ferrante, D. Emotive branding and consumer behavior: Testing the compensatory and spillover effects between visual and auditory domains. Psychology & Marketing 2017, 34(10), 927–943. [Google Scholar]
- Dapretto, M.; Davies, M.S.; Pfeifer, J.H.; Scott, A.A.; Sigman, M.; Bookheimer, S.Y.; Iacoboni, M. Understanding emotions in others: Mirror neuron dysfunction in children with autism spectrum disorders. Nature Neuroscience 2006, 9(1), 28–30. [Google Scholar] [CrossRef]
- Darwin, C. The Expression of the Emotions in Man and Animals. John Murray 1872.
- Decety, J.; Lamm, C. The role of the right temporoparietal junction in social interaction: How low-level computational processes contribute to meta-cognition. The Neuroscientist 2007, 13(6), 580–593. [Google Scholar] [CrossRef] [PubMed]
- Dehghani, M.; Kashima, Y.; Tehranian, M.; Jamarani, E.; Verkuyten, M.; Garan, A. Emojis: Insights, affordances, and possibilities for psychological science. Trends in Cognitive Sciences 2017, 21(2), 259–268. [Google Scholar]
- Derks, D.; Fischer, A.H.; Bos, A.E. The role of emotion in computer-mediated communication: A review. Computers in Human Behavior 2008, 24(3), 766–785. [Google Scholar] [CrossRef]
- Duan, X.; Li, W.; He, C.; Tan, Y.; Zhang, X. Deception detection in a simulated fMRI concealed information test. Frontiers in Human Neuroscience 2019, 13, 1–8. [Google Scholar]
- Duchenne, G.B. Mécanisme de la physionomie humaine ou Analyse électro-physiologique de l'expression des passions. Baillière et Fils 1862.
- Ekman, P. Universals and cultural differences in facial expressions of emotion. In J. Cole (Ed.), Nebraska Symposium on Motivation, 1971 1972, pp. 207-283. University of Nebraska Press.
- Ekman, P. An argument for basic emotions. Cognition and Emotion 1992, 6(3-4), 169-200.
- Ekman, P.; Friesen, W.V. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 1971, 17(2), 124–129. [Google Scholar] [CrossRef]
- Ekman, P.; Friesen, W.V. Facial Action Coding System. Consulting Psychologists Press 1978.
- Ekman, P.; O'Sullivan, M. Who can catch a liar? American Psychologist 1991, 46(9), 913–920. [Google Scholar] [CrossRef]
- Ekman, P.; Sorenson, E.R.; Friesen, W.V. Pan-cultural elements in facial displays of emotion. Science 1969, 164(3875), 86–88. [Google Scholar] [CrossRef]
- Ferguson, G. Signs and Symbols in Christian Art. Oxford University Press 1996.
- Frith, U.; Frith, C.D. Development and neurophysiology of mentalizing. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 2003, 358(1431), 459–473. [Google Scholar] [CrossRef]
- Galton, F. Inquiries into Human Faculty and Its Development. Macmillan and Company 1883.
- Gauthier, I.; Skudlarski, P.; Gore, J.C.; Anderson, A.W. Expertise for cars and birds recruits brain areas involved in face recognition. Nature Neuroscience 2000, 3(2), 191–197. [Google Scholar] [CrossRef] [PubMed]
- Hall, E.T. The Hidden Dimension. Doubleday 1966.
- Hillier, A.; Fish, T.; Cloppert, P.; Beversdorf, D.Q. Outcomes of a social and vocational skills support group for adolescents and young adults on the autism spectrum. Focus on Autism and Other Developmental Disabilities 2011, 26(1), 27–38. [Google Scholar] [CrossRef]
- Hillier, A.; Fish, T.; Siegel, J.H.; Beversdorf, D.Q.; Minshew, N.J. A pilot study examining the relationship between meanings of gestures and social–communicative competence in children with high functioning autism spectrum disorder. Journal of Autism and Developmental Disorders 2011, 41(6), 727–737. [Google Scholar]
- Jabbi, M.; Swart, M.; Keysers, C. Empathy for positive and negative emotions in the gustatory cortex. NeuroImage 2007, 34(4), 1744–1753. [Google Scholar] [CrossRef] [PubMed]
- James, W. What Is an Emotion? Mind 1884, 9(34), 188–205. [Google Scholar] [CrossRef]
- Jones, T.S.; Yarbrough, A.E. A naturalistic study of the meanings of touch. Communication Monographs 1985, 52(1), 19–56. [Google Scholar] [CrossRef]
- Kanwisher, N.; McDermott, J.; Chun, M.M. The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 1997, 17(11), 4302–4311. [Google Scholar] [CrossRef]
- Kedar, Y.; Zhang, J. Nonverbal Communication in Ancient China: The Evidence from Oracle Bone Inscriptions. China Media Research 2015, 11(1), 85–92. [Google Scholar]
- Kendon, A. Gesture: Visible Action as Utterance. Cambridge University Press 2004.
- Klin, A.; Jones, W.; Schultz, R.; Volkmar, F.; Cohen, D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry 2002, 59(9), 809–816. [Google Scholar] [CrossRef] [PubMed]
- Koenigsberg, H.W.; Fan, J.; Ochsner, K.N.; et al. Neural correlates of the use of psychological distancing to regulate responses to negative social cues: A study of patients with borderline personality disorder. Biological Psychiatry 2010, 66(9), 854–863. [Google Scholar] [CrossRef] [PubMed]
- Le Bon, G. The Crowd: A Study of the Popular Mind. Macmillan 1895.
- Lewis-Williams, D. The Mind in the Cave: Consciousness and the Origins of Art. Thames & Hudson 2002.
- Lombroso, C. LʹUomo Delinquente. Fratelli Bocca Editori 1876.
- Matsumoto, D.; Hwang, H.C. Evidence for training the ability to read microexpressions of emotion. Motivation and Emotion 2011, 35(2), 181–191. [Google Scholar] [CrossRef]
- Mehrabian, A.; Ferris, S.R. Inference of attitudes from nonverbal communication in two channels. Journal of Consulting Psychology 1967, 31(3), 248–252. [Google Scholar] [CrossRef]
- Meyer, A.; Rose, D.H. The Future is in the Margins: The role of technology and disability in educational reform. Educational Policy 2005, 19(2), 207–225. [Google Scholar]
- Molnar-Szakacs, I.; Wu, A.D.; Robles, F.J.; Iacoboni, M. Do you see what I mean? Corticospinal excitability during observation of culture-specific gestures. PLoS ONE 2006, 1(1), e88. [Google Scholar]
- Pan, X.; Gillian, N.; Bailenson, J.N. The avatar mind effect: The presence and influence of non-physical, digital minds on human thought, emotion, and behavior. PLoS ONE 2020, 15(10), e0240325. [Google Scholar]
- Pan, X.; Gillies, M.; Barker, C.; Clark, D. Uncanny valley or a more comfortable interface? The role of experience in perception of an intelligent virtual agent. Frontiers in Robotics and AI 2020, 7, 62. [Google Scholar]
- Parkin, D. Sacred Void: Spatial Images of Work and Ritual among the Giriama of Kenya. Cambridge University Press 1992.
- Pastoureau, M. Heraldry: Its Origins and Meaning. Thames & Hudson 1997.
- Saggese, D.; Cordasco, G.; Maldonado, M.N.; Bourbakis, N.; Vinciarelli, A.; Esposito, A. Power Poses Affect Risk Tolerance and Skin Conductance Levels. 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI), Volos, Greece, 2018; 1033; 1039. [Google Scholar] [CrossRef]
- Schneider, E.F.; Slaughter, J.E. Social Cognition and Nonverbal Communication: How Do Children Process Socio-Emotional Information in Others' Face and Body Gestures? Frontiers in Psychology 2013, 4, 675. [Google Scholar] [CrossRef]
- Stein, M.B.; Goldin, P.R.; Sareen, J.; Zorrilla, L.T.; Brown, G.G. Increased amygdala activation to angry and contemptuous faces in generalized social phobia. Archives of General Psychiatry 2002, 59(11), 1027–1034. [Google Scholar] [CrossRef] [PubMed]
- Theodoridou, A.; Rowe, A.C.; Penton-Voak, I.S.; Rogers, P.J. Oxytocin and social perception: Oxytocin increases perceived facial trustworthiness and attractiveness. Hormones and Behavior 2009, 56(1), 128–132. [Google Scholar] [CrossRef]
- Tuer, A.W. The History of the Horn-Book. Leadenhall Press 1890.
- Tusche, A.; Bode, S.; Haynes, J.D. Neural responses to unattended products predict later consumer choices. Journal of Neuroscience 2010, 30(23), 8024–8031. [Google Scholar] [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).