Prerpints.org logo
Preprint
Review

This version is not peer-reviewed.

Cognitive Functioning and Symptomatic Correlations during Audiovisual Multisensory Integration in Schizophrenia: A Systematic Review and Meta-Analysis

Submitted:

15 August 2024

Posted:

20 August 2024

You are already at the latest version

Abstract
Background: Multisensory integration (MSI) avoids cognitive overload, and the traditional all-or-nothing perspective for dysfunctional MSI in schizophrenia has been increasingly challenged. This study aimed to systematically review and meta-analyze extant evidence on audiovisual MSI in schizophrenia. Methods: We searched Web of Science, PubMed, PsycINFO and PsycARTICLES databases from inception until April 16th, 2024. Studies using appropriate audiovisual integration tasks to compare either behavioral or neural responses between schizophrenia individuals and healthy controls were included. Primary outcome measures included between-group differences (Hedge’s g) and symptomatic correlations (Fisher’s z) in three components of audiovisual integration tasks, including audiovisual processing, adaptation and integration. The multilevel random-effects models were used to address statistical dependency. Results: In total, 209 between-group and 116 correlational effect sizes were synthesized from 46 studies, representing 1,108 non-overlapped schizophrenia participants and 1,084 healthy controls. The overall dysfunctional audiovisual MSI in schizophrenia (g = -0.50, [95% CI, -0.64 to -0.37]) was primarily ascribed to the impaired audiovisual processing (g = -0.53, [95% CI, -0.71 to -0.35]) and adaptation (g = -0.71, [95% CI, -0.99 to -0.42]), rather than integration itself (g = -0.23, [95% CI, -0.56 to 0.11]). Pooled correlational effect size was extremely small (z = 0.16, [95% CI, 0.03 to 0.30]). Conclusions: Individuals with schizophrenia show dysfunctional multisensory processing and adaptation, but preserve a less impaired multisensory integration ability. Our findings provide implications for clinical interventions, as the relatively preserved audiovisual integration ability may serve as a possible avenue for cognitive training programs.
Keywords: 
audiovisual integration
;  
schizophrenia
;  
systematic review
;  
meta-analysis
;  
symptomatic correlation
;  
multisensory
Subject: 
Medicine and Pharmacology  -   Psychiatry and Mental Health

Introduction

Facing the overwhelming information flood in real-life situations, we normally have to appropriately unify the incoming inputs from segregated sensory events. Multisensory integration (MSI) thus prevents us from cognitive overload and scaffolds our cognitive responses to life events. Schizophrenia is characterized by dysfunctional MSI capability, such as the widened temporal binding window (TBW) [1], mitigated temporal acuity [2] and reduced illusory perception. [3] These changes in multisensory processing usually influence cognitive and social functioning of people with schizophrenia in real-life scenarios, [4] such as attention, [5,6] memory, [7] speech-related communication, [8,9] and emotion interpretation. [10,11,12] In contrast, schizophrenia individuals have also been found to preserve the ability to benefit from multisensory presentations in certain MSI tasks. [13,14] Considering that MSI is a complex concept encompassing several components beyond the integration of information from different sensory modalities, [15] is it possible that individuals with schizophrenia show only partial abnormality in certain subordinate procedures during MSI processing?
Of various sensory modalities (e.g., visual, auditory, tactile and olfactory), the present study focused on audiovisual integration, one particular type of MSI, for a few reasons. Audiovisual integration is crucial for face-to-face communications and the exchange of emotional messages, and difficulties in audiovisual integration may lead to socio-communicative problems, especially for people with mental disorders, thus negatively influencing their quality of life. [16] Specifically for schizophrenia patients, perceptual dysfunction in visual and auditory modalities has been regarded as a hallmark symptom, [17,18] and the integration of these two modalities has been substantially investigated in this population, [1,19,20,21] which seemed to be correlated with schizophrenic symptoms like hallucination. [22] Therefore, since the widely attended audiovisual MSI could deepen our understanding of schizophrenia and potentially improve patients’ quality of life, the present study has primarily synthesized MSI findings from the audiovisual domain.
Drawing on existing review articles concerning the MSI ability in populations with neuropsychiatric disorders, this ability could be investigated from three essential components. [23,24,25] First, the multisensory gain [26] denotes the benefits or performance enhancement obtained from multisensory stimuli compared to unisensory stimuli, which is also known as the redundant signals effect [27] or congruent facilitation effect. [28] A few paradigms, including simultaneity judgement [2] and temporal order judgement [29] have been commonly used to explore the multisensory gain. A second MSI component is simply the direct processing of multisensory inputs. For example, in the McGurk [30] and sound-induced flash illusion (SIFI) [3] paradigms, participants need to fuse information concurrently from different sensory modalities into one percept, which reflects the outcome of MSI processing. Third, the processing discrepancy between different multisensory conditions, as a reflection of the MSI adaptation ability, [31] has also been attended as part of the MSI capability. For example, in a typical emotion identification task, the identification of emotions in one modality can be influenced differently by different emotions in the other modality. [11] Recent meta-analyses suggested that neural correlates of audiovisual integration in the human brain are highly context-dependent, with both commonly and uniquely engaged brain regions as a function of analytical contrast, stimulus complexity and attention. [15,32] In support of the rationale to classify audiovisual MSI into different components, unique activation patterns of brain regions were identified if respectively analyzing the bimodal-unimodal contrasts (e.g., bilateral temporal cortices), audiovisual processing (e.g., right insula, right middle temporal gyrus) and bimodal-bimodal contrasts (e.g., right inferior frontal gyrus, right superior temporal gyrus, left occipital gyrus). [15] Motivated by such findings, it shall be necessary to analyze the behavioral and neurological changes of schizophrenia patients across different audiovisual MSI components, as unique patterns may be identified when distinct neural correlates are involved.
Considering distinct neural mechanisms underlying these subordinate components of audiovisual MSI, extant results on different MSI components would also be respectively aggregated in the current study (for more detailed illustrations, see eIntroduction in Supplementary File S1; Supplementary Tables S1 and S2). By definition, the multisensory-unisensory (MU) contrast denotes the performance enhancement or processing benefit of audiovisual information, compared to unisensory stimuli. The audiovisual processing (AVP) component refers to the direct processing of audiovisual stimuli, without contrasting other unisensory or multisensory stimuli. The multisensory-multisensory (MM) contrast means the differences in processing different types of multisensory stimuli involved in particular MSI tasks, which reflects the MSI adaptation ability between various multisensory conditions.
Another interesting question is, to which degree the clinical symptoms of schizophrenia individuals could predict their audiovisual MSI performance? Most prior studies reported non-significant correlations between MSI performance and schizophrenia symptom severity, [5,29,33,34,35,36,37] but some still identified significant clinical relevance. [21,38,39,40,41] These inconsistencies encourage us to reconsider the clinical implications of such correlations. For example, are such correlations indicative of the state or trait characteristics of schizophrenia? [42] Can neurological and behavioral changes during MSI tasks serve as potential endophenotypes of schizophrenia? [8,19]
To our knowledge, none to date has derived meta-analytical estimates for the subordinate components of audiovisual MSI processing or the clinical correlations between MSI and symptomatic measures in schizophrenia. Most extant review articles only qualitatively focused on specific cognitive domains [25,43,44] or neural connectivity [24] during MSI processing in schizophrenia, and very few have conducted meta-analyses to examine certain measurement metrics. [45,46] Therefore, this systematic review and meta-analysis had two primary objectives. First, we aimed to quantitatively synthesize the behavioral and neurological statistics about the differences between schizophrenia patients and healthy controls in audiovisual MSI tasks, and also examine the effects of various moderators through subgroup analyses and meta-regression. Second, we also aggregated correlational coefficients between symptom severity and audiovisual MSI performance in the schizophrenia population.

Methods

Search Strategy

The present systematic review and meta-analysis follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (PRISMA checklists: Supplementary Tables S3 and S4). [47] Four term blocks were designed and used in literature search, which respectively attended to the participant, modality, task and audiovisual integration aspects. We used designed retrieval queries (Supplementary Table S5) for literature search in the Web of Science, PubMed, PsycINFO and PsycARTICLES databases from inception to June 12th, 2023 initially, and updated the search on April 16th, 2024. With the combination of these retrieval queries, studies that investigated MSI capability in schizophrenia patients, especially across the auditory and visual modalities, shall be maximally and exclusively retrieved. To capture potentially unattended publications, forward and backward citation searches were conducted for six highly relevant review articles [4,24,25,43,44,48] and two meta-analytical studies. [45,46]

Inclusion and Exclusion Criteria

In accordance to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [42] PICO criteria, [49] the inclusion criteria of this systematic review and meta-analysis were formulated in four aspects, including population, intervention, control and outcome. First, studies had to include a schizophrenia clinical group and a comparable healthy control (HC) group (i.e., no history of any mental disorders and not at genetic risk for schizophrenia). Second, studies should use appropriate audiovisual integration tasks that could reflect the MSI capability, such as the McGurk, SIFI, simultaneity judgement (SJ) and temporal order judgement (TOJ) paradigms. Here, studies that observed the integration of other modalities (e.g., tactile and olfactory senses), [50] measured parent assessments or subjective feelings (e.g., Audio-Visual Abnormalities Questionnaire), [51] or required the participants to only respond to catch trials irrelevant to the MSI analyses [52,53,54] were excluded. Third, studies had to include at least one behavioral or neurological measure for the MSI tasks, and report necessary statistics (e.g., t, F, mean, SD, N, SE, r) for calculating the between-group or correlational effect sizes. Last, studies were published in a peer-reviewed journal in English, and non-empirical publications (e.g., systematic reviews, editorials, commentaries) were excluded. We were also encouraged to conduct a supplementary fMRI meta-analysis. For this additional meta-analysis, we further removed studies that did not report the raw peak coordinates or inferential statistics (e.g., t or z) of clusters in standard stereotaxic coordinate space (e.g., Talairach or Montreal Neurological Institute, MNI).
In a multi-step screening procedure, two researchers (E.T. and N.C.) independently performed the literature search and screening. With all collected literature after duplication removal, they first inspected titles and abstracts according to the inclusion criteria, and removed the obviously inappropriate articles. Before the independent screening, both researchers agreed on the inclusion criteria and independently screened the titles and abstracts of 20 articles. At this step, disagreements were addressed and consistent screening criteria were settled by discussion. Then, the two researchers independently screened all non-duplicated articles (k = 4,762) and reached agreements on the articles that were not excluded by the title and abstract screening step through discussion. Afterwards, both researchers reviewed the full text of saved articles (k = 130), and independently selected the eligible studies for our systematic review and meta-analysis. Disagreements and uncertainties were resolved by discussion with experienced professors (H.D. and H.C.).

Data Extraction

Essential statistics for the calculation of between-group and correlational effect sizes were extracted, and demographic and methodological factors were coded for moderator analysis (see eMethods in supplementary File S1). Based on the descriptions and statistical meaning of these statistics in original articles, they were further labeled by three components of MSI (e.g., AVP, MM, MU). Two researchers (E.T. and N.C.) independently extracted raw statistics and coded moderators, and they reached an agreement on the extracted statistics by discussion. All extracted data were double screened for accuracy by two other researchers (J.L. and H.D.), and disagreements were resolved by discussion.

Quality Assessment

To assess the quality and risk of bias of individual studies, we modified the Downs and Black’s Checklist for Measuring Quality [55] and Newcastle-Ottawa Scale for Assessing the Quality of Nonrandomized Studies in Meta-Analysis. [56] The complete checklist for quality assessment used in this study is provided (see eMethods in supplementary File S1). Each included study was evaluated independently by two researchers (E.T. and N.C.). With a substantial inter-rater agreement (cohen’s kappa = 0.76), the remaining disagreements were addressed by discussion.

Statistical Analysis

Effect Size Aggregation

Two primary outcomes were calculated and reported. First, we calculated the pooled effect size (Hedge’s g) that indicated between-group differences in MSI capability. A few subgroup analyses were conducted to decrease the between-study and within-study heterogeneity. We calculated the aggregated effect sizes of behavioral and neurological measures both together and separately, and further examined the pooled effect sizes of three MSI components, respectively. Second, we calculated the correlational effect sizes (Fisher’s z) that reflected the correlational relationships between MSI capability and schizophrenic symptoms.
Random-effects meta-analyses were performed using the Metafor [57] and clubSandwich [58] packages in R, version 4.3.1 (R Core Team). The Hedge’s g was reported for the between-group analyses, which accounted for the bias caused by small sample sizes. According to the commonly acknowledged benchmarks, a g value of 0.2, 0.5 and 0.8 indicates a small, medium and large effect size, respectively. For all analyses, we combined the effect sizes from overlapping samples or the same sample into larger clusters by fitting the correlated and hierarchical effects (CHE) model59 with robust variance estimation, [60,61] which addressed the unit-of-analysis error (i.e., multivariate and multilevel dependency). Detailed descriptions for our fitted models and principles for sample cluster specification are provided in eMethods in supplementary File S1. Between-study heterogeneity was primarily evaluated using the τ2, I2, and prediction intervals.
The supplem [entary fMRI meta-analysis was performed using the Effect-size Seed-based d Mapping software (ES-SDM) [62] which accounted for both increases and decreases of the outcomes with random-effects modeling. All raw z-scores were converted to t-values, and all coordinates were transformed to the MNI space. The default kernel size and thresholds of ES-SDM were maintained (full width at half maximum, FWHM = 20 mm, cluster extent = 10 voxels, voxel P = 0.005). [63]
Publication bias was diagnosed by inspections of contour-enhanced funnel plots, [64] Egger’s test [65] and the trim-and-fill analyses. [66] For the sensitivity evaluation, we conducted leave-one-out analyses by excluding one single unique effect size at a time to assess whether our findings were strongly influenced by any single cases or outliers. In addition, the fail-safe N [67] was also performed to evaluate the robustness of our results.

Moderator Analyses

Meta-regression models with random effects were constructed to demonstrate the association of participant-related and study-related moderators with the pooled group difference and correlation effects, respectively. Participant-related moderators encompassed demographic (e.g., age, sex distribution, education level and IQ) and clinical information, including illness duration, defined daily dose (DDD), and symptom measures (e.g., PANSS, SANS, SAPS). Study-related moderators included sample size, experimental tasks (e.g., McGurk, SIFI, SJ, emotion identification), audiovisual metrics (e.g., hit rates, illusion rates, TBW and neural responses), stimuli types (e.g., visual face/flash paired with auditory beep/syllable/sentence), and stimuli complexity (i.e., social and linguistic complexity).

Results

Literature Search

Figure 1 presents the PRISMA flowchart, reflecting the identification, screening and selection of our literature search. In total, 4,762 non-repetitive articles were collected from the original electronic search and cross-referencing. After screening the titles and abstracts based on our inclusion and exclusion criteria, 130 articles were identified as eligible for full-text assessment, among which 57 articles were qualified for the qualitative review. Of the 73 excluded articles here, 53 studies did not adopt eligible audiovisual integration tasks to measure the MSI capability, 14 studies did not involve a schizophrenia patient group, five studies were not empirical experiments, and one study did not explicitly refer to any standardized diagnosis criteria. Afterwards, due to the lack of sufficient data for effect size calculation, 11 articles were further excluded from our quantitative analyses, thus leaving 46 eligible articles in our final sample for the meta-analytical review (the complete reference list is provided in supplementary File S2).
Published between 1998 and 2024, the 46 included articles contributed to a total of 209 between-group effect sizes and 116 correlation coefficients, which aggregated statistics from 2,192 unique individual participants (1,108 non-overlapped schizophrenia patients and 1,084 non-overlapped HCs). Most of them were originated from the United States (k = 14, 30.43%), followed by Germany (k = 13, 28.26%), China (k = 6, 13.04%), the United Kingdom (k = 4, 8.70%), the Netherlands (k = 3, 6.52%), France (k = 2, 4.35%), Belgium (k = 1, 2.17%), Greece (k = 1, 2.17%), Israel (k = 1, 2.17%), and Switzerland (k = 1, 2.17%). Sample sizes ranged from 6 to 53 for the patient group, from 7 to 57 for the HC group, and from 13 to 109 for the total number of participants. For all the patient participants, the mean age ranged from 14.84 to 48.43 years, the proportion of male participants ranged from 32.26% to 100%, and the education level ranged from 8.4 to 14.8 years. Specific descriptive characteristics for literature search and qualitative synthesis are demonstrated in Table 1 (group differences) and Supplementary Table S6 (correlational coefficients). Qualitative syntheses of existing evidence are demonstrated in eResults in supplementary File S1.

Primary Outcomes

Group Differences

Primary outcomes and important subgroup analyses are displayed in Table 2. Aggregated statistics suggested that schizophrenia patients showed overall impairments at a moderate magnitude in audiovisual MSI tasks (g = -0.50, SE = 0.07; Figure 2). There existed high heterogeneity among these studies at both the between-cluster (τ2between-cluster = 0.053, I2 = 10.87%) and within-cluster (τ2within-cluster = 0.334, I2 = 69.00%) levels. However, the impairment of audiovisual MSI in schizophrenia seemed to be ascribed to the AVP (g = -0.53, SE = 0.09, p < .001; Supplementary Figure 1) and MM (g = -0.71, SE = 0.15, p < .001; Supplementary Figure 2) components, rather than the MU (g = -0.23, SE = 0.17, p = .225; Supplementary Figure 3) component. Subgroup Analyses further suggested that both behavioral (g = -0.52, SE = 0.07, p < .001; Supplementary Figure 4) and neural (g = -0.51, SE = 0.12, p < .001; Supplementary Figure 5) measures of the group differences in audiovisual MSI obtained similar results.
Our fMRI meta-analysis was conducted based on 35 available peak coordinates and t-values of clusters extracted from five studies. [19,31,68,69,70] There was no significant residual heterogeneity between studies (τ = 0.00, Q = 6.41, p = .17). The aggregated fMRI data reflected decreased activation in the left supramarginal gyrus (Brodmann area 48, cluster size = 161 voxels, MNI coordinates = [19,31,68,69,70], z = -2.07, puncorrected = .02, I2 = 10.69%) among schizophrenia patients during audiovisual MSI tasks. However, the significance disappeared after FWER-correction, so this aggregated evidence should be considered weak.

Correlations

Existing evidence only suggested weak correlations between the symptomatic measures and audiovisual MSI performance in schizophrenia (z = 0.16, SE = 0.07, p = .030; Figure 3). Moderate heterogeneity was identified at both the between-cluster (τ2between-cluster = 0.030, I2 = 28.47%) and within-cluster (τ2within-cluster = 0.041, I2 = 38.27%) levels. Neither the behavioral (z = 0.10, SE = 0.10, p = .348; Supplementary Figure 6) nor neural (z = 0.25, SE = 0.11, p = .077; Supplementary Figure 7) measures in MSI tasks were significantly associated with patients’ symptoms.

Moderator Analyses

Detailed statistics of all meta-regression models are displayed in Supplementary Table S7(continuous moderators for group difference), S8 (categorical moderators for group difference) and S9 (all moderators for correlational coefficients). Significant moderators associated with the group difference and correlation effects are presented in Supplementary Table S10. Detailed descriptions for moderator analyses are illustrated in eResults in supplementary File S1.
For MSI in general, schizophrenia patients with higher scores of the Brief Psychiatric Rating Scale (BPRS) showed increased audiovisual MSI capability (b = 0.038, p = .035). In addition, various measuring metrics were predictive of different group effects (F(8, 200) = 2.358, p = .019), with priming effects (PE; g = 0.55, 95 % CI [0.11, 1.00]) eliciting the best MSI performance in schizophrenia and d’-value (g = -1.04, 95 % CI [-1.23, -0.86]) resulting in the largest between-group difference. Neither social (F(1, 207) = 0.066, p = .798) nor linguistic (F(1, 207) = 0.181, p = .671) complexity was predictive of group effect sizes. Of the three components, AVP was associated with clinical characteristics, while MM and MU were significantly associated with experimental stimuli. For AVP, higher BPRS scores (b = 0.033, p = .013) were associated with smaller between-group differences, and various measuring metrics (F(8, 132) = 3.396, p = .001) could also differentiate the discrepancy in MSI performance between patients and HCs. As for MM, higher proportion of male participation (b = 0.027, p = .011) was predictive of better MSI adaptation within the schizophrenia group. As for the measuring indices for MM, the neural response (g = -1.04, 95 % CI [-1.66, -0.41]) could sensitively detect the largest negative effect sizes. Last, the between-group differences in integration ability, as reflected by the MU component, were only predicted by different stimuli (F(7, 23) = 2.685, p = .035), with soccer-beep stimuli (g = 0.64, 95 % CI [-0.29, 1.56]) predicting the most integration benefits for schizophrenia patients and number-noise stimuli the least (g = -1.01, 95 % CI [-1.86, -0.15]).
Interestingly, the more socially (F(1, 114) = 6.470, p = .012) or linguistically (F(1, 114) = 13.131, p = .0004) complex the stimuli were, the more possible the correlations between severe symptoms and worse MSI capability could be identified in schizophrenia. Of different measures, eye movements (z = 0.63, 95 % CI [-0.16, 1.42]) predicted the highest positive correlations, while PE (z = -0.17, 95 % CI [-2.82, 2.48]) and illusion rates (z = -0.02, 95 % CI [-0.94, 0.89]) were predictive of negative correlations.

Quality Assessment, Publication Bias and Sensitivity Analyses

Quality assessment (QA) scores (see eResults in supplementary File S1; Supplementary Table S11) in our meta-analysis ranged from moderate (score = 8) to high (score = 17) for individual studies, with an average of 13.67 (SD = 2.15, median = 14). The QA scores were not significantly associated with either group (b = 0.02, SE = 0.03, 95% CI [-0.05, 0.09], p = .46) or correlation (b = 0.07, SE = 0.03, 95% CI [-0.08, 0.22], p = .21) effects.
By Egger’s test, the funnel plots (Supplementary Figure 8) showed symmetrical distributions of both between-group (b= -0.814, 95% CI [-2.05, -0.42], p = .20) and correlation (b = 0.182, 95% CI [-0.76, 1.13], p = .71) effect sizes. The trim-and-fill analyses added 24 studies for the group effects, resulting in a corrected medium pooled effect size (k = 233; g = -0.57, SE = 0.08, 95 % CI [-0.73, -0.42], prediction interval [-1.88, 0.73], p < .001), and five studies were added for the correlation effects, remaining a corrected small pooled effect size (k = 121; z = 0.18, SE = 0.07, 95 % CI [0.05, 0.31], prediction interval [-0.37, 0.73], p = .018).
Sensitivity analyses by the leave-one-out method identified 54 outliers for the group effects (Supplementary Figure 9), and the pooled effect size without outliers remained moderately negative (g = -0.54, SE = 0.05, 95 % CI [-0.63, -0.44], prediction interval [-1.04, -0.04], p < .001) with decreased overall heterogeneity (τ2 = 0.063, I2 = 39.78%). For the correlation effects, 16 outliers were eliminated (Supplementary Figure 10) and the pooled effect size remained small (z = 0.25, 95 % CI [0.15, 0.34], prediction interval [0.02, 0.47], p < .001) with low heterogeneity (τ2 = 0.011, I2 = 24.81%). The fail-safe N analysis suggested an addition of 30,131 and 1,776 unpublished studies could respectively negate the two pooled effect sizes, which corroborated the robustness of current findings.

Discussion

This systematic review and multilevel meta-analysis synthesized extant evidence on the audiovisual MSI capability of schizophrenia patients, which calculated pooled between-group effect sizes and aggregated the correlational effect sizes between schizophrenia symptoms and patients’ MSI performance. For the first time, we analyzed patients’ MSI capability respectively in three subordinate MSI components, and inspected the clinical correlations and significant moderators. Our findings challenged the convention that schizophrenia patients showed systematic impairments in MSI processing, and could address a few inconsistent previous findings by providing a new research perspective. We would like to demonstrate two noteworthy results here, and provide supplementary discussions in eDiscussion (supplementary File S1).
First, our findings acknowledged that the moderately impaired audiovisual MSI capability in schizophrenia was primarily ascribed to audiovisual processing (AVP) and adaptation (MM), rather than the integration (MU) itself. Our identified partially deficient audiovisual MSI capability (overall, AVP and MM) in schizophrenia supported previous review articles, which indicated the overall dysfunctional MSI in schizophrenia at the temporal, [45,46] (non-)emotional [25] and neurological [24] domains. Based on our meta-regression results (Supplementary Table S8), the typically used temporal tasks (e.g., SJ) and illusion-based tasks (e.g., McGurk) consistently revealed worse MSI performance of patients in the AVP and MM components. Surprisingly, our aggregated data identified only small between-group differences (g = -0.23) concerning the benefit brought by the multisensory integration (MU) itself. This corroborated sporadic evidence claiming that schizophrenia patients showed comparable [14] or even greater [71,72] performance enhancement than HCs while processing audiovisual stimuli, compared to the unisensory information. As displayed in Supplementary Table S8, paradigms involving spatial distance judgement, lexical decision, long-term memory and speech-in-noise recognition distinguished the impaired AVP and MM components from the less impaired or even enhanced MU component. Pertaining to different measuring indices, the TBW and d’-difference detected the largest deficits of patients in the AVP and MM components, while the RT difference between audiovisual and unisensory conditions was found to best reflect the relatively preserved integration ability of patients. However, since only 31 MU effect sizes were pooled, our meta-regression findings are currently tentative and require further examinations.
In addition to the traditional meta-analysis evaluating the between-group effect sizes, our additional fMRI meta-analysis identified weak evidence for decreased activation in the left supramarginal gyrus in schizophrenia patients during audiovisual MSI tasks, which supplemented one recent systematic review reporting patients’ abnormal EEG oscillations and amplitudes in temporal-parietal regions during MSI processing. [24] Empirical studies have also observed reduced activation in the left temporoparietal junction, [69] the general parietal areas (dorsal visual stream), and delayed peak in the intraparietal sulcus [73] in schizophrenia during audiovisual MSI tasks. However, since only five studies reported sufficient statistics for this meta-analysis and 88.57% (31 out of 35) of our extracted raw data reflected the AVP and MM components, our current fMRI meta-analysis only provided preliminary evidence and required further investigations. More qualitative discussions on the neurological findings are provided in eDiscussion (Supplementary file S1).
A few mechanisms have been proposed to interpret the between-group differences. First, impairments in audiovisual MSI performance could be attributed to the deficits in unisensory representations. [3] Since schizophrenia patients showed diminished decoding of the auditory speech signals [14,74] and abnormally disturbed visual perception, [17] the between-group differences in audiovisual MSI might possibly result from the combination of the weakness and instability of unisensory perception in schizophrenia. [2] Moreover, during audiovisual MSI tasks, the mutual influences between the two modalities may either facilitate or prohibit information processing, [12,75] and unisensory deficits can thus sway the MSI outcomes, which would even be amplified when participants were asked to selectively attend to one modality and inhibit distractions from the other modality. [6,76] However, recent research has reported that schizophrenia patients could still display insensitive audiovisual temporal acuity even if their unisensory temporal processing was normal, [77] so patients’ altered audiovisual MSI could not be satisfactorily explained by unisensory dysfunction alone. [21] To determine the mechanisms underlying MSI changes in schizophrenia, the Bayesian modelling method has shown its merits in a few atypical populations (e.g., children, older adults and autism populations). [78] Since MSI Bayesian models involve the computational process itself (i.e., the integration process) and the weighted unisensory estimates (i.e., unisensory processing), such models may supplement previous experimental evidence and help reveal the specific procedures resulting in the changes of MSI performance in schizophrenia. In addition to these interpretations, a few studies have also ascribed MSI changes in schizophrenia to impaired top-down modulation [79] or inaccurate cognitive event structures (e.g., temporal, spatial), [30] which might be independent of sensory binding operations. Although an investigation of the mechanisms underlying audiovisual MSI changes in schizophrenia is beyond the current scope, we encourage more sophisticated investigations on the pathological causes for MSI changes in schizophrenia beyond the phenomenal revelation.
Our second important finding was the weak correlation between schizophrenic symptoms and MSI performance, and the predictability of higher social or linguistic complexity for stronger correlations. During data extraction, most included studies did not report the correlation coefficients because of non-significant results. [5,29,33,34,35,36,37] Our analyses synthesized 116 coefficients from a subset of 17 studies, and the small pooled effect size in our meta-analysis in turn corroborated the widely acknowledged weak or null correlations in individual studies. One reason for the absence of clinical correlations with MSI capability was the relatively small sample size in previous studies. [14] Another possibility could be the inclusion of patients with only moderate to low levels of symptoms in most studies. Of all included samples in our study, the score ranges for different symptom measures were similarly restricted (PANSSP: 8.21 to 17.4; PANSSN: 10.13 to 23.9; PANSSG: 18.3 to 38.29; SAPS: 5.1 to 37.9; SANS: 5.6 to 46; BPRS: 15.4 to 46.93). Hence, with the restricted range of symptoms, the performance of symptomatically severe patients has been rarely observed, thus excluding the possibility of obtaining significant overall correlations between patients’ cognitive performance (e.g., audiovisual MSI) and clinical symptoms. Although our meta-regression analyses (Supplementary Table S9) suggested that the weak correlation was consistent across different symptom measures (e.g., BPRS, PANSS, SANS and SAPS) and symptom types (e.g., positive and negative), these findings concerning correlational effects might not reflect the whole schizophrenia spectrum and required extra caution for interpretation.
Besides the two primary findings, our meta-regressive results could also inspire future investigations. Although the conclusiveness of our findings was challenged because of the limitations in original studies, such as the small number of particular types of effect size, the restricted symptom coverage, and insufficient report of raw data, a few feasible directions can still be enlightened. Interestingly, we identified that social and speech complexity of stimuli was predictive of different correlational strengths, but did not seem to influence the overall group differences or any subordinate components. In specific, stimuli with complex social (e.g., face, body, gesture) or linguistic (e.g., word, phrase, sentence) characteristics may help detect severe symptoms when patients performed poorly in audiovisual MSI tasks. One possible explanation was that MSI processing of complex stimuli imposed extra difficulty by introducing higher cognitive demands and flexibility engagement, so they could detect the abnormalities in schizophrenia patients more sensitively without potential ceiling effects.11,70 However, the number of stimuli with high or low complexity for the correlation estimate was unbalanced, with much more socially complex (69.83%) but linguistically simple (86.21%) stimuli used in previous studies. As such, certain bias might be introduced due to the design of original studies, thus mitigating the accountability and predictability of our findings on stimuli complexity.
As for other significant meta-regressive findings, the unexpected relationship between higher BPRS score and smaller between-group difference requires further exploration, because higher scores in other scales of symptom measures did not consistently predict better MSI performance of patients (Supplementary Table S7). It should be noted that the significantly positive correlations here did not indicate better MSI performance of patients than healthy controls, but instead reflected mitigated impairments (i.e., smaller negative between-group effect sizes) of patients’ MSI capability. Here, we tentatively propose that the coexistence of hyposensitivity and hypersensitivity to sensory stimuli in schizophrenia may account for this correlation. [80,81,82] Specifically, patients tend to omit relevant sensory information and are inactive in seeking sensory input (hyposensitivity), and they, at the same time, perceive all concurrent sensory stimuli as similarly salient without effectively filtering irrelevant information because of the defective inhibitory gating (hypersensitivity). [18,80] Recent evidence found that higher sensory hypersensitivity was correlated with higher schizotypal traits and symptom severity. [82] Therefore, it could be possible that the patient group showed overall deficits in MSI than HCs partially due to sensory hyposensitivity, but patients with more severe or typical schizophrenic symptoms integrated multisensory inputs better than those with fewer symptoms because of their higher hypersensitivity level. In this case, the clinical implications of correlations between clinical characteristics (e.g., symptoms and illness duration) and patients’ performance still need clearer illustrations. For example, prior studies proposed that non-significant correlations might suggest the MSI performance as a trait marker of schizophrenia, [1] which was independent of the current status and illness duration. [42] It has also been recently pointed out that neurological changes (e.g., cerebellar activation and N1 amplitude reduction) during sensory integration may serve as a possible endophenotype of schizophrenia. [8,19] Therefore, both significant and non-significant correlational statistics are clinically indispensable for our understanding about schizophrenia, and their exact clinical meanings are still open for more substantial investigations.
Our findings demonstrated certain clinical implications, as the relatively preserved audiovisual integration ability in schizophrenia may promote novel designs of cognitive training programs. Traditional cognitive training for schizophrenia patients was mainly carried out through single sensory modalities from two weeks to 12 months. [83] Although such training improved patients’ certain higher-order cognition, it was still challenged for the sustainability of improvements, efficacy on symptom mitigation and the time course of training. [83,84,85] Our meta-analytical findings identified less impaired audiovisual integration capability (MU) in schizophrenia, so multisensory training programs are predicted to be more effective for patients, as they could comparably benefit from the information presented in different modalities. In support of our predictions, recent studies comparing the outcomes of audiovisual and unisensory training programs found that a short-term audiovisual training led to significantly stronger cognitive improvement (e.g., emotion identification, TBW) in schizophrenia patients, and the improvements lasted a week or even longer. [86,87] Based on our meta-regression findings, those MSI tasks that sensitively elicited patients’ less impaired MU procedure, such as distance judgement, lexical decision and speech-in-noise recognition, may also lead to satisfactory training outcomes. Another promising topic for training program design is to examine the transfer effects of MSI tasks, [88] that is, whether the MSI avenue is cost-efficient in enhancing any different but malleable cognitive domains. Our other reported statistics are also informative for clinical practice. For example, stimuli with high social or linguistic complexity were found to predict significantly stronger symptom-performance correlations, indicating that such complex stimuli may be used to modulate the outcomes of audiovisual training [89] and to evaluate symptom severity during early screening and prognosis of schizophrenia. Since research concerning the clinical interventions of MSI training on the recovery of patients’ cognitive functioning is still emerging, substantially more studies are urgently needed.
Our study also showed certain implications for academic research. Future studies on this topic should evaluate the MSI components in greater details, including the audiovisual processing (AVP), adaptation (MM) and integration benefits (MU). Of the three principles in MSI, [90] the spatial and causal inference principles have attracted much less attention than the temporal principle in existing schizophrenia-oriented MSI research. Additionally, the clinical relevance (e.g., symptom measures, illness duration, medication) and experimental manipulation (e.g., stimuli complexity, MSI tasks, measuring indexes) to each of these components also need to be explored. For example, the influence of medication on patients’ MSI performance has been mentioned as a limitation in most studies, [12,40,68,71] but there are still few studies specifically controlled and reported the effects of medication. Of all samples included in this study, almost all patients were under medication, some of whom were even taking multiple antipsychotic drugs. It is highly possible that behavior-symptom correlations in MSI tasks are altered across patients with or without taking medications, and even with different types of medications. More limitations and implications for primary studies are provided in eDiscussion (supplementary File S1).
There were important limitations to the current meta-analysis. We synthesized behavioral and neural effect sizes of audiovisual MSI performance in schizophrenia, which have introduced high heterogeneity in aggregated analyses. Although we have controlled the heterogeneity by multilevel models, subgroup analyses and moderator meta-regression, there may still be unaccountable heterogeneity. Moreover, our correlation analyses were highly constrained by the data report in the original articles. Most included studies only reported the significant correlational coefficients and omitted the non-significant ones. Due to the noticeably high variance in patients’ performance, future MSI studies in the schizophrenia population are encouraged to enroll more participants and provide the exact correlational statistics. Third, our current meta-analysis emphasized the clinical relevance of MSI ability on schizophrenic symptoms, but some other dimensions, such as cognitive ability and social functioning, may provide more direct implications on how MSI capability influences real-life living of schizophrenia patients. Of all 46 quantitatively reviewed studies, however, only 15 (32.61%) measured these dimensions. Moreover, different studies used various scales or test batteries, [5,33,40,68,73] and few have established correlations between MSI performance and the score of such measures. In this case, it was infeasible to quantitatively synthesize correlational findings between MSI ability and scores other than symptoms in the current study. We encourage future studies to continue establishing the critical links between MSI and higher-order abilities, especially for schizophrenia patients. Finally, our meta-analysis provided less information for neural activation in passive viewing paradigms, and for individuals with first psychotic episodes or early-stage psychoses, due to the lack of inclusion.
In conclusion, the current study, for the first time, quantitatively synthesized effect sizes in a systematic way on audiovisual MSI in schizophrenia, which not only revealed between-group differences in a series of behavioral and neural metrics, but also illustrated the strengths of the clinical correlations with great details. Moderate-level dysfunctions were identified in the overall audiovisual MSI capability in schizophrenia, but such impairments mainly existed in audiovisual processing (AVP) and adaptation (MM), rather than the integration itself (MU). Furthermore, extremely weak clinical correlations were observed between schizophrenic symptoms and patients’ MSI performance. Clinical characteristics (e.g., BPRS score) were predictive of overall between-group differences, and stimuli with complex social or linguistic characteristics could help observe stronger clinical correlations. Our aggregated findings of audiovisual MSI, which involved a less impaired subordinate integration procedure, warrant more sophisticated investigations in future studies and provide important clinical insights towards the design of cognitive intervention programs, which will hopefully improve the quality of life for this population.

Author Contributions

Enze Tang: Conceptualization, Data extraction, Formal analysis, Literature search, Methodology, Software, Visualization, Writing-original draft. Nuo Chen: Data extraction, Formal analysis, Literature search, Writing-review & editing. Jian Li: Data examination, Writing-review & editing. Hongwei Ding: Conceptualization, Data examination, Funding acquisition, Supervision, Writing-review & editing. Hui Chen: Conceptualization, Funding acquisition, Supervision, Writing-review & editing.

Funding

This work was supported by grants from Science and Technology Innovation 2030- “Brain Science and Brain-like Research” Major Project (No.2022ZD0210800), National Natural Science Foundation of China (No.32171046), Fundamental Research Funds for the Central University (226-2024-00118), Emerging Enhancement Technology under Perspective of Humanistic Philosophy supported by National Office for Philosophy and Social Sciences (No.20&ZD045) awarded to Hui Chen; and by grants from Major Program of National Office for Philosophy and Social Sciences of China (No.18ZDA293) awarded to Hongwei Ding.

Acknowledgments

None.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Haß K, Sinke C, Reese T, et al. Enlarged temporal integration window in schizophrenia indicated by the double-flash illusion. Cogn Neuropsychiatry. [CrossRef]
  2. Noel JP, Stevenson RA, Wallace MT. Atypical audiovisual temporal function in autism and schizophrenia: Similar phenotype, different cause. Eur J Neurosci. 1230. [CrossRef]
  3. Vanes LD, White TP, Wigton RL, Joyce D, Collier T, Shergill SS. Reduced susceptibility to the sound-induced flash fusion illusion in schizophrenia. Psychiatry Res. [CrossRef]
  4. Wallace MT, Stevenson RA. The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia. [CrossRef]
  5. Mayer AR, Hanlon FM, Teshiba TM, et al. An fMRI study of multimodal selective attention in schizophrenia. Br J Psychiatry. [CrossRef]
  6. Wood SM, Potts GF, Hall JF, Ulanday JB, Netsiri C. Event-related potentials to auditory and visual selective attention in schizophrenia. Int J Psychophysiol. [CrossRef]
  7. Gao T, Wang X, Cen H, et al. Cross-modal associative memory impairment in schizophrenia. Neuropsychologia, 1087. [CrossRef]
  8. Ghaneirad E, Saenger E, Szycik GR, et al. Deficient audiovisual speech perception in schizophrenia: an ERP study. Brain Sci. [CrossRef]
  9. Ross LA, Saint-Amour D, Leavitt VM, Molholm S, Javitt DC, Foxe JJ. Impaired multisensory processing in schizophrenia: Deficits in the visual enhancement of speech comprehension under noisy environmental conditions. Schizophr Res. [CrossRef]
  10. Mangelinckx C, Belge JB, Maurage P, Constant E. Impaired facial and vocal emotion decoding in schizophrenia is underpinned by basic perceptivo-motor deficits. Cogn Neuropsychiatry. [CrossRef]
  11. Van den Stock J, de Jong SJ, Hodiamont PP, de Gelder B. Perceiving emotions from bodily expressions and multisensory integration of emotion cues in schizophrenia. Soc Neurosci. [CrossRef]
  12. Zvyagintsev M, Parisi C, Chechko N, Nikolaev AR, Mathiak K. Attention and multisensory integration of emotions in schizophrenia. Front Hum Neurosci. [CrossRef]
  13. Liu T, Pinheiro AP, Zhao Z, Nestor PG, McCarley RW, Niznikiewicz M. Simultaneous face and voice processing in schizophrenia. Behav Brain Res. [CrossRef]
  14. Senkowski D, Moran JK. Early evoked brain activity underlies auditory and audiovisual speech recognition deficits in schizophrenia. Neuroimage Clin, 1029. [CrossRef]
  15. Gao C, Green JJ, Yang X, Oh S, Kim J, Shinkareva SV. Audiovisual integration in the human brain: a coordinate-based meta-analysis. Cereb Cortex, 5574. [CrossRef]
  16. Ding H, Zhang Y. Speech prosody in mental disorders. Annu Rev Linguist. [CrossRef]
  17. Adámek P, Langová V, Horáček J. Early-stage visual perception impairment in schizophrenia, bottom-up and back again. Schizophrenia. [CrossRef]
  18. Javitt DC, Sweet RA. Auditory dysfunction in schizophrenia: integrating clinical and basic features. Nat Rev Neurosci. [CrossRef]
  19. Li Z, Huang J, Hung KSY, et al. Cerebellar hypoactivation is associated with impaired sensory integration in schizophrenia. J Abnorm Psychol. [CrossRef]
  20. Pearl D, Yodashkin-Porat D, Katz N, et al. Differences in audiovisual integration, as measured by McGurk phenomenon, among adult and adolescent patients with schizophrenia and age-matched healthy control groups. Compr Psychiatry. [CrossRef]
  21. Stevenson RA, Park S, Cochran C, et al. The associations between multisensory temporal processing and symptoms of schizophrenia. Schizophr Res. [CrossRef]
  22. Surguladze SA, Calvert GA, Brammer MJ, et al. Audio-visual speech perception in schizophrenia: An fMRI study. Psychiatry Res Neuroimaging. [CrossRef]
  23. Feldman JI, Dunham K, Cassidy M, Wallace MT, Liu Y, Woynaroski TG. Audiovisual multisensory integration in individuals with autism spectrum disorder: a systematic review and meta-analysis. Neurosci Biobehav Rev. [CrossRef]
  24. Gröhn C, Norgren E, Eriksson L. A systematic review of the neural correlates of multisensory integration in schizophrenia. Schizophr Res Cogn. [CrossRef]
  25. Tseng HH, Bossong MG, Modinos G, et al. A systematic review of multisensory cognitive-affective integration in schizophrenia. Neurosci Biobehav Rev. [CrossRef]
  26. Bremner AJ, Lewkowicz DJ, Spence C. The multisensory approach to development, 1: AJ, Lewkowicz DJ, Spence C, eds. Multisensory Development. Oxford, UK: Oxford University Press; 2012, 2012.
  27. Hershenson, M. Reaction time as a measure of intersensory facilitation. J Exp Psychol. [CrossRef]
  28. Miller, J. Divided attention: Evidence for coactivation with redundant signals. Cogn Psychol. [CrossRef]
  29. de Boer-Schellekens L, Stekelenburg JJ, Maes JP, Van Gool AR, Vroomen J. Sound improves diminished visual temporal sensitivity in schizophrenia. Acta Psychol. [CrossRef]
  30. Martin B, Giersch A, Huron C, van Wassenhove V. Temporal event structure and timing in schizophrenia: preserved binding in a longer “now”. Neuropsychologia. [CrossRef]
  31. Szycik GR, Ye Z, Mohammadi B, et al. Maladaptive connectivity of Broca’s area in schizophrenia during audiovisual speech perception: An fMRI study. Neuroscience. [CrossRef]
  32. Gao C, Weber CE, Shinkareva SV. The brain basis of audiovisual affective processing: evidence from a coordinate-based activation likelihood estimation meta-analysis. Cortex. [CrossRef]
  33. Balz J, Roa Romero Y, Keil J, et al. Beta/Gamma oscillations and event-related potentials indicate aberrant multisensory processing in schizophrenia. Front Psychol, 1896. [CrossRef]
  34. Roa Romero Y, Keil J, Balz J, Niedeggen M, Gallinat J, Senkowski D. Alpha-band oscillations reflect altered multisensory processing of the McGurk illusion in schizophrenia. Front Hum Neurosci. [CrossRef]
  35. Stone DB, Urrea LJ, Aine CJ, Bustillo JR, Clark VP, Stephen JM. Unisensory processing and multisensory integration in schizophrenia: a high-density electrical mapping study. Neuropsychologia, 3178. [CrossRef]
  36. Thaler NS, Strauss GP, Sutton GP, et al. Emotion perception abnormalities across sensory modalities in bipolar disorder with psychotic features and schizophrenia. Schizophr Res. [CrossRef]
  37. Zvyagintsev M, Parisi C, Mathiak K. Temporal processing deficit leads to impaired multisensory binding in schizophrenia. Cogn Neuropsychiatry. [CrossRef]
  38. Foucher JR, Lacambre M, Pham BT, Giersch A, Elliott MA. Low time resolution in schizophrenia: lengthened windows of simultaneity for visual, auditory and bimodal stimuli. Schizophr Res. [CrossRef]
  39. Müller VI, Kellermann TS, Seligman SC, Turetsky BI, Eickhoff SB. Modulation of affective face processing deficits in schizophrenia by congruent emotional sounds. Soc Cogn Affect Neurosci. [CrossRef]
  40. Vogel BD, Bruck C, Jacob H, Eberle M, Wildgruber D. Effects of cue modality and emotional category on recognition of nonverbal emotional signals in schizophrenia. BMC Psychiatry. [CrossRef]
  41. Zhou HY, Cui XL, Yang BR, et al. Audiovisual temporal processing in children and adolescents with schizophrenia and children and adolescents with autism: evidence from simultaneity-judgment tasks and eye-tracking data. Clin Psychol Sci. [CrossRef]
  42. Lema YY, Gamo NJ, Yang K, Ishizuka K. Trait and state biomarkers for psychiatric disorders: Importance of infrastructure to bridge the gap between basic and clinical research and industry. Psychiat Clin Neurosci. [CrossRef]
  43. Garrido-Vásquez P, Jessen S, Kotz SA. Perception of emotion in psychiatric disorders: on the possible role of task, dynamics, and multimodality. Soc Neurosci. [CrossRef]
  44. Lin Y, Ding H, Zhang Y. Multisensory integration of emotion in schizophrenic patients. Multisens Res. [CrossRef]
  45. Thoenes S, Oberfeld D. Meta-analysis of time perception and temporal processing in schizophrenia: differential effects on precision and accuracy. Clin Psychol Rev. [CrossRef]
  46. Zhou HY, Cai XL, Weigl M, Bang P, Cheung EF, Chan RC. Multisensory temporal binding window in autism spectrum disorders and schizophrenia spectrum disorders: a systematic review and meta-analysis. Neurosci Biobehav Rev. [CrossRef]
  47. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med, 1000. [CrossRef]
  48. Hornix BE, Havekes R, Kas MJH. Multisensory cortical processing and dysfunction across the neuropsychiatric spectrum. Neurosci Biobehav Rev. [CrossRef]
  49. Aslam S, Emmanuel P. Formulating a researchable question: a critical step for facilitating good clinical research. Indian J Sex Transm Dis Aids. [CrossRef]
  50. Moran JK, Keil J, Masurovsky A, Gutwinski S, Montag C, Senkowski D. Multisensory processing can compensate for top-down attention deficits in schizophrenia. Cereb Cortex, 5536. [CrossRef]
  51. Nikitova N, Keane BP, Demmin D, Silverstein SM, Uhlhaas PJ. The audio-visual abnormalities questionnaire (AVAQ): development and validation of a new instrument for assessing anomalies in sensory perception in schizophrenia spectrum disorders. Schizophr Res. [CrossRef]
  52. Curtis MT, Ren X, Coffman BA, Salisbury DF. Attentional M100 gain modulation localizes to auditory sensory cortex and is deficient in first-episode psychosis. Hum Brain Mapp. [CrossRef]
  53. Stekelenburg JJ, Maes JP, Van Gool AR, Sitskoorn M, Vroomen J. Deficient multisensory integration in schizophrenia: an event-related potential study. Schizophr Res. [CrossRef]
  54. Hirano Y, Nakamura I, Tamura S. Abnormal connectivity and activation during audiovisual speech perception in schizophrenia. Eur J Neurosci, 1918. [CrossRef]
  55. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Commun Health. [CrossRef]
  56. Wells GA, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses, /: Health Research Institute. Available from: http.
  57. Viechtbauer W, Cheung MW. Outlier and influence diagnostics for meta-analysis. Res Synth Methods. [CrossRef]
  58. Pustejovsky, JE. clubSandwich: cluster-robust (sandwich) variance estimators with small-sample corrections. 2022. Available from: URL: https://CRAN.R-project.
  59. Pustejovsky JE, Tipton E. Meta-analysis with robust variance estimation: expanding the range of working models. Prev Sci. [CrossRef]
  60. Hedges LV, Tipton E, Johnson MC. Robust variance estimation in meta-regression with dependent effect size estimates. Res Synth Methods. [CrossRef]
  61. Tipton, E. Small sample adjustments for robust variance estimation with meta-regression. Psychol Methods. [CrossRef]
  62. Radua J, Mataix-Cols D, Phillips ML, et al. A new meta-analytic method for neuroimaging studies that combines reported peak coordinates and statistical parametric maps. Eur Psychiatry. [CrossRef]
  63. Albajes-Eizagirre A, Solanes A, Vieta E, Radua J. Voxel-based meta-analysis via permutation of subject images (PSI): theory and implementation for SDM. Neuroimage. [CrossRef]
  64. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol. [CrossRef]
  65. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ, 7109. [CrossRef]
  66. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. [CrossRef]
  67. Rosenthal, R. Meta-analytic procedures for social research. Thousand Oaks, CA: SAGE; 1991.
  68. Sass K, Heim S, Sachs O, et al. Neural correlates of semantic associations in patients with schizophrenia. Eur Arch Psychiatry Clin Neurosci. [CrossRef]
  69. Smucny J, Rojas DC, Eichman LC, Tregellas JR. Neural effects of auditory distraction on visual attention in schizophrenia. PLoS One, 6060. [CrossRef]
  70. Straube B, Green A, Sass K, Kirner-Veselinovic A, Kircher T. Neural integration of speech and gesture in schizophrenia: evidence for differential processing of metaphoric gestures. Hum Brain Mapp, 1696. [CrossRef]
  71. Stone DB, Coffman BA, Bustillo JR, Aine CJ, Stephen JM. Multisensory stimuli elicit altered oscillatory brain responses at gamma frequencies in patients with schizophrenia. Front Hum Neurosci. [CrossRef]
  72. Surguladze S, Rossell S, Rabe-Hesketh S, David AS. Cross-modal semantic priming in schizophrenia. J Int Neuropsychol Soc. [CrossRef]
  73. Sanfratello L, Aine C, Stephen J. Neuroimaging investigations of dorsal stream processing and effects of stimulus synchrony in schizophrenia. Psychiatry Res Neuroimaging. [CrossRef]
  74. Wynn JK, Jahshan C, Green MF. Multisensory integration in schizophrenia: a behavioral and event-related potential study. Cogn Neuropsychiatry. [CrossRef]
  75. de Jong JJ, Hodiamont PP, de Gelder B. Modality-specific attention and multisensory integration of emotions in schizophrenia: reduced regulatory effects. Schizophr Res. [CrossRef]
  76. de Gelder B, Vroomen J, de Jong SJ, Masthoff ED, Trompenaars FJ, Hodiamont P. Multisensory integration of emotional faces and voices in schizophrenics. Schizophr Res. [CrossRef]
  77. Zhou HY, Lai IYS, Hung KSY, et al. Audiovisual temporal processing in adult patients with first-episode schizophrenia and high-functioning autism. Schizophrenia. [CrossRef]
  78. Jones SA, Noppeney U. Multisensory integration and causal inference in typical and atypical populations, 5: Y, Zaidel A, eds. Advances of Multisensory Integration in the Brain. Adv Exp Med Biol. Singapore: Springer; 2024;1437, 2024. [CrossRef]
  79. Roa Romero Y, Keil J, Balz J, Gallinat J, Senkowski D. Reduced frontal theta oscillations indicate altered crossmodal prediction error processing in schizophrenia. J Neurophysiol, 1396. [CrossRef]
  80. Brown C, Cromwell RL, Filion D, Dunn W, Tollefson N. Sensory processing in schizophrenia: missing and avoiding information. Schizophr Res. [CrossRef]
  81. van den Boogert F, Klein K, Spaan P, et al. Sensory processing difficulties in psychiatric disorders: a meta-analysis. J Psychiatr Res. [CrossRef]
  82. Zhou HY, Yang HX, Cui XL, et al. Self-reported sensory responsiveness patterns in typically-developing and early-onset schizophrenia adolescents: its relationship with schizotypal and autistic traits. J Psychiatr Res. [CrossRef]
  83. Dondé C, Mondino M, Brunelin J, Haesebaert F. Sensory-targeted cognitive training for schizophrenia. Expert Rev Neurother. [CrossRef]
  84. Bechi M, Bosia M, Spangaro M, et al. Visual and audio emotion processing training for outpatients with schizophrenia: an integrated multisensory approach. Neuropsychol Rehabil, 1131. [CrossRef]
  85. Scoriels L, Genaro LT, Keffer S, et al. Changes in emotion processing and social cognition with auditory versus visual neuroscience-informed cognitive training in individuals with schizophrenia. Schizophr Res. [CrossRef]
  86. Jeong JW, Kim HT, Lee SH, Lee H. Effects of an audiovisual emotion perception training for schizophrenia: a preliminary study. Front Psychiatry, 5220. [CrossRef]
  87. Zerr M, Freihorst C, Schütz H, et al. Brief sensory training narrows the temporal binding window and enhances long-term multimodal speech perception. Front Psychol, 2489. [CrossRef]
  88. Powers AR III, Hillock-Dunn A, Wallace MT. Generalization of multisensory perceptual learning. Sci Rep, 2337. [CrossRef]
  89. De Niear MA, Gupta PB, Baum SH, Wallace MT. Perceptual training enhances temporal acuity for multisensory speech. Neurobiol Learn Mem. [CrossRef]
  90. Stein BE, Stanford TR. Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci. [CrossRef]
Preprints 115311 g001
Figure 1. PRISMA flow diagram for literature search and study selection.
Figure 1. PRISMA flow diagram for literature search and study selection.
Preprints 115311 g001
Preprints 115311 g002
Figure 2. Forest Plot of Individual Effect Sizes of Group Differences in Audiovisual Multisensory Integration between Schizophrenia and Healthy Controls.
Figure 2. Forest Plot of Individual Effect Sizes of Group Differences in Audiovisual Multisensory Integration between Schizophrenia and Healthy Controls.
Preprints 115311 g002
Preprints 115311 g003
Figure 3. Forest Plot of Individual Effect Sizes of Correlations Between Audiovisual Multisensory Integration and Symptom Measures in Schizophrenia.
Figure 3. Forest Plot of Individual Effect Sizes of Correlations Between Audiovisual Multisensory Integration and Symptom Measures in Schizophrenia.
Preprints 115311 g003
Table 1. Characteristics of Studies Included in Quantitative Analyses for Group Effects.
Table 1. Characteristics of Studies Included in Quantitative Analyses for Group Effects.
ESa SZ HC Experiment Methods
Cluster/Study g r n % male Mage n % male Mage Task(s) Stimuli
Alain et al. (1998) 5 0 15 100.00% 46.00 15 53.33% 46.00 TI flash-beep
Balz et al. (2016) 3 0 15 80.00% 33.87 15 80.00% 36.13 SIFI flash-beep
de Gelder et al. (2002) 1 0 18 83.33% 36.40 12 91.67% 41.20 McGurk face-syllable
de Gelder et al. (2005) 2 0 13 84.62% 37.00 13 N/A N/A EI face-sentence/prosody
Foucher et al. (2007) 2 1 29 70.00% 33.00 33 66.67% 32.00 SJ flash-beep
Froud et al. (2010) 5 0 6 50.00% 44.90 7 42.86% 43.20 LD word-word
Gao et al. (2023) 1 1 32 59.38% 28.30 29 51.72% 28.00 LTM picture-sound
Ghaneirad et al. (2023a) 8 0 21 61.90% 37.86 20 33.33% 31.90 LTM picture-sound
Ghaneirad et al. (2023b) 8 5 20 60.00% 41.90 21 57.14% 38.20 SN: WI face-word
Hanlon et al. (2016) 8 0 33 81.82% 38.09 34 82.35% 37.50 TI flash-beep
Haß et al. (2017) 2 0 15 46.67% 41.60 15 46.67% 39.50 SIFI flash-beep
Li et al. (2021) 5 2 52 51.90% 26.33 52 44.20% 27.85 TI flash-beep
Liu et al. (2016) 10 0 18 94.12% 46.90 19 94.44% 44.60 TI face-burst
Lu & Pan (2020) 1 0 33 51.50% 31.79 26 46.15% 32.38 MJ face-syllable
Mangelinckx et al. (2017) 2 0 32 53.13% 47.31 32 N/A 46.41 EI face-word/prosody
Martin et al. (2013) 7 0 26 65.38% 37.00 26 65.38% 38.60 McGurk, SJ face-syllable
Mayer et al. (2015) 6 0 33 87.88% 36.00 33 87.88% 34.55 TI number-number
Müller et al. (2014) 4 18 15 73.33% 35.10 15 80.00% 40.80 EI face-burst
Pearl et al. (2009) 3 0 30 76.67% 26.40 20 60.00% 25.90 McGurk face-syllable
Roa Romero et al. (2016a) 6 1 17 70.59% 35.24 17 76.47% 36.00 SI face-syllable
Roa Romero et al. (2016b) 6 0 14 71.43% 35.57 14 71.43% 36.79 McGurk face-syllable
Ross et al. (2007) 1 0 18 94.44% 39.00 18 61.11% 35.00 SN: WI face-word
Sanfratello et al. (2018) 4 32 53 83.02% 39.50 56 69.64% 36.30 DJ soccer-beep
Sass et al. (2014) 8 3 14 57.14% 36.40 14 85.71% 31.20 LD word-word
Senkowski & Moran (2022) 5 0 24 62.50% 39.12 21 57.14% 37.76 SN: SI face-syllable
Simpson et al. (2013) 3 0 31 51.61% 40.61 30 50.00% 40.87 EI face-sentence/prosody
Smucny et al. (2013) 10 3 21 66.67% 46.90 23 56.52% 39.40 TI number-noise
Stone et al. (2014) 3 0 46 84.78% 39.20 57 70.18% 39.40 DJ soccer-beep
Straube et al. (2013) 21 0 16 62.50% 38.00 16 100.00% 27.90 TI face/gesture-sentence
Surguladze et al. (2001) 1 0 14 71.43% 35.90 7 71.43% 35.70 WI face-word
Surguladze et al. (2002) 3 1 20 55.00% 34.00 24 46.15% 30.30 LD word-word
Szycik et al. (2013) 2 0 15 46.67% 38.20 15 46.67% 36.50 TI face-word
Toumaian et al. (2024) 1 0 35 74.29% 28.29 32 43.75% 26.62 TI flash-beep
Tschacher & Bergomi (2011) 2 6 34 79.41% 27.88 34 76.47% 27.94 CJ flash-beep
Van den Stock et al. (2011) 2 0 16 93.80% 36.80 16 56.30% 38.00 EI body action-human burst
Vanes et al. (2016) 4 0 40 80.00% 37.00 22 86.00% 36.40 SIFI flash-beep
Vogel et al. (2016) 1 3 21 61.90% 36.52 21 61.90% 36.29 EI face-word/prosody
Wallace Cluster
Noel et al. (2018) 2 0 14 50.00% 42.30 15 37.50% 41.90 SJ face-syllable
Stevenson et al. (2017) 4 6 16 50.00% 42.30 16 37.50% 41.90 SJ flash-beep; face-syllable
White et al. (2014) 1 2 30 90.00% 39.03 24 87.50% 36.67 McGurk face-syllable
Wood et al. (2006) 6 0 13 N/A 48.43 13 N/A 49.71 TI flash-beep
Wu et al. (2017) 5 0 22 63.55% 29.00 16 43.75% 27.44 MJ face-phrase
Wynn et al. (2014) 9 2 33 69.70% 48.40 30 70.00% 48.00 TI letter-beep
Zhou et al. (2021) 7 22 23 32.26% 14.84 28 36.67% 14.20 SJ, AD flash-beep; face-syllable; face-sentence
Zhou et al. (2022) 4 8 34 62.79% 25.93 48 62.50% 25.02 SJ flash-beep; face-syllable
Zvyagintsev et al. (2013) 5 0 15 65.00% 42.90 18 65.00% 42.50 TI, EI flash-beep; face-pseudowords
Note. ES = Effect sizes, SZ = Schizophrenia, HC = Healthy control, %male = Percentage of male participants, Mage = Mean age in years, TI = Target identification, SIFI = Sound induced flash illusion, McGurk = McGurk effect, N/A = Not available, EI = Emotion identification, SJ = Simultaneity judgement, LD = Lexical decision, LTM = long-term memory, SN = Speech-in-noise, WI = Word identification, MJ = Match judgement, SI = Syllable identification, DJ = Distance judgement, CJ = Causality judgement, AD = Audiovisual asynchrony detection task. aThe number of between-group (g) and correlational (z) effect sizes extracted from each study.
Table 2. Primary Outcomes and Subgroup Analyses for Between-group and Clinical Correlational Effects.
Table 2. Primary Outcomes and Subgroup Analyses for Between-group and Clinical Correlational Effects.
Outcome k g/z (95% CI) PI p τ2 I2, %
Group difference
Overall 209 -0.50 (-0.64 to -0.37) -1.73 to 0.72 <.001 0.387 79.87
MSI components
MM 37 -0.71 (-0.99 to -0.42) -1.86 to 0.45 < .001 0.324 73.31
AVP 141 -0.53 (-0.71 to -0.35) -1.73 to 0.67 < .001 0.369 79.68
MU 31 -0.23 (-0.56 to 0.11) -1.23 to 0.78 .225 0.234 71.14
Measures
Behavioral 108 -0.52 (-0.66 to -0.37) -1.65 to 0.62 < .001 0.330 77.16
Neural 101 -0.51 (-0.74 to -0.27) -1.87 to 0.86 < .001 0.469 82.78
Correlation
Overall 116 0.16 (0.03 to 0.30) -0.38 to 0.70 .030 0.071 66.74
Measures
Behavioral 56 0.10 (-0.10 to 0.29) -0.57 to 0.76 .348 0.107 70.42
Neural 60 0.25 (0.04 to 0.46) -0.25 to 0.75 .077 0.053 63.95
Abbreviations: PI, prediction interval; MSI, multisensory integration; MM, multisensory-multisensory contrast; AVP, audiovisual processing; MU, multisensory-unisensory contrast.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
bsky
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

Privacy Settings

© 2025 MDPI (Basel, Switzerland) unless otherwise stated