Brain-Network Temporal Variability in Resting-State fMRI: A Test-Retest Reliability Study

1. Introduction

Recent years have witnessed the growing application of brain connectomic methods to investigate the intricate organization of brain functions and the neural underpinnings of various dysfunctions [1,2]. On the basis of functional magnetic resonance imaging (fMRI) and graph theory, functional brain connectomics provides a non-invasive window into the brain functioning by measuring blood-oxygen-level-dependent (BOLD) signals, which serve as a proxy for neural activity. By analyzing the statistical dependencies between BOLD time series from different brain regions, researchers can reconstruct functional connectivity network maps of the brain’s functional communication pathways and further estimate topological metrics of these networks [1,2,3]. Using the brain connectomic approach, disrupted interactions within and between large-scale brain networks have been demonstrated in multiple common psychological conditions and disorders, such as addictions [4], schizophrenia [5], autism spectrum disorders [6], and mood disorders [7,8].

The earlier brain connectome studies were predominantly performed under a critical assumption: that the architecture of large-scale functional brain networks remains stationary over the duration of a typical fMRI scanning session. However, this “static” functional connectivity model has been argued to be too simplistic by some researchers later since the brain is not a static organ; it is a highly adaptive system whose functional configuration evolves continuously in response to internal goals and external demands [9,10]. In response to this limitation, the “dynamic functional connectivity/dynamic brain network” model has rapidly gained prominence and has been widely adopted in recent years [9,11,12]. Dynamic functional connectivity/dynamic brain network methods aim to capture the fluctuations of functional interaction patterns over time, providing a more precise picture of brain function. Approaches in such a framework can capture transient brain-network states [13], rapid reconfigurations of dynamic brain community structures [14,15], and meta-stable dynamics of edges (connections) in brain network that can be ignored by conventional static analyses [16]. The emergence of this dynamic framework has profoundly shifted our understanding of brain function from a “static” to a dynamic viewpoint.

One of the influential metrics within the dynamic brain connectome framework is the temporal variability of functional brain networks. As proposed by Zhang et al. [17], this measure quantifies the degree to which the functional connectivity profile within a given brain network, or between particular pairs of brain networks, fluctuates over time. The study by Zhang et al. [17] demonstrated significantly lower temporal variability within regions of the default-mode network, but higher temporal variability within the subcortical regions, in patients with autism/attention deficit hyperactivity disorder compared with normal controls. The latter studies have further reported excessively increased or decreased temporal variability of brain networks in various neuropsychiatric disorders, such as schizophrenia [18,19,20], bipolar disorder [18], major depressive disorder [21,22], Parkinson’s disease [23], alcohol use disorder [24], and internet gaming disorder [25]. Alterations in brain-network temporal variability have also been associated with aging [26], sleeping [27], verbal creativity [28], social behaviors [29], problematic smartphone use [30], as well as depressive symptoms in general populations [31]. Generally, both excessive decreases and increases in brain-network temporal variability can be considered indicative of brain dysfunction [12]. Reduced temporal variability may imply an impaired ability to flexibly integrate information and switch between different states; in contrast, excessively increased temporal variability may indicate neural dynamics that are overly flexible and even randomized [16]. Thus, this metric has emerged as a powerful tool to characterize brain functional changes in both healthy and clinical populations, and as a potential core biomarker of psychopathology.

Despite its wide application, however, a fundamental question remains not adequately investigated: the reliability of using brain-network temporal variability to estimate brain dynamics, particularly during the resting state. In the field of neuroimaging, satisfactory test-retest reliability is of great importance for any proposed brain connectomic metric to be a potential biomarker for brain dysfunction [32]. Test-retest reliability, which is often estimated by the intraclass correlation coefficient (ICC) in neuroimaging studies, assesses the consistency of a measurement when the same individual is scanned repeatedly [33,34]. High test-retest reliability is essential for precisely characterizing an object and its associations with other variables of interest [35]. To date, while the test-retest reliabilities of static brain functional connectivity [33,34,36] and several other dynamic metrics [32,37,38,39] are relatively well-documented, the test-retest reliability of brain-network temporal variability is still little known to our knowledge. Furthermore, it is unclear whether the reliability of brain-network temporal variability could be influenced by key parameters in the data acquisition and processing steps (e.g., the total fMRI scan duration), which can affect the reliability of resting-state fMRI metrics as reported in other studies [40,41].

Therefore, this study is designed to directly address the above questions by evaluating the test-retest reliability of brain-network temporal variability using a relatively large-sample, test-retest resting-state fMRI dataset. Our investigation will also explore the influence of several critical factors: (1) the step length and window width parameters chosen during the sliding-window steps when constructing dynamic brain networks, and (2) the total fMRI scan duration. By examining how these factors affect test-retest reliability, we seek to establish evidence-based methodological guidelines for applying brain-network temporal variability. The findings will provide practical guidance for future studies utilizing this metric.

2. Materials and Methods

2.1. Data Acquirement and Preprocessing

The present study utilized a sample of 337 healthy, unrelated individuals from the Human Connectome Project (HCP) [42] “S1200” dataset, which is consistent with a prior published study [32]. The participants consisted of 157 males and 180 females, ranging in age from 22 to 37 years (mean age = 28.61 ± 3.69). To ensure that results were not dependent on the specific sample, we randomly split the entire sample into two subsamples (N = 168 and N = 169, respectively) (Figure 1A). All subsequent analyses were performed separately in these two subgroups to see whether the results were consistent across different sets of participants. There were no significant differences in age, sex distribution, or head motion measured by mean framewise displacement (FD) between the two subsamples (all p > 0.05, see Table 1).

For each subject, four distinct resting-state fMRI scans were acquired over two separate sessions (Figure 1B). Each scanning run was acquired with a repetition time (TR) of 0.72 seconds and contained 1200 time points. The HCP’s minimal preprocessing pipelines were applied to all fMRI data, which incorporated ICA-FIX denoising. This procedure is suggested as a more targeted and effective method for removing non-neural noise components compared to conventional global signal regression. For comprehensive details regarding the sample and preprocessing, readers are referred to prior publications [43,44,45,46,47]. Informed consent was obtained from all participants under the protocol approved by the Institutional Review Board at Washington University. The ethical committee of the Second Xiangya Hospital, Central South University (Changsha, China) also granted approval for this analysis (2024013).

2.2. Constructing Dynamic Brain Networks

A dynamic brain network is characterized by the temporal evolution of connections between its nodes [48]. In the current study, to ensure that results were not dependent on the specific atlas, we used two different atlases to define brain nodes: the automated anatomical labeling (AAL) [49] and the Power [50] atlas. Both these two atlases have been validated for neuroimaging analyses [32,34] and widely used in previous studies of dynamic brain networks [15,30,51,52,53]. All subsequent analyses were performed separately using the two atlases. A total of 90 nodes were defined based on the AAL atlas, and 264 nodes were defined based on the Power atlas. According to previous published work [15,32,50,54,55], these nodes were assigned into nine large-scale brain networks including the default-mode, salience, visual, subcortical, auditory, fronto-parietal, cingulo-opercular, sensorimotor, and attention networks.

Time-varying dynamic brain networks were constructed using the commonly used sliding-window approach [21,32,56,57]. Briefly, mean fMRI signals were extracted from all 90 or 264 nodes and segmented into a sequence of successive, partly overlapping time windows. In the primary analyses, we employed a 100-second window width (approximately 139 TRs) with a 40-second sliding step length (56 TRs), yielding a total of 19 time windows. The selection of such window width was based on recommendations in previous work [21,32,58,59], and we examined how window width and step length affected the results in subsequent analyses. For each time window, we computed the dynamic functional connectivity between all node pairs using Pearson correlation, resulting in a 90 × 90 or 264 × 264 connectivity matrix. Each of these continuous correlation matrices represents a weighted, time-dependent “snapshot” of brain-network structures. Finally, we assembled these sequential matrices into a multilayer dynamic graph, G = (G_t)_{t = 1, 2, 3, …,T}, where the tth “snapshot” (G_t) represents the brain-network structures at the tth time window. Note that while T = 19 in the primary analysis, its value varied in subsequent analyses depending on the specific combinations of window width, step length, and scan duration.

2.3. Temporal Variability

After constructing dynamic brain networks, brain-network temporal variability was calculated to quantify the degree to which the network connectivity profiles fluctuate over different time windows (Figure 1C). Temporal variability was computed at two spatial scales: the network level (encompassing connectivity within a single network or between pairs of networks) and the global whole-brain level. Briefly, the within-network temporal variability for a given network (e.g., the visual network) was computed as:

$${Temporalvariability}_{\left(visual\right)}=1-\overline{corrcoef\left(F_{i\left(visual\right)}, F_{j\left(visual\right)}\right)} i, j=1,2,3,\dots ,T;i\ne j$$

(1)

where F_{i (visual)} is a vector representing the dynamic function connectivity between all node pairs within the visual network at the ith time window; T is the total number of time windows and equals to 19 in the primary analysis; the value of T changed in subsequent analyses with different combinations of parameters (e.g., window width and step length). Similarly, the between-network temporal variability for a pair of networks (e.g., the visual and auditory networks) was computed as:

$${Temporalvariability}_{(visual-auditory)}=1-\overline{corrcoef\left(F_{i(visual-auditory)},F_{j(visual-auditory)}\right)} i, j=1,2,3,\dots ,T;i\ne j$$

(2)

where F_{i (visual-auditory)} is a vector representing the dynamic function connectivity between all nodes in the visual network and all nodes in the auditory network at the ith time window; T is the total number of time windows (equals to 19 in the primary analysis and changed in subsequent analyses). The temporal variabilities for all other individual networks and pairs of networks were calculated using the same method. Additionally, global temporal variability was computed by treating all 90 or 264 nodes as a single whole-brain network. Consequently, the above analyses produced a total of 45 network-level (9 within-network and 36 between-network) and one global temporal variability values. More details about the process of computing brain-network variability can be found in previous publications [17,18,19,28].

2.4. Test-Retest Reliability

As in previous studies [32,33,34,60,61,62], following computing brain-network temporal variability, the test-retest reliability of temporal variability across four fMRI scans was evaluated by the intraclass correlation coefficient (ICC) with a two-way mixed, single measure model (ICC(3,1)) (Figure 1D). The ICC analysis was performed separately for each of the two brain atlases, each of the two subsamples, and each of the network-level/global temporal variability values. Theoretically, values of ICC range from 0 (suggesting no reliability) to 1 (suggesting perfect reliability). Based on established criteria in prior research [32,33,34,63,64,65,66], test-retest reliability was categorized into four levels using the following criterion: poor (ICC < 0.4), moderate (0.4 ≤ ICC < 0.6), good (0.6 ≤ ICC < 0.75), and excellent (ICC > 0.75). In the primary analysis, the ICCs were calculated with the following fixed data acquisition and processing parameters: scan duration = 14.4 minutes (1200 TRs; using the entire scan), window width = 100 seconds (139 TRs), and step length = 40 seconds (56 TRs).

2.5. Effects of Data Acquisition and Processing Parameters

Following the primary analysis, we performed several follow-up analyses to assess the effects of data acquisition and processing parameters. This was done by repeating the test-retest analyses with different settings for a particular parameter while keeping others constant. Details are as follows:

(1) Effect of step length: Following our primary analysis (which used a 40-second step length), we tested the possible impact of step length during the sliding-window approach by repeating the analyses with different values (including 10, 20, 60, 80, and 100 seconds) while keeping all other parameters fixed at their default values.

(2) Effect of window width: Following our primary analysis (which used a 100-second window width), we tested the possible impact of window width during the sliding-window approach by repeating the analyses with different values (including 40, 70, 150, 200, 300, and 400 seconds) while keeping all other parameters fixed.

(3) Effect of scan duration: Following our primary analysis (which used the full 14.4-minute scan), we tested the possible impact of scan duration by repeating the analyses using shorter segments of the fMRI data (3, 4, 5, 6, 8, 10, and 12 minutes) while keeping all other parameters fixed.

2.6. Exploratory Analyses of Sex and Age Effects

Previous publications have widely reported the significant effects of sex and age in neuroimaging and psychiatric research [3,32,67,68]. Thus, for exploratory purposes, we performed additional analyses to investigate the possible relationships between sex/age and brain-network temporal variability. The analyzing steps are as follows: (1) for each participant, the overall values of brain-network temporal variability were firstly obtained by averaging the values across the four fMRI scans; (2) to assess possible sex effects, the overall brain-network temporal variability was compare between the male and female participants using the analysis of covariance (ANCOVA) controlling for age and head motion (mean FD); (3) to assess possible sex effects, partial correlations were performed between the overall brain-network temporal variability and age, adjusted for sex and head motion (mean FD). The above analyses were performed for both the network-level and global temporal variability values. False discovery rate (FDR) corrections were performed across multiple tests at the network level, and results were considered significant at corrected p < 0.05. These analyses were performed independently for each combination of brain atlas (AAL and Power) and subsample, similar to the test-retest reliability analyses.

3. Results

3.1. Primary Analysis

The test-retest reliability of brain-network temporal variability from the primary analysis is shown in Figure 2. It was shown that with the default settings (window width = 100 seconds, step length = 40 seconds, and scan duration = 14.4 minutes), the temporal variability exhibited moderate (0.4 ≤ ICC < 0.6) to good (0.6 ≤ ICC < 0.75) test-retest reliability at the whole-brain level (Figure 2B), and for most networks/network pairs at the network level (Figure 2A). Such findings were generally consistent across the AAL and Power atlases, with the exceptions of slightly higher reliability for the Power atlas at the network level and for the AAL atlas at the whole-brain level. Furthermore, although similar trends were observed in both subsamples, the ICCs were overall higher in the first subsample (N = 168) than in the second subsample (N = 169). Among the network-level temporal variability metrics, temporal variability within the subcortical network was the only one showing poor reliability (ICC < 0.4) in both subsamples.

3.2. Effects of Data Acquisition and Processing Parameters

Effects of data acquisition and processing parameter on the test-retest reliability of brain-network temporal variability are presented in Figure 3, Figure 4 and Figure 5. The effect of step length used in the sliding-window approach is shown in Figure 3. Generally, we found that ICC values for temporal variability were similar across step lengths from 10 to 100 seconds. This consistency was evident at both the network (Figure 3A) and global (Figure 3B) levels, in both subsamples, and for both the AAL and Power atlases. Therefore, our results that the test-retest reliability of brain-network temporal variability is not significantly influenced across the tested range.

The effect of window width used in the sliding-window approach is shown in Figure 4. We found that ICC values for temporal variability generally decreased as the window width increased from 40 to 400 seconds, with a more pronounced decline beyond 100 seconds. Such a trend was observed for most networks at the network level (Figure 4A) and for the whole brain at the global level (Figure 4B). Furthermore, this trend was consistent across both subsamples, and for both the AAL and Power atlases. Therefore, our results demonstrate that excessively long window widths can reduce the test-retest reliability of brain-network temporal variability.

The impact of fMRI scan duration on reliability is shown in Figure 5. We found that ICC values for temporal variability generally decreased with shorter fMRI scan durations from 14.4 to 3 minutes. This trend was consistent at both the network (Figure 5A) and global (Figure 5B) levels, in both subsamples, and for both the AAL and Power atlases. Notably, the reliability of global temporal variability fell into the poor range (ICC < 0.4) in both samples when the scan duration was shorter than 5 minutes. These results suggest that shorter fMRI scan durations can substantially reduce the test-retest reliability of brain-network temporal variability.

3.3. Exploratory Analyses of Sex and Age Effects

As for analyses on sex and age effects of temporal variability, no results survived corrections at the network level (corrected p > 0.05). At the global level, most tests for the effects of sex and age on whole-brain temporal variability were not significant (p > 0.05). However, a consistent, albeit non-significant, trend was observed across both subsamples and atlases: female participants tended to show higher whole-brain temporal variability than males, and a positive relationship was suggested between age and whole-brain temporal variability (Figure 6A and Figure 7A). Notably, these trends became statistically significant (p < 0.05) when the entire sample was pooled together (Figure 6B and Figure 7B).

4. Discussion

In this study, we assessed the test-retest reliability of brain-network temporal variability during the resting state, as well as potential impacts of multiple data acquisition and processing parameters on its reliability using a test-retest fMRI dataset. Our main findings include: (1) when using conventional parameters during the data acquisition and dynamic network construction, temporal variability demonstrated at least moderate (ICC > 0.4) test-retest reliability across the whole brain and in most networks. (2) with other parameters held constant, the reliability was not significantly altered by different step lengths but decreased with longer window widths and shorter total fMRI scan durations.

The first important result of this study was that, when using a conventional window width of 100 seconds and a step length of 40 seconds for dynamic network construction with the sliding-window approach, the temporal variability exhibited moderate (0.4 ≤ ICC < 0.6) to good (0.6 ≤ ICC < 0.75) test-retest reliability at both the network and whole-brain levels (Figure 2). In the field of neuroimaging, at least moderate test-retest reliability (with an ICC > 0.4) is generally considered necessary for a brain connectomic metric to convincingly measure alterations in brain functions [32,33,34,63,64]. Despite the wide applications in both healthy [27,28,30] and clinical [17,18,19,20,21,22,23,24,25] populations, however, previous studies have seldom investigated the reliability of brain-network temporal variability based on resting-state fMRI. Here, our study filled this gap, providing evidence to support temporal variability as a relatively reliable metric in estimating brain dynamics.

Notably, while at least moderate test-retest reliability was observed for most brain networks, the subcortical network was an exception, with its temporal variability showing only poor reliability (ICC < 0.4) for both the AAL/power atlases and across both subsamples (Figure 2B). Several previous studies have reported that functional connectivity profiles of the subcortical network are more variable over time than those of other cortical brain networks [18,32]. This characteristic may be attributed to the central role of the thalamus and other subcortical structures as “relay stations” for cortical information flow, which involves complex and fluctuating dynamics [18,69,70]. Possibly due to such a reason, the subcortical network has consistently demonstrated the lowest reliability across multiple static and dynamic metrics (beyond temporal variability) in previous research [32]. Therefore, our results are in line with previous reports, suggesting that findings related to the subcortical network should be interpreted with more caution.

Our findings that both network-level and whole-brain temporal variability exhibited at least moderate test-retest reliability were generally consistent across two different atlases (AAL and Power) and across two randomly split subsamples. Although we observed slightly higher reliability with the functional Power atlas at the network level (Figure 2A) and with the AAL atlas at the whole-brain level (Figure 2B), the results suggest that brain-network temporal variability is a reliable metric largely independent of the specific atlas chosen. This may represent an advantage of temporal variability over some other functional brain network metrics, which have shown significantly poorer test-retest reliability with certain atlases (e.g., AAL) [34]. Furthermore, ICCs were overall lower in one subsample than in the other. Therefore, while generally satisfactory, the reliability of temporal variability may vary slightly across different study populations.

With other parameters fixed, we found that the test-retest reliability of brain-network temporal variability was not significantly altered by different step lengths (across 10 to 100 seconds) at both network and global levels (Figure 3). A key practical implication of such results is that researchers can opt for longer step lengths without significantly compromising the reliability of temporal variability when constructing dynamic brain networks. During the sliding-window approach, longer step lengths and longer window widths will reduce the number of windows, thereby decreasing computational cost for dynamic brain network analyses [71]. Consequently, a moderate increase in step length can be an effective strategy to achieve an optimal balance between methodological rigor and computational efficiency in large-scale studies. In contrast to the findings on step length, our results demonstrated a clear decline in test-retest reliability with longer window widths, which became particularly pronounced beyond 100 seconds (Figure 4). Therefore, employing a moderately sized window is crucial, as it ensures a sufficient number of time windows to reliably estimate fluctuations in connectivity strength between brain networks [72].

As for the effect of fMRI scan duration, our results revealed that test-retest reliability of the brain-network temporal variability exhibited a clear and progressive decrease as the total fMRI acquisition time was shortened (Figure 5). This finding directly corroborates many previous recommendations for employing longer scans to ensure data reliability in fMRI studies [73,74,75]. Here, we extend this principle to the domain of brain-network temporal variability, confirming that sufficient scan duration is necessary for computing temporal variability. A particularly critical finding was that the reliability of global temporal variability fell into the poor range (ICC < 0.4) in both subsamples when the scan duration was truncated to less than 5 minutes (Figure 5B). Therefore, too short fMRI scan durations can substantially reduce the test-retest reliability of brain-network temporal variability.

As an exploratory analysis, we additionally investigated the potential sex and age effects of brain-network temporal variability. Although most tests were not significant, we observed consistent trends across both subsamples and atlases: female participants tended to show higher whole-brain temporal variability than males, and a positive relationship was suggested between age and whole-brain temporal variability (Figure 6A and Figure 7A). Notably, these trends became statistically significant when the entire sample was pooled together (Figure 6B and Figure 7B). These findings align with some previous studies which report that the temporal stability of functional brain network is higher in females than males [32,76] and decreases with aging [26,32]. Thus, our study supports these conclusions from the perspective of temporal variability, highlighting the necessity of controlling for sex/age effects in related research. Furthermore, the emergence of significance in the larger pooled sample may highlight the enhanced sensitivity of larger sample sizes for detecting subtle effects in neuroimaging research.

Our study has several limitations. First, we used ony the AAL and Power atlases; future work may benefit from comparing a broader range of parcellation schemes. Second, the limited duration of each HCP run (14.4 minutes) precluded a thorough investigation of the effects of longer scan lengths on our findings. Third, our investigation was confined to the resting state; extending this research to task-based fMRI would be a valuable direction for future studies.

In conclusion, this study demonstrated that brain-network temporal variability is a relatively reliable metric for identifying inter-subject differences in brain dynamics. This reliability was shown to be robust across different brain atlases. However, producing consistent results requires the use of moderate window widths and sufficient scan durations. The exploratory analysis indicates that whole-brain temporal variability is higher in females and increases with age, highlighting the necessity of controlling for these demographic factors in related research. Accordingly, this study provides a valuable guide for future research that utilizes this metric.