Naturalistic stimuli in neuroscience: a gentle introduction
1Psychoinformatics lab, Institute of Neuroscience & Medicine (INM-7), Research Centre Jülich, Germany; 2Institute of Systems Neuroscience, Medical Faculty, Heinrich-Heine-University Düsseldorf, Germany
Naturalistic stimuli are defined as „a class of stimuli that aim to evoke more naturalistic patterns of neural responses than traditional controlled artificial stimuli“ (Vanderwal et al., 2019). For example, the reproducible timecourses of audio-visual movies or auditory narratives offer a continuous and rich stimulation, and thus better mimic our everyday environment that we, and our brains, are accustomed to. On the one hand, the talk will address advantages of naturalistic stimuli like improved validity, better compliance and higher data quality. On the other hand, challenges like the unknown confound structure of naturalistic stimuli will be discussed. The talk will illustrate solutions to these challenges and introduce methodological advances as employed and developed further by the following speakers of the symposium. For that reason, the talk will give an overview of seminal studies in the field that stimulated current research: most importantly, studies have shown that naturalistic stimuli evoke time-locked and synchronized spatiotemporal responses across multiple subjects in large parts of the brain. Lastly, the talk will cover potential future, clinical applications, and introduce publicly available datasets that can be downloaded and used by anybody to investigate independent research questions.
Studying brain functional organization with naturalistic viewing: reliability and cognitive relevance
1Centre for Sleep & Cognition & Centre for Translational Magnetic Resonance Research, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore 2Clinical Imaging Research Center, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore; 2Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore; 3Integrative Sciences and Engineering Programme (ISEP), National University of Singapore, Singapore, Singapore; 4Key Laboratory of Behavioral Sciences, Research Center for Lifespan Development of Mind and Brain, Institute of Psychology, Chinese Academy of Sciences, Beijing, China; 5Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
Resting-state functional connectivity (rsFC) using fMRI is a leading method to study human brain function. However, its unconstrained nature often introduces unwanted behavioural confounds such as excessive motion and sleep, challenging data reliability. Naturalistic viewing paradigm has been proposed to improve compliance and reduce variability, but to what extent it could improve FC reliability remains unknown. To this end, we compared the test-retest (TRT) reliability of FC derived from RS and three different videos re-watched over three sessions. We found that videos with high level of engagement improved TRT reliability in terms of increased subject-wise reproducibility. Videos with low engagement did not necessarily have better reliability than RS. These results supported the use of naturalistic stimuli with stronger cognitive engagement when examining FC.
Naturalistic stimuli also differ in narrative structure, which may recruit different cognitive processes and brain functional organization. To better understand this relationship, inter-subject FCs (ISFCs) of each video were derived from each session separately. Analysis of the longitudinal changes in ISFC showed that video with minimal plot (Inscape) yielded widespread decrease in ISFC, while those with stronger plot yielded more widespread increase. These divergent changes in inter-subject synchrony may stem from the recruitment of memory and comprehension-related processing in narrated videos during re-watching, gradually aligning event representation and anticipation across individuals; videos without a plot instead invite more idiosyncratic interpretations or mind wandering. Together, proper use of naturalistic paradigm may improve the quality of FC metrics and enrich their functional interpretations.
Intra- and inter-individual differences in network connectivity during naturalistic stimuli
1Inst. of Neurosci. and Medicine, INM-7, Forschungszentrum Jülich; 2Institute of Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf, Germany; 3Center for Sleep and Cognition & Center for Translational MR Research, Yong Loo Lin School of Medicine, National University of Singapore
Recent work on naturalistic viewing (NV) paradigms has investigated if the complex and dynamic processing of movie clips can evoke individually unique patterns, that are robust across sessions. By reducing the intrasubject variability, movies could potentially increase the reliability of fMRI measurements and facilitate their use as a biomarker for various neuropsychological diseases. A previous study by Vanderwal et al. 1 showed that movie watching increased intra- as well as inter-subject functional connectivity (FC) correlations in comparison to resting state. While this study, as well as the majority of related studies, has focused on a whole-brain atlas, little attention has yet been paid to the influence of NV on the FC variability in single networks. During this talk, FC patterns of various selected functional networks under movie stimulation and resting-state will be explored. The focus will be on the ratio of intra- to inter-individual variability. The ultimate goal is to select a paradigm that decreases intra-subject variability and therefore increases the reliability of the measurement, but still remains significant inter-subject variability, and is, therefore, able to capture individual differences. Since brain networks differ in their functions and are consequently expected to differ in their level of engagement in the processing of movie clips, it is of special interest to see how NV influences the intra- and inter-subject variability on a single network level.
1. Vanderwal T, Eilbott J, Finn ES, Craddock RC, Turnbull A, Castellanos FX. Individual differences in functional connectivity during naturalistic viewing conditions. NeuroImage. 2017;157:521-530. doi:10.1016/j.neuroimage.2017.06.027
Commonality and idiosyncrasy in responding to naturalistic stimulus during movie watching
Research Centre Juelich, Germany
When exposed to a naturalistic stimulus, different individuals have both commonality and idiosyncrasy in responding to the same complex stimulus. Previous studies have shown that the similarity of brain activity across individuals during movie-watching, which is typically characterized by inter-subject synchrony, varies across different brain areas. However, to what extent the brain activity is driven by the naturalistic stimulus and to what extent individuals respond differently to the stimulus have not been systematically examined yet. This study aims to establish a robust topography for the stimulus-driven response and the idiosyncrasy in response to the naturalistic stimulus during movie-watching. A principal component analysis (PCA) was employed to disentangle the stimulus-evoked response from the observed BOLD signals of multiple individuals and quantify its contribution to each brain region. In the meantime, the individual differences in response to the stimulus was captured in terms of subject loadings. Movie fMRI data of 178 subjects from the Human Connectome Project acquired at 7T with three different movie clips were analyzed. We show that the movie stimulus has the most influence on the sensory areas and the Wernicke's area, and areas in the frontal cortex exhibit highest inter-subject variability in response to the movie stimulus. Furthermore, we show that the individual differences in response to the stimulus have great potential in predicting cognitive phenotypes. This study provides novel insights into how naturalistic stimulus influences the brain activity and how individual responses to the stimulus could be used to investigate human behaviour for future studies.
Mapping activity in language areas during listening to a naturalistic audio stimulus
1Department of Psychology, Bielefeld University, Bielefeld, Germany; 2Psychoinformatics Lab, Institute of Neuroscience and Medicine (INM-7), Research Centre Jülich, Germany; 3Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Germany; 4Center of Excellence Cognitive Interaction Technology (CITEC), University of Bielefeld, Bielefeld, Germany
Auditory narratives are engaging stimuli which allow to acquire rich data of brain function. However, in the case of language, passive listening is less able to depict the common left-lateralization of language functions. The present work uses naturalistic fMRI to map language areas in individual participants and to characterize which stimulus features can predict activity in the language network.
An audio version of the movie Forrest Gump was presented to 19 participants while collecting fMRI data (studyforrest.org). A training set of the data was used to identify a language network and fit different features (e.g. syntactical properties of sentences) to the network's time course. The weights of these features were then applied to transcripts of held-out segments of the stimulus, to predict each participant's pattern of language-related brain activity.
Group-ICA of the training data revealed a network of left inferior frontal and posterior temporal areas, shared by most participants. This network followed a common time course, allowing to model its rise and fall through a weighted combination of different syntactic features. Applying this model to each participant's fMRI data from previously held-out runs, produced individual maps which were most similar to each participant's respective training data.
The analyses illustrate how language functions can be mapped by using a naturalistic stimulus and its annotation. While mapping networks on the level of individual participants is potentially useful for clinical applications, important next steps will be to test the approach in participants with atypical language representations, as well as in patients.