Duration of the Workshop:
2,5 hours

Target Groups:
Researchers with general methodological interests in survey research and/or interest in the mathematical background of sampling, estimations, errors and assessment procedures.

Is the workshop geared at an exclusively German or an international audience?
International audience

Workshop Language:
English

Description of the content of the workshop:
The workshop focuses on the role of probability theory in the development and assessment of new techniques for data collection in survey research. As survey research advances more quickly and vigorously, the importance of understanding the fundamentals of probability theory is becoming more evident. We will emphasize the fundamentals of probability theory and explore how it relates to the evolution of survey research in the academic and business worlds. The topics will be presented with examples, simulations, brief proofs, and calculations that are relevant to the application and evaluation of innovative methods.

Goals of the workshop:
Course objectives are:
1) To explore the connection between probability theory and the application and evaluation of new methods in survey research;
2) to provide a systematic overview of the use of mathematical tools in mixed-method surveys and;
3) to demonstrate the relevance of probability theory to this field.

Necessary prior knowledge of participants:
None

Literature that participants need to read for preparation
None

Recommended additional literature
None

Information about the instructors:

Blanka Szeitl is a survey methodologist and PhD candidate in applied mathematics. She is lecturer at Department of Statistics at University of Eötvös Lorand and at Bolyai Institute of Mathematics at University of Szeged. She is the head of Survey Methods Room Budapest research group focusing on innovative sampling procedures and data correction methods. She is researcher at HUN-REN Centre for Social Sciences, where she analyzes data of the European Social Survey (ESS). She is a member of the ESS Sampling and Weighting Expert Panel working on the implementation of the ESS sampling strategy for countries participating in the ESS data collection. She is co-founder of Panelstory Opinion Polls, which is the first mixed-method probability panel in Hungary. Her research interests are survey sampling, innovative methods, probability theory and assessment procedures. She loves to read about the history of probability and statistics.

Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop?
No

Duration of the Workshop:
2.5 hours

Target Groups:
Persons who are interested in computational social science, big data, data science, open data, and open science.

Is the workshop geared at an exclusively German or an international audience?
international audience

Workshop Language:
English

Description of the content of the workshop:
The emerging availability of big data in the past decade has overcome traditional constraints in research, especially in the discipline of humanities & social sciences. Increasing availability of big data is changing our world and transforming conventional thoughts about decision-making. Data science aims to cope with issues of big data. By definition, it consists of three disciplines, i.e., hacking skills, advanced mathematics and statistics, and domain knowledge. Taking full advantage of big data requires not only knowledge about fundamentals of data science, but also the ability of implementation. Big data do not offer us enough insight and vision. We need to go further to build smart data through the process of enriching and integrating the quantity and quality from different sources of big data. In the meanwhile, open data and open science have emerged simultaneously in the past decade in light of growing calls for the need to examine research reproducibility.
This workshop aims at
(1) addressing how open data and smart data sets are built by integrating hacking skills, advanced math/statistics methods, and domain knowledge of various disciplines on the basis of data science and open science
(2) the role of online open data repositories in promoting crowd collaboration.
In the three disciplines of data science, the workshop focuses solely on how hacking skills and advanced math/stat are applied to build big data and smart data In the context of extracting valuable information embedded in source individual data, enriching the extracted information through the processes of cleaning, cleansing, crunching, reorganizing, and reshaping the source data. The data enrichment processes produce a number of data sets that contain no individual information but retain most of the source data information. The enriched data sets thus can be open to the public as open data.

Because the corresponding domain knowledge about hard-to-reach population research and Taiwan Indigenous Peoples (TIPs) is not easy to understand for the audience, the instructor will make a very short introduction. The workshop uses a set of open data in TIPD (Taiwan Indigenous Peoples Open Research Data, for details, see https://osf.io/e4rvz/) as an example to demonstrate big data, open data, smart data, data science, and open science. TIPD complies with FARE (Findable, Accessible, Interoperable, Reusable) data principle.
It consists of the following categories of open data from 2007 to 2022:
(1) categorical data,
(2) multi-dimensional data,
(3) population dynamics (e.g. see TPDD: https://www.rchss.sinica.edu.tw/capas/posts/11621),
(4) temporal geocoding data (e.g. see High-resolution visualizations of population distribution, migration dynamics, traditional communities at https://www.rchss.sinica.edu.tw/capas/posts/11393),
(5) household structure data,
(6) traditional TIPs community data (TICD at https://www.rchss.sinica.edu.tw/capas/posts/11205),
(7) generalized TICD query system as a smart data (see https://TICDonGoogle.RCHSS.sinica.edu.tw),
(8) genealogical data (not open to the public).
In the end, the workshop will briefly highlight the impact of open data on promoting crowd collaboration and that of smart data on making effective policy decision-making by using interactive migration dynamics derived from TIPD as an example (TIPD at https: https://www.rchss.sinica.edu.tw/capas/posts/11206; Interactive migration visualizations at https://www1.rchss.sinica.edu.tw/jplin/TIPD_Migration/).

Goals of the workshop:
(1) illustrating methods such as “old-school” multi-dimensional tables that are applied to build&update big open data in automation mode;
(2) demonstrating how open data is built to comply with FAIR, ethical, and legal requirements under the principles of open science;
(3) introducing techniques in record linkage&highly precise address-matching geocoding that enable to enrich temporal&spatial information in big data;
(4) to introduce techniques of data engineering&data sharing that enable us to build and integrate open data repositories systematically and automatically;
(5) to demonstrate why the process of online crowd collaboration to improve open data quality as an effective way to build smart data.

Necessary prior knowledge of participants:
No prior knowledge is required.Participants with knowledge or experince in hacking skills (e.g. digital infrastructure, programming, perfomance tuning of computing system, data engineering, etc.), and/or individual data processing skills (e.g.data cleanse, record linkage), and/or spatial data structure (e.g.spatial data, attribute data, fundamentals of GIS system, etc.) are particularly welcome.

Literature that participants need to read for preparation
None

Recommended additional literature
(1) Lin, Ji-Ping. 2017a. "Data Science as a Foundation towards Open Data and Open Science: The Case of Taiwan Indigenous Peoples Open Research Data (TIPD)," in Proceedings of 2017 International Symposium on Grids & Clouds, PoS (Proceedings of Science).

(2) Lin, Ji-Ping, 2017b, "An Infrastructure and Application of Computational Archival Science to Enrich and Integrate Big Digital Archival Data: Using Taiwan Indigenous Peoples Open Research Data (TIPD) as Example," in Proceedings of 2017 IEEE Big Data Conference, the IEEE Computer Society Press.

(3) Lin, Ji-Ping. 2018. "Human Relationship and Kinship Analytics from Big Data Based on Data Science: A Research on Ethnic Marriage and Identity Using Taiwan Indigenous Peoples as Example," pp.268-302, in Stuetzer et al. (ed) Computational Social Science in the Age of Big Data. Concepts, Methodologies, Tools, and Applications. Herbert von Halem Verlag (Germany), Neue Schriften zur Online-Forschung of the German Society for Online Research.

(4) Lin, Ji-Ping. 2021. "Computational Archives of Population Dynamics and Migration Networks as a Gateway to Get Deep Insights into Hard-to-Reach Populations: Research on Taiwan Indigenous Peoples," Proceedings of 2021 IEEE International Conference on Big Data, IEEE Computer Society Press.

Information about the instructor:

Dr. Ji-Ping Lin received his B.Sc. in Geography from National Taiwan University (Taiwan) in 1988, M.Sc. in Statistics from National Central University (Taiwan) in 1990, and Ph.D. in Geography in 1998 from McMaster University (Ontario, Canada). His main research specialty and interests include migration and population studies, labor study, survey study, scientific & statistical computing, big & open data, data science, and open science. He is serving as associate research fellow at Academia Sinica, Taiwan. The instructor worked in Taiwan’s Bureau of Statistics & Census as research scientist, with abundant real-world experiences in processing, integrating, and enriching various sources of large-scale raw data, as well as in survey planning, sampling design, and conducting surveys. Lin has been serving as consultant for a number of Taiwan’s central government agencies. Since 2013, the instructor devotes himself to the research on hard-to-reach population (HRP) and Taiwan Indigenous Peoples (TIPs). Based on the fundamentals of computational social science, data science and open science, he has been building a number of big open data and smart data.

Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop?
Participants are suggested to bring their own laptop or tablet computer with internet access.

Relevance & Research Question

This study examines media consumption patterns among Israeli users, during the 2023 Israeli-Hamas war. Drawing from the extensive body of literature on media use during wartime, this study investigates how civilians utilize different channels and platforms to fulfill their needs and perspectives amid this violent conflict. Specifically, consumption patterns will be analyzed as a function of users’ level of anxiety, and their psychological distance from the war. We hypothesized that (1) The extent of individual anxiety will predict differences in mainstream versus online media usage, and that (2) Psychological distance from the war will mediate the relationship between anxiety and media usage patterns.

Methods & Data

A structured questionnaire was delivered among a nationally representative sample of Jewish -Israelis aged 18 and above (n=500) during the third week of the war, October 2023. Maximum standard error was set at 4.5%. Sample size calculations conducted using G*Power were based on a medium-sized effect size to achieve 90% power in detecting significant differences.

Results

To test our first hypothesis (H1), a multiple regression analysis assessed the impact of anxiety on the usage of mainstream versus online media. The results indicated that anxiety significantly predicted an increase in mainstream media usage (B = .039, p < .05) but had no significant impact on alternative media usage (B = -.097, p > .05). suggesting that higher levels of anxiety were associated with a preference for mainstream media.

The second hypothesis (H2) involved a mediation analysis using Hayes' PROCESS macro. The analysis showed full mediation; the direct effect of anxiety on media usage became nonsignificant when accounting for psychological distance (B = .012, p > .05). However, the indirect effect of anxiety on media usage through psychological distance was significant (B = .053, 95% CI [.023, .129]), indicating that psychological distance completely mediates the relationship between anxiety and media usage patterns during wartime.

Added Value

This study contributes to the current literature on media consumption during wartime, by focusing on war-related anxiety as a drive, and by adopting ‘psychological distance’ to this field, analyzing it as another relevant variable.

Relevance & Research Question
In this era of digital convergence, our study examines how psychological factors such as engagement, transportation, enjoyment, and media event perception influence dual-screen usage during the 2022 FIFA World Cup. It aims to unravel the complex dynamics between these factors and assess their impact on viewers' interactions with match-related and unrelated content across dual screens.
Methods & Data
We surveyed a representative sample of 515 Israeli participants using a structured online questionnaire, which assessed variables including transportation, enjoyment, media event perception, and dual-screen usage. Our study utilized Pearson correlations and Hayes’ PROCESS model for advanced statistical analysis, exploring psychological factors' direct and indirect effects on dual-screen usage patterns.
Results

We found significant positive correlations between engagement, transportation, enjoyment, and media event perception with match-related and unrelated dual-screen usage. Specifically, the Pearson correlation coefficients were r = .56 for engagement with match-related dual-screen usage (p < .001) and r = .37 for engagement with match-unrelated dual-screen usage (p < .001), highlighting the strong association between these psychological factors and dual-screen behaviors.
Engagement significantly mediated the relationships between media event perception, transportation, enjoyment, and dual-screen usage. In particular, for match-related dual-screen usage, the indirect effect of media event perception through engagement was significant (95% CI, 0.067–0.149; F[2, 498] = 128.53; p < .001). For match-unrelated content, while direct effects were significant, indirect effects through engagement were not (95% CI [-.288, -.015] for direct; [-.014, .088] for indirect), indicating varied influence patterns for different content types.
All independent variables were positively correlated with match-related dual-screen usage and negatively correlated with match-unrelated usage. This suggests that higher levels of psychological engagement lead to more dual-screen activity related to the sports event.

Added Value

This study shows how psychological factors influence dual-screen usage during major sports events like the FIFA World Cup. It provides critical insights for media producers, advertisers, and digital strategists in developing engagement strategies and content for dual-screen platforms. It enriches the discourse on media consumption patterns in the context of global sports events, significantly enhancing our understanding of contemporary media engagement in a multi-screen world.

Relevance & Research Question

The wealth of survey data amassed over the last century represents an invaluable tool for understanding human beliefs, attitudes, and behaviors and how these have evolved. But although thousands of datasets are available to researchers, scholars are often unable to use more than a handful for any given project. One challenge is that many questions, even those asking about similar topics, employ different wordings and response options. Hence, it is often difficult to tell whether differences between responses to questions are indicative of items that track subtly different topics, methodological choices, or changes over time. Instead, scholars examining trends often limit analyses to the subset questions asked identically at multiple time points. The current study proposes a novel solution to identifying common questions across data collections.

Methods & Data

Using microdata from over 2000 distinct probability US surveys of abortion attitudes, we produce a vector of means for each abortion measure at the intersections of age, gender, race, religion, and location. These can then be correlated across surveys (with appropriate weighting) to determine how similar the measures are and to identify measures that appear to capture similar underlying constructs (through clustering and other dimension reduction). We then parameterize how estimates of that similarity shift depending on the data collection methods, survey firms, and the temporal distance between surveys.
Results

We show that this technique allows us to (1) identify the different types of historical questions that exist to measure views on abortion, (2) discern the similarity of those different types of questions, and (3) estimate how attitudes toward different types of questions have trended over time both overall and within population subgroups. We also find that stability of measures is relatively consistent for relations between items asked within 100 days of one-another, whereas it drops notably with longer time differences between measures.
Added Value

The study opens up novel methods for analysis of historical survey data.

Relevance & Research Question
In standardized surveys, the quality of responses is essential. Numerous studies discuss how respondents’ care and effort in answering survey questions is linked to personality, cognitive ability and socio-demographic variables such as age, education, and income, but only few researchers have studied the effect of political attitudes on response quality. Voogt & van Kempen (2002) find that differences between survey respondents and non-respondents in terms of political attitudes and behaviour exceed socio-demographic differences, and Barth & Schmitz (2016) argue that response quality systematically varies with ideological positions. Therefore, in this study we ask whether political attitudes are related to response quality in a standardized survey of the general population in Germany.
Methods & Data

German General Social Survey (ALLBUS/GGGS 2021), https://doi.org/10.4232/1.14002. The GGGS is a biennial survey based on a random sample of the German population. In 2021, due to the Covid-19 pandemic, it was fielded as a mail / web survey for the first time. The questionnaire was distributed in three randomized split versions. In the first step, indicators for response quality are constructed separately for each split version. These include non-substantial “response styles”, such as extreme responding and non-differentiation, as well as the proportion of item non-response. Subsequently, we conduct regression analyses with political attitudes (e.g. political interest, positions towards cultural and economic issues, intention to vote in upcoming election) as explanatory factors of response quality while controlling for socio-demographic variables, survey mode and number of contact attempts.
Results

The analyses show that differences in response quality in the GGGS 2021 are systematically related to age, education and political interest as well as other political attitudes.
Added Value
Research on the nexus between response quality and political attitudes is highly relevant, as systematic relationships between response quality and substantive variables of interest may seriously compromise surveys’ ability to capture “public opinion”. Whereas the GGGS is an offline-recruited population survey, effects of political attitudes on response quality are likely to be even more pronounced in (non-probability) online panels.

Relevance & Research Question
Understanding citizens’ architectural style preferences are important for aesthetically pleasing and sustainable urban environments. However, the opinions of the people are seldom considered in contemporary urban planning. In recent years, an intense debate has unfolded in Sweden about the role of politicians versus architects versus urban planners on how future urban landscapes should be built and, specifically, in what architectural style tradition. Thus, we explore architectural style preferences (classic vs modernist) of Swedish citizens. We model how preferences are predicted by sociodemographic and political factors.
Methods & Data
Data consisted of a non-probability sample (N=3119) and a probability sample (N=2125) from the online Swedish Citizen Panel at the University of Gothenburg. Participants were asked to state their preference for classic or modernist architecture and to associate design elements (building materials, level of details in facades and costs) with each tradition. Trust in various professional groups involved in urban planning, including architects, were also assessed.
Results
Findings demonstrate a general preference for classic over modernist architecture in the non-probability and probability-based sample (p <.001). Notably, no differences among different party supporters where observed, indicating low political polarization on architectural preferences. Further, a logit regression model underscored the negative association between classic architectural preferences and social and political trust, as well as trust in architects (p <.001).
Added Value
We show that Swedish citizens have a clear preference towards classical architecture and that this support is stable across sociodemographic groups and party preference. The results provide insights for policy makers in urban planning. Replication and extension of these findings is currently being collected in a large population representative mixed-mode survey on Swedish citizens. Results from the additional data collection will be presented at the conference.

Relevance & Research Question
With the increasing prevalence of digital work environments, understanding the impact of online interruptions on work efficiency becomes crucial. This study probes into how online interruptions' frequency and information richness affect the pace of work. It specifically examines two hypotheses: whether slow-frequency interruptions lead to a more efficient work pace compared to fast-frequency ones and how the nature of interruption information (lean: text-only vs. rich: image + text) influences the speed of work-related tasks.
Methods & Data
A 2 × 2 factorial experimental design was implemented, involving 250 participants in a simulated online trading game in which participants had to gain profits. The experiment manipulated two main variables: interruption frequency (slow vs. fast) and information richness (lean vs. rich). Participants' task completion times were recorded to measure work pace. Statistical analyses, including ANOVA and post hoc tests, were conducted to determine the effects of these variables on work efficiency.
Results
The ANOVA revealed significant main effects for interruption frequency on work pace, F(1, 246) = 8.97, p < .01, and for information richness, F(1, 246) = 6.54, p < .05. Participants dealing with slow-frequency interruptions had a mean work pace of 12.29 tasks per hour (SD = 4.21) compared to 13.87 tasks per hour (SD = 4.02) for those with fast-frequency interruptions. Surprisingly, lean information interruptions resulted in a faster work pace (M = 12.68, SD = 3.82) than rich information (M = 13.48, SD = 4.50). The interaction effect was significant, F(1, 246) = 9.33, p < .01, indicating that the most efficient work pace occurred under slow-frequency and lean information interruptions.
Added Value
This research sheds light on the nuanced effects of online interruptions in digital workplaces, challenging prevailing notions in media richness theory. By demonstrating that not only the frequency but also the type of information in interruptions can significantly influence work pace, it provides actionable insights for designing more productive digital work environments. These findings have implications for human-computer interaction designers, organizational psychologists, and workplace strategists aiming to optimize productivity in multi-tasking settings.

Relevance & Research Question

Research on AI has a long history, spanning seven decades (Jiang et al., 2022), but only recently have scholars began exploring AI's impact on everyday activities (Ertal, 2018). Over the last two years, one could witness a surge in the use of large language models like ChatGPT, Bard, and Dall-e2. This study investigates people's concerns about AI replacing their roles and their willingness to embrace these technologies, focusing on traditional predictors of fear and adoption: income, education, and age.

Methods & Data

A representative survey of the adult (18+) Jewish population in Israel was conducted (n=502) via an internet panel (iPanel) in the beginning of 2023. It was comprised of demographic and perspectives on AI technologies questions.

Results

Results indicate a significant negative correlation between income, education, and age with fears of AI replacing jobs (rs = -.179, p < .001; rs = -.108, p < .01; rs = -.096, p < .05). Additionally, a borderline significant positive correlation between willingness to adopt AI models and education (rs = .071, p = .055) and a significant negative correlation with age (rs = -.088, p < .05) were found. No correlation was observed between income and the willingness to adopt these technologies (rs = .019, p > .05).

Added Value

Notably, this research reveals a unique finding; Contrary to previous studies showing negative correlations between fear of technology and income or education, the fear of adopting new technologies is inversely related to age. As people grow older, their fear of adopting technology diminishes, likely because these tools offer a user-friendly interface resembling existing chat bots, requiring no new technological literacy. Another possible explanation is that the respondents feel secure in their workplace positions regardless to the new technologies.

Moreover, the lack of a correlation between income and willingness to adopt may stem from the low (sometimes free) cost associated with these technologies.

In an era of rapid AI development and integration into daily life, studies like this one hold significance in understanding public sentiments surrounding these tools and their implications for personal and professional life.

Relevance & Research Question: The research addresses a significant question in the era of advanced digital technology: "Are humans ready to detect AI-generated content?" This question is pivotal as it explores human perception and understanding in the face of rapidly evolving AI capabilities in times of deep fakes on all media platforms.

Methods & Data:

The empirical approach is using a digital interview with n>1000 Germans exposed to a variety of AI-generated and human-created content. In three categories (pictures, audio, videos) the participants were asked to identify the source of each piece of content, whether it was produced by AI or by a human..The content itself was created using AI Tools and stock content sources. The questionnaire is using implicit measurement and pairwise comparisons using the Analytic Hierarchy Process (AHP) methodology.

Results: The findings reveal intriguing insights into the human ability to discern AI-generated content. A significant proportion of participants are heavily challenged in correctly identifying the nature of the content, with varying degrees of accuracy across different types of media. These results highlight the sophistication of current AI technology in mimicking human creativity and the challenges faced by individuals in distinguishing between the two.

Added Value: This study adds substantial value to the discourse on AI and human interaction. It provides empirical evidence on the current state of human perception regarding AI-generated content in Germany, offering a foundation for further research in this area. The findings have implications for fields ranging from digital media and communication to AI ethics and policy-making, emphasizing the need for increased awareness and understanding of AI capabilities among the general public.

Relevance & Research Question
The use of various AI technologies in market research has increased significantly in recent years, and 2023 was a special year: industry publications clearly show that since the beginning of the year, large language models have also been used and new application areas have been tested. These new models are often described as game changers, particularly in qualitative research and analysis. However, there has been little cross-industry sharing of lessons learned. There is a limited understanding of how qualitative researchers use and experience these technologies in their day-to-day work, and how their work may change as a result. Our study aims to fill these gaps by collecting experiences and identifying concerns and challenges. We want to find out what qualitative researchers are actually doing after this year and how the sector has evolved. The aim of our study is to learn what experiences have been gathered so far and what methodological implications, expectations, challenges and opportunities exist.
Methods & Data
German-speaking qualitative researchers in the fields of market, social and UX research are invited to take part in the study. The survey consists of open and closed questions to capture different perspectives on the topic and takes approximately 7 minutes to complete. Questions cover experiences, methods, workflows and the real benefits of AI. Participants will answer completely anonymously so that experiences can be shared openly. Invitations will be sent out via newsletters, social media and industry media to reach as wide an audience as possible.
Results
The results are not yet available, but we will be able to present them at the conference. The data will be collected in December and January.
Added Value
The use of AI poses several challenges for our industry. Sharing experiences is essential to properly assess the potential and develop common standards. We will make the results available to interested parties and communicate them through a variety of channels to encourage a dialogue within the industry.

Relevance & Research Question
News sharing on digital platforms is a crucial activity that determines the digital spaces millions of users navigate. Yet, we know little about general patterns of news sharing – previous studies have focused on sharing of misinformation or specific/partisan outlets. To address this gap, we utilize a combination of three data sources to elucidate the extent to which sharing patterns of certain political user groups consist of specific outlets/topics/articles or have unknown diversity. Which types of news are shared in different political regions of Twitter? Are there news that are shared across the political spectrum?
Methods & Data

We combine multiple data sources via state-of-the-art network embedding methods and automated text analysis:

- we collected all tweets which contained a link to one of 26 legacy of alternative news outlets for March/2023 (2.5M tweets).
- we crawled the full texts of the articles if available (30K texts); articles were assigned topics with a paragraph-based BERTopic model.
- we collected the follower network of German MPs; we embedded all followers and MPs in a latent political space using correspondence analysis; CA reveals two clearly interpretable dimensions: one shows a clear distinction between AfD and MPs of all other parties; in the other dimension, all parties except AfD are arranged on a left-right axis.

Results
We investigate which types of articles are shared in which political regions of the latent space. We observe interesting, partly counterintuitive sharing patterns: Left-leaning outlets are shared by users in different political regions if the topic serves their political cause (qualitative example: an article of Die Zeit critical of working conditions at Deutsche Bahn was shared mostly by users following AfD or CDU/FDP politicians). On the other hand, soft/non-political news seem to be shared only by users in the 'mainstream' political region of the network (example: article on Lena Meyer-Landrut (Bild-Zeitung) with thousands of shares was not shared a single time by AfD followers). We explore these patterns systematically.
Added Value
We use digital trace data from a broad selection of news outlets. This is one of the first works that combine network embeddings with automated full-text analysis of news.

Relevance & Research Question

Facebook allows for direct communication with voters in the electorates. An issue that is divisive or polarizing on social media and political discourse is migration. This raises the guiding research question, of whether MPs who have positive or negative attitudes toward migration are more likely to speak in parliament on the issue or post about it on Facebook.
Methods & Data

This study compares the classical form of political speeches in parliament with social media communication on Facebook by members of parliament of the 18th German Bundestag (2013-2017). While prior studies compared political speech in parliamentary speeches and on social media focused on Twitter messages, this study uses a unique data set linking parliamentary speeches with election data, a candidate survey (GLES), and MPs’ social media communication on Facebook. The linked data allows to control for a number of candidate characteristics and test the influence of party or migration-attitudes on speaking and posting behaviour.

The first part of the analysis examines factors associated with general political communication activity in parliament and on Facebook and deploys a generalized linear quasi-Poisson mode, whilst the second part identifies migration-related speeches and posting using a dictionary approach and also analyses the association with candidate characteristics in a quasi-Poisson model.

Results

The first part of the analysis finds that party differences and candidacy play a role in speech activity, whereas being from a ’left-centrist’ party (DIE LINKE, SPD, GRÜNE) is positively associated with the number of Facebook messages issued by MPs.

The second part focuses on migration-related communication activity. Against the expectation that MPs with negative migration stances might have used Facebook more intensively to post about migration, the findings indicate that MPs who are in favour of migration were more likely to speak about migration-related issues in parliament and post about it on Facebook.
Added Value
The study uses a unique linked data set combining candidate studies with social media data and parliamentary speech data. The analysis could be improved by using contemporary large language models instead of a dictionary approach and the author would like to discuss added value with fellow conference attendees.

Relevance & Research Question

The emergence of social media has transformed the way people consume and share information. As such platforms widely lack mechanisms to ensure content quality, their increasing popularity has raised concerns about the spread of fake news and conspiracy beliefs – with potentially harmful effects on public opinion and social cohesion.

Our research aims to understand the underlying mechanisms of media perception and sharing behaviour when people are confronted with factual vs conspiracy-based media contents. Under which circumstances do people believe in a media content? Do traditional indicators of quality matter? Are pre-existing views more important than quality (confirmation bias)? How is perceived credibility linked to sharing behaviour?

Methods & Data

To empirically assess these questions, we administered a survey experiment to a general population sample in Germany via Bilendi in August 2023. As respondents with a general susceptibility to conspiracy beliefs are of major substantive interest, we made use of responses from a previous survey to oversample “conspiracy thinkers”.

Respondents were asked to evaluate the credibility of different media contents related to three vividly debated topics: vaccines against Covid-19, the climate crisis, and the Ukraine war. We analyze these evaluations regarding the objective quality of the content (measured by author identity and data source), its visual layout (newspaper vs tweet), and previous respondent beliefs on the respective topic to measure confirmation bias.

Results

Our findings suggest that the inclination to confirm pre-existing beliefs is the most important predictor for believing a media content, irrespective of its objective quality. This general tendency applies to both, the mainstream society and “conspiracy thinkers”. However, according to self-reports, the latter group is much more likely to share media contents they believe in.

Added Value

Methodologically, we use an interesting survey experiment that allows us to vary opinion (in)consistency and objective quality of media contents simultaneously, meaning that we can estimate the relative effect of these features on the credibility of media contents. We provide insights into the underlying mechanisms of the often debated spread of conspiracy beliefs through online platforms, with their practical implications for public opinion formation.

Relevance & Research Question
This study addresses a pressing concern in the digital age: the evolution of (online) political participation among young adults. As digital platforms reshape how society engages with politics, traditional definitions and measurements of political involvement require reassessment. The research seeks to unravel the perceived dichotomy between declining conventional political activities and the burgeoning new forms of engagement in digital spaces. Specifically, our research questions aim to identify and comprehend the spectrum of political participation online and offline among young adults, understand topic-centric engagements, and analyze how participation behaviors differ based on factors like education and digital service utilization. Ultimately, by gauging the behaviors of young adults in the realm of political engagement, this research contributes to both the refinement of existing definitions of political participation and the debate on youth's political engagement trajectory in contemporary settings.
Methods & Data
We will conduct an online survey of 16–29-year-olds in December 2023. The respondents for this survey will be recruited via Meta advertisements.
Results
Since the survey will take place in December 2023, nothing can be said about the results at this point. The results will be available in early February 2024.
Added Value
This study delves into the evolving definitions of political participation and offers methodological insights. It, therefore, explores what can be seen as political participation from the new possibilities digital space offers. Using both closed and open-ended survey questions, we aim to capture a broader spectrum of (online) political participation, potentially filling some gaps in conventional survey techniques. This approach allows us a more comprehensive understanding of the subject. Additionally, we are working to adjust and propose survey items that reflect current (online) political participation patterns. Through this, our research provides a clearer picture of young adults' political engagement and suggests ways to improve data collection for future research. Finally, our study also provides insight into the extent to which Meta advertisements are suitable for recruiting young people into surveys.

Relevance & Research Question:

Within the backdrop of Israel's turbulent 2022 elections, the fifth round of elections within three years, This study delves into the complex interplay between political trust, efficacy, and engagement. It seeks to unravel how individuals' trust in politicians and the political system, coupled with their sense of political efficacy, influences their online and offline engagement in the political process. The research question focuses on identifying the specific predictors of political engagement in a context characterized by political unpredictability and frequent elections.

Methods & Data:

The study analyzes a representative survey of 530 Israeli respondents during the 2022 Israeli election period. The research evaluates the influence of various variables. These include trust in politicians, the political system, and political efficacy in online and offline political engagement. The analysis focuses on the differentiation between online engagement, such as social media activity, and offline engagement, like attending rallies or voting.

Results:

Statistical analysis reveals a robust correlation between political efficacy and both forms of political engagement (r = .62 for online, r = .57 for offline, p < .01). Trust in the political system emerges as a significant predictor of offline engagement (β = .36, p < .01), while trust in politicians is more strongly associated with online engagement (β = .41, p < .01). Notably, a mediation analysis indicates that political efficacy serves as a mediator in the relationship between trust in politicians and online engagement (indirect effect = 0.15, 95% CI [0.07, 0.24], p < .01). In contrast, such mediating effects between system trust and offline engagement are not observed.

Added Value:

By examining the nuanced factors influencing political engagement during political uncertainty, this study offers new insights into the differentiated impact of trust in politicians and the political system. It underscores the distinct psychological pathways that drive online and offline political engagement, enhancing our understanding of citizen behavior in democracies facing political instability. These findings have critical implications for political strategists, policymakers, and scholars seeking to foster civic engagement in similar contexts.

Relevance & Research Question:
The recent development of large language models (LLMs) has spurred discussions about whether these models might provide a novel method of collecting public opinion data. As LLMs are trained on large amounts of internet data, potentially reflecting attitudes and behaviors prevalent in the population, LLM-generated “synthetic samples” could complement or replace traditional surveys. Several mostly US-based studies have prompted LLMs to mimic survey respondents, finding that the responses closely match the survey data. However, the prevalence of native-language training data, structural differences between the population reflected therein and the general population, and the relationship between a country’s socio-political structure and public opinion, might affect the generalizability of such findings. Therefore, we ask: To what extent can LLMs estimate public opinion in Germany?
Methods & Data:
We use the example of vote choice as an outcome of interest in public opinion. To generate a “synthetic sample” of the voting-eligible population in Germany, we create personas matching the individual characteristics of the 2017 German Longitudinal Election Study respondents. Prompting GPT-3.5 with each persona, we ask the LLM to predict each respondents’ vote choice. We examine how the average party vote shares obtained through GPT-3.5 compare to the survey-based estimates, assess whether GPT-3.5 is able to make accurate estimates for different population subgroups, and compare the determinants of voting behavior between the two data sources.
Results:
Based on our prompt design and model configuration, we find that GPT-3.5 does not accurately predict citizens’ vote choice, exhibiting a bias towards the Left and Green parties on aggregate, and making better predictions for more “typical” voter subgroups, such as political partisans. Regarding the determinants of its predictions, it tends to miss out on the multifaceted factors that sway individual voter choices.
Added Value:
By examining the prediction of voting behavior using LLMS in a new context, our study contributes to the growing body of research about the conditions under which LLMs can be leveraged for studying public opinion. The findings underscore the limitation of applying LLMs for public opinion estimation without accounting for the biases and potential limitations in their training data.

Relevance & Research Question
Since the launch of ChatGPT in November 2022, large language models (LLMs) have been the talk of the town. LLMs are artificial intelligence systems that are trained to understand and generate human language based on huge data sets. In all areas where language data play a central role, they have great potential to become part of a researcher’s methodological toolbox. One of these areas is the cognitive pretesting of questionnaires. We identify three tasks where LLMs can augment current cognitive pretesting procedures and potentially render them more effective and objective: (1) identifying potential problems of draft survey questions prior to cognitive testing, (2) suggesting cognitive probes to test draft survey questions, and (3) simulating or predicting respondents’ answers to these probes (i.e., generating ‘synthetic samples'). In this case study, we examine how well ChatGPT performs these tasks and to what extent it can improve current pretesting procedures.
Methods & Data
We conducted a cognitive interviewing study with 24 respondents, testing four versions of a survey question on children’s activity levels. Half of the respondents were parents of children aged 3 to 15 years, the other half were adolescents aged 11 to 17 years. In parallel to applying our common pretesting procedures, we prompted ChatGPT 3.5 to perform the three tasks above and analyzed similarities and differences in the outcomes of the LLM and humans.
Results
With respect to tasks (1) and (2), ChatGPT identified some question problems and probes that were not anticipated by humans, but it also missed important problems and probes identified by human experts. With respect to task (3), the answers generated by ChatGPT were characterized by a relatively low variation between individuals with very different characteristics (i.e., gender, age, education) and the reproduction of gender stereotypes regarding the activities of boys and girls. All in all, they only marginally matched the answers of the actual respondents.
Added Value
To our knowledge, this is one of the first studies examining how LLMs can be incorporated into the toolkit of survey methodologists, particularly in the area of cognitive pretesting.

Relevance & Research Question: The recent advances and availability of large language models (LLMs), such as OpenAI’s GPT, have created new opportunities for research in the social and behavioral sciences. Questionnaire development and evaluation is a potential area where researchers can benefit from LLMs: Trained on large amounts of text data, LLMs might serve as an easy-to-implement and inexpensive method for both assessing and improving the design of survey questions, by detecting problems in question wordings and suggesting alternative versions. In this paper, we examine to what extent GPT-4 can be leveraged for questionnaire design and evaluation by addressing the following research questions: (1) How accurately can GPT-4 detect problematic linguistic features in survey questions compared to existing computer-based evaluation methods? (2) To what extent can GPT-4 improve the design of survey questions?

Methods & Data: We prompt GPT-4 with a set of survey questions and ask to identify features in the question stem or the response options that can potentially cause comprehension problems, such as vague terms or a complex syntax. For each survey question, we also ask the LLM to suggest an improved version. To compare the LLM-based results with an existing computer-based survey evaluation method, we use the Question Understanding Aid (QUAID; Graesser et al. 2006) that rates survey questions on different categories of comprehension problems. Based on an expert review among researchers with a PhD in survey methodology, we assess the accuracy of the GPT-4- and QUAID-based evaluation methods in identifying problematic features in the survey questions. We also ask the expert reviewers to evaluate the quality of the new question versions developed by GPT-4 compared to their original versions.

Results: We compare both evaluation methods with regard to the number of problematic question features identified, building upon the five categories used in QUAID: (1) unfamiliar technical terms, (2) vague or imprecise relative terms, (3) vague or ambiguous noun phrases, (4) complex syntax, and (5) working memory overload.

Added Value: The results from this paper provide novel evidence on the usefulness of LLMs for facilitating survey data collection.