Relevance & Research Question
Online research methods are imperative for collecting accurate data on behaviors and cognitions. However, when respondents answer sensitive questions, they tend to provide socially desirable answers - overreporting desirable norms (e.g., exercising), underreporting norm violations (e.g., prejudicial attitudes), or skipping items. Reducing this social desirability bias (SDB) is essential for improving data validity. To reduce desirable responding, we conducted two survey experiments employing face-saving strategies: survey formulations that soften or justify norm violations. We addressed two questions: (1) To what extent does the effectiveness of face-saving answer options depend on the question format?; (2) Do respondents attend to face-saving preambles, and does engagement reduce desirable responses?
Methods & Data
Two online survey experiments were conducted among residents in a large Dutch city (N study 1 = 529; study 2 data collection currently ongoing). Study 1 examined face-saving strategies across four question formats: open; yes-no; apply-does not apply, and; very often-never. Experimental manipulations varied (a) the presence of a face-saving preamble and (b) the availability of face-saving answer options. Sensitive items were about volunteering, disturbances, safety, and poverty. Reading time of question introductions was recorded to assess respondents’ attention. Study 2 replicated question formats but introduced new items and different operationalizations of face-saving answer options and question introduction engagement. Similar reductions in SDB are expected for study 2.
Results
In study 1, face-saving answer options consistently increased the selection of socially undesirable responses across all closed question formats and domains, indicating reduced bias. In contrast, preambles did not meaningfully shift response patterns, even when participants spent sufficient time reading them. Data collection for study 2 will be completed in December 2025.
Added Value
This study offers the first systematic comparison of face-saving strategies across multiple online question formats. By demonstrating that face-saving answer options reduce SDB - while preambles did not - study 1 clarified the mechanisms behind face-saving techniques and offered practical guidance for online survey design. The oral presentation will integrate findings from both experiments, highlighting how face-saving strategies can strengthen the validity of online surveys on sensitive topics and thereby advance online research methodologies.

Relevance & Research Question:

High rates of item nonresponse to questions on income and expenditures compromise data quality, leading to sampling bias and limiting the generalizability of findings. This study investigates the effectiveness of follow-up (probing) questions in reducing nonresponse to financial questions within a Russian context and identifies the profiles of non-respondents.

Methods & Data:

Using data from the 6th wave of the HSE University's "Economic Behavior of Households" survey (N=6000), the analysis employs a two-stage approach combining the Random Forest method with the Boruta algorithm and logistic regressions.

Results:

Results indicate that probing questions successfully converted 36-41% of initial nonresponses into substantive answers, decreasing the overall nonresponse rate from 6-17% to 4-10%. Key predictors of nonresponse were found to be unawareness of household expenditures (a 3- to 5-fold increase in odds), poor health, lack of savings, and residence in small towns. The technique proved most effective for respondents with lower education levels, no savings, and those from small towns.

Added Value:

The findings demonstrate that while probing is a valuable tool, its primary mechanism is reducing cognitive complexity rather than mitigating question sensitivity. Based on this evidence, the paper offers practical recommendations for improving the design of surveys that include financial questions.

Relevance & Research Question:

Careless or insufficient-effort responding (C/IER) is a major threat to data quality in online surveys. Existing detection approaches face substantial limitations: indicator-based methods (e.g., straightlining indices) require subjective threshold-setting, while model-based approaches rely on strong and often unrealistic assumptions and are difficult to implement in applied research.

Attention checks offer an objective-to-score and easy-to-use alternative. However, traditional attention checks, such as instructed-response items (Please select strongly agree) or bogus items (Orange is a fruit), suffer from limited validity, and high respondent reactivity (Daikeler et al., 2024; Gummer et al., 2021; Silber et al., 2022). This project addresses the lack of attention checks that would mimic ordinary questionnaire items to limit reactivity while reliably identifying inattentive respondents.

The central research question is: How can we design attention checks that outperform existing approaches in validity and non-reactivity?

Methods & Data:

First, 460 candidate ACES were developed across diverse domains (e.g., personality, technology, political attitudes), drawing on the concept of frequency/infrequency items (Kay & Saucier, 2023), and tested on 1498 respondents. Items were evaluated using distributional properties and socio-demographic invariance.

The selected ACES were validated in a between-subjects experiment (N = 880) comparing four conditions: (1) ACES, (2) IRIs, (3) bogus items, and (4) a no-check control group. The questionnaire included measures of perceived clarity and seriousness, as well as open-ended feedback on attention checks.

The findings were replicated in another survey (N = 1113) targeting less experienced online panel members. All studies used non-probability quota samples (age, gender, education) from Polish online research panels.

Results:

ACES showed stronger associations with independent indicators of inattentiveness (e.g., response times, straightlining indices) and demonstrated higher classification accuracy (careless-not careless) than IRIs and bogus items. Respondents generally did not recognize ACES as attention checks, despite being highly familiar with traditional checks.

Added Value:

This project delivers the first validated ACES set and provides empirical evidence that ACES improve detection of C/IER in comparison to other attention checks while minimizing respondent reactivity. This enables using ACES in probability samples or interviewer-based surveys where traditional attention checks were not used due to their coarseness and potential for reactivity.