Relevance & Research Question

Smartphones have become promising tools for collecting digital behavioral, sensor, and survey data in the social sciences. However, the recruitment of study participants who are willing to install a smartphone app and fully participate throughout the study period remains a challenge. This paper reports on results from qualitative in-depth interviews to better understand the mechanisms underlying the decision to participate in smartphone app data collection. The study addresses the following research questions: (1) What difficulties and risks do people perceive in smartphone app data collection? (2) How do people perceive the collection of different forms of data, in particular digital behavioral, sensor, and survey data? (3) Under what conditions would people be more willing to participate and adhere in smartphone app data collection? (4) Which strategies for increasing participation and adherence might work best for whom?

Methods & Data

In-depth interviews with n = 30 participants with different sociodemographic characteristics (age, gender, education) and types of smartphone use are conducted in November/December 2025. Participants are presented with a hypothetical research app that involves the collection of survey, GPS, Internet browsing, and app usage data. They are subsequently asked a series of open-ended questions and probes based on a semi-structured discussion guide. The interview data are analyzed using a thematic analysis approach.

Results

The study will examine reasons for and against participating in smartphone app data collection and the extent to which these vary across the different data components. The potential reasons include those related to the study design, its perceived relevance to science and society, participants’ interest in the study topic, and their concerns about privacy and data security. In addition, it will be investigated how different study designs, such as different monetary and non-monetary incentive structures, affect people’s willingness to participate, and for which population subgroups these might be more effective.

Added Value

The study provides insights into people’s decision-making processes regarding their participation in smartphone app data collection and aims to inform researchers about how to best design and implement smartphone-based studies.

Relevance & Research Question

Household budget surveys provide an excellent opportunity for the implementation of smart services. The response burden for reporting expenditures in a questionnaire is high and is likely to result in underreporting. Over the past years, Statistics Netherlands has worked on implementing smart features in the household budget survey, aiming to reduce response burden and increase data quality. The fieldwork with the app will start at the end of 2025.

In this presentation, the main findings and efforts of the qualitative and quantitative tests done in 2025 will be presented.

Methods & Data

Five tests were conducted using the StatNL app for the household budget survey: two usability tests, a field test with a fresh sample and two internal StatNL tests with the mobile application for the household budget survey. The usability tests included cognitive interviews and talk-aloud methods to gather in-depth qualitative insights in the entire process from invitation letter to final participation using the app. In the field test, we experimented with different interviewer roles and gathered data in a realistic setting to gain quantitative insights in the responses, which was enriched with qualitative evaluation questionnaires and telephone interviews. The two internal tests each served a different goal: with the first, receipts were collected for training and validating the machine learning algorithm, whereas the second set-up quick iterative evaluation rounds of the final redesigns of the app.

Results
A timeline of the different projects that were completed before the implementation will be presented and key findings will be highlighted. The qualitative tests demonstrated over and over that abilities of respondents vary widely. With the results of the field and internal tests the application and algorithms were further developed and improved.

Added Value
In this presentation we outline the work and testing needed for successful implementation of an app for the household budget survey. We demonstrate the need to engage a multidisciplinary team of experts to cover all disciplines involved in such an innovative project.

Relevance & Research Question
Providing participants with control over the data collection process has been linked to increased willingness to participate in app-based studies. As the mechanism for this relation is not yet well understood, it is unclear under which conditions and for whom this effect holds. We use the Leverage Saliency Theory of survey participation (LST; Groves et al. 2000) which states that the interplay of personal leverages and the saliency of study characteristics influence whether someone participate in a study to answer the research question: ‘How does control over data collection influence willingness to participate in app-based data collection?’.
Methods & Data
In December 2024, we conducted a vignette study in the Centerpanel, a longitudinal panel of the general population in the Netherlands. In a vignette study (n=1200, response rate: 86.8%), participants rated whether several study characteristics influenced their decision to participate in app-based data collection. Participants were presented with a hypothetical study description and mock-up research app screen, and indicated how willing they would be to download this app. Participants were randomly assigned to be presented with either 7 control and transparency features in the app description and app mock-up screen or no control features in the presented app at all.
Results
Participants generally rated control features as important in their decision to participate in app-based studies. The data retention period, control over third-party sharing, and the study’s relevance are rated as most important. When controlling for speeding and attention check, we find a main effect of control over data collection on willingness to download the app (t = 3.85, p < .001). However, we find no significant interactions between experimental conditions and rated importance of control features. Therefore, we find no support for the LST.
Added Value
Our findings show what type of control and transparency features people find most important when participating in app-based studies, and replicates the effect of control on willingness to participate. Therefore, it can be valuable for researchers to invest time and resources in developing control functionalities when conducting app-based research, as this might cause participants to be more willing to adopt a research app.