Relevance & Research Question
Longitudinal- and panel studies are based around the repeated interviewing of the same respondents. However, all panel studies are confronted with the loss of respondents who stop participating over time, i.e., panel attrition. Few studies have had the opportunity to observe attrition in the context of a panel study that features both frequent interviews and has been conducted over a long period of time and therefore offers many data points. In this contribution we investigate attrition rates over time and changes in sample composition for three samples in a probability-based online panel over a period of eleven years and 68 panel waves.
Methods & Data
We analyze participation data and respondent characteristics (e.g., socio-demographics) from 68 waves of the German Internet Panel (GIP) covering a time period from September 2012 to present. The GIP is the longest-running probability-based online panel in Germany and allows us to observe respondents from three recruitments samples drawn in 2012, 2014, and 2018, respectively.
Results
Preliminary results indicate a high attrition rate over the first panel waves and a slower yet steady loss of respondents in the long term. On average, about 25% of recruited respondents were lost over the first year. The average annual attrition rate across all samples then falls to around 10% for the second and third year and a single-digit percentage for every year after that. Over time, a larger proportion of respondents in the remaining sample are married and hold academic degrees. The sample also slightly shifts towards a higher proportion of female respondents and persons living in single households. The proportion of respondents living in east or west Germany, their mean year of birth and employment status remain relatively unchanged.
Added Value

For longitudinal research and panel practitioners, it is important to understand how much attrition to expect over time and which groups of respondents are especially at risk. These insights aid in guiding researchers in determining how many respondents to recruit, when to refresh the sample and which respondents should be especially targeted with strategies for improving recruitment rates or reducing attrition.

Relevance & Research Question

The steady trend in declining response rates and the rise of non-probability surveys makes it increasingly important to conduct nonresponse and selection bias analyses for social science surveys or conduct robustness checks to evaluate if the results are robust across population subgroups. Although this is important for any research project, it can be very time-consuming. The new R-Package SampcompR was created to provide easy-to-apply functions for those analyses and make it easier for any researcher to compare their survey against benchmark data for bias estimation on a univariate, bivariate, and multivariate level.

Methods & Data

To illustrate the functions of the package, we compare three web surveys conducted in Africa in March 2023 using Meta advertisements as a recruitment method (Ghana n = 527, Kenya n = 2,843, and South Africa n =313) to benchmarks from the cross-national Demographics and Health Survey (DHS). The benchmarks will be socio-demographics and health-related variables such as HIV knowledge. In univariate comparison, bias is measured as the relative bias for every variable and, on an aggregated level, the average absolute relative bias (AARB). In bivariate estimation, we compare Pearson’s r values against each other, and in multivariate comparison, different regression models are compared against each other.

Results

Our poster will show examples of output from the package, including visualizations and tables for each comparison level. While the focus will be on figures, tables can also be useful for documentation and more detailed inspection. As to the specific content of our example, we will see that the social media surveys show a high amount of bias on a univariate level. In contrast, the bias is less pronounced on a bivariate or multivariate level. We will also report country differences in sample accuracy.

Added Value

Our R-Package will provide an easy-to-use toolkit to perform bias analyses and survey comparisons and, therefore, will be a valuable tool in the social research workflow. Using the same or similar procedures and visualizations for the various comparisons will increase comparability and standardization. The visualization is based on the commonly used R-package ggplot2, making it easily customizable.