Relevance & Research Question
The steady trend in declining response rates and the rise of non-probability surveys makes it increasingly important to conduct nonresponse and selection bias analyses for social science surveys or conduct robustness checks to evaluate if the results are robust across population subgroups. Although this is important for any research project, it can be very time-consuming. The new R-Package SampcompR was created to provide easy-to-apply functions for those analyses and make it easier for any researcher to compare their survey against benchmark data for bias estimation on a univariate, bivariate, and multivariate level.
Methods & Data
To illustrate the functions of the package, we compare three web surveys conducted in Africa in March 2023 using Meta advertisements as a recruitment method (Ghana n = 527, Kenya n = 2,843, and South Africa n =313) to benchmarks from the cross-national Demographics and Health Survey (DHS). The benchmarks will be socio-demographics and health-related variables such as HIV knowledge. In univariate comparison, bias is measured as the relative bias for every variable and, on an aggregated level, the average absolute relative bias (AARB). In bivariate estimation, we compare Pearson’s r values against each other, and in multivariate comparison, different regression models are compared against each other.
Results
Our poster will show examples of output from the package, including visualizations and tables for each comparison level. While the focus will be on figures, tables can also be useful for documentation and more detailed inspection. As to the specific content of our example, we will see that the social media surveys show a high amount of bias on a univariate level. In contrast, the bias is less pronounced on a bivariate or multivariate level. We will also report country differences in sample accuracy.
Added Value
Our R-Package will provide an easy-to-use toolkit to perform bias analyses and survey comparisons and, therefore, will be a valuable tool in the social research workflow. Using the same or similar procedures and visualizations for the various comparisons will increase comparability and standardization. The visualization is based on the commonly used R-package ggplot2, making it easily customizable.