A method for optimizing data collection efficiency in an online panel: A case study
Jul 26, 13:45
New techniques to improve data collection have to be found. Online surveys are generally cheaper in terms of implementation and interventions during the data collection. However, little is known about the evolution of daily response rates in such surveys and about the impact of interventions such as sending reminders. This paper seeks to understand whether data collection efficiency can be optimized to increase response rates and save costs. To do so, we use data from the German Internet Panel, which is a probability-based online panel interviewing respondents every second month. We use data from the refreshment sample from 2013 until 2017, including a total of 17 waves.
First, we model the shape of the evolution of the daily response rate defined as the number of completed questionnaires in a given day divided by the total number of invited panelists. The results of a multi-level model with days clustered in waves show that the evolution is quadric. The daily response rate first decreases. This decrease slows down to be followed by a slight momentary increase in daily response rate around the sending of reminders.
Second, characteristics that can influence this shape are introduced in the model on the wave-level and day level. None of the wave-level characteristic affected the shape of the evolution of the daily response rate (e.g. weekday on which the wave started, mean respondent satisfaction with the previous wave, mean questionnaire length). At the day level, both the day of the week (Sunday, Monday, Tuesday) and the day a reminder was sent had a significant positive effect on the daily response rate. However, public holidays and school holidays had no effect.
Finally, we apply the shape of the daily response rate evolution to monitor panel wave 17. To do so, the shape of waves 1 to 16 is plotted together with its 95% confidence band resulting in a benchmark. The daily response rate of wave 17 is then plotted against the benchmark. We find several daily response rates to fall below the confidence band, showing that the days of sending reminders could be adjusted.
To conclude, modeling the daily response rate across waves of an online panel can inform survey conductors about the efficiency of their data collection. By monitoring daily response rates survey conductors can adapt the data collection process and hence, unit nonresponse reduction strategies might be more efficient in terms of response rates and/or costs.