Responsive designs in practice
Jul 27, 09:00
Responsive designs for data collection are one of the most promising theoretical approaches that attempt to counter falling response rates for surveys. Adapting the data collection strategy during the field period to incorporate available information about the expected probability of cooperation from the sample cases is expected to increase efficiency and reduce the cost of data collection. Responsive design is especially promising for panel surveys because (after the first wave) so much more information is available about the sampled units and their response patterns, compared to cross-sectional surveys, which can be used to mitigate attrition. However, while appealing in theory, in practice responsive designs pose a number of implementation challenges in face-to-face surveys. First, the design must consider both the predicted propensity to respond of each case and the overall sample composition in order to minimize biases from differential cooperation rates among subgroups. Second, the suggested workplan for the cases each interviewer is directed to work has to account for the geographical location of the respondents to ensure that the recommendations are feasible and optimize the interviewer’s travel schedule. Finally, instructions must be communicated in a timely way and accounting for the potential resistance of interviewers to follow instructions based on predictions made by the home office. In this paper, we discuss the experience of Westat in the design, development, implementation, and testing of a full responsive design model for the PATH Reliability study, a two-wave study on tobacco use. We focus on the analytical components of the design, including the predictive models for response propensity and sample balance, the optimal routing model to help interviewers plan the day. We discuss a set of experiments that we deployed to measure compliance with alternative presentations of the instructions and the total effect on operational costs of alternative sample collection designs.