A methodological guide to the analysis of panel conditioning effects
Jul 26, 11:00
Panel conditioning refers to the phenomenon whereby respondents' attitudes, behavior, reporting of behavior and/or knowledge are changed by repeated participation in a panel survey. Uncovering such effects in panel studies, however, is difficult due to three major methodological challenges that we discuss in detail in this paper.
First, researchers need to disentangle changes in behavior from changes in the reporting of behavior as panel conditioning may result in both, even at the same time and in opposite directions. Second, studies of panel conditioning usually aim to identify the causal effect of panel participation on the various changes in survey outcomes mentioned above. To identify such an effect, researchers need to come up with a suitable control group of people who have not been interviewed before. However, finding such control groups that allow the estimation of an unbiased panel conditioning effect is difficult. Researchers have come up with two approaches (within-subject and between-subject comparisons) that come with different strengths and weaknesses. Third, other sources of error in (panel) surveys complicate the identification of panel conditioning effects as they may be mistaken for panel conditioning if not properly accounted for. Error sources that can easily be mistaken for panel conditioning are, for example, initial nonresponse and panel attrition, mode or interviewer effects.
We review the three challenges mentioned above in detail and discuss the strengths and weaknesses of the various designs that researchers have developed to tackle them. We conclude with a practical guide to the analysis of conditioning effects in longitudinal surveys and a discussion of a future research agenda.