Thank you, sample members! Reframing a prenotification message to encourage response in a longitudinal survey
Jul 25, 13:45
High response rates over time are critical for the integrity of longitudinal surveys as previous participation is an indicator of future participation. Additionally, data across waves are often analyzed together, so nonresponse to any particular wave can produce large amounts of missing data for individual sample members, leading to poor survey estimates over time. Thus, encouraging participation helps ensure the success of longitudinal surveys. A best practice for encouraging participation is to send sample members a prenotification letter about an upcoming survey; in longitudinal surveys, this also serves to remind sample members of their previous contributions to the study.
The National Longitudinal Study of Adolescent to Adult Heath (Add Health) is now in its fifth wave of data collection. The study has been conducted over the course of more than 20 years. Sample members, who were teens during the first interview in 1994-95, are now in their early 30s to early 40s. As 20 years was a milestone for the study, we wanted to test whether increasing the saliency of the longevity of sample members’ participation would increase current participation. We tested a prenotice in a unique form—a greeting card—which thanked sample members for their ongoing contributions to the study over the years.
Using a 2x2 factorial design on a randomly-selected subset of the sample (n=2,716), we tested the effectiveness of the prenotice thank-you greeting card against our current method for boosting participation rates ($10 prepaid cash incentive included in a more traditional letter). Sample members were randomly assigned to one of four groups: (1) control, i.e., no prenotice or pre-incentive, (2) prenotice only, (3) pre-incentive only, and (4) both prenotice and pre-incentive.
As expected, participation rates were significantly lower for the control group. Among the treatment conditions, the prenotice produced similar participation rates to the pre-incentive, with only a slight increase in response for the combination of the two factors. Based on these findings, we omitted the pre-incentive and used only the prenotice thank-you card for a subsequent subset of sample members (n=7,631)—a significant cost savings. Our findings suggest that long-term longitudinal study participants evaluate researchers’ gratitude as a type of benefit on par with monetary incentives—offering researchers opportunities to reduce incentive costs. In the future, we seek to experimentally investigate whether the prenotice’s impact is attributed to its greeting card format, thank-you wording, or both.