Methodology of Longitudinal Surveys II


Does the laptop always help? Non-compliance and interviewer effects in cognitive tests

My Programme   Add 
Type:Contributed Paper
Jul 25, 11:30
  • Nadine Geiger - Munich Graduate School of Economics

Individual performance in a cognitive test does not only reflect inherent ability but also contextual circumstances and differences in interviewers’ styles. One way to scale down the impact of interviewers is to shift administration of a cognitive test to a technical device. However, this approach can introduce mode effects, as some interviewers may deviate from the study protocol and administer the test without the help of the technical device for some but not all respondents.

In this work, we study test scores of the word recall test, collected in Understanding Society. Under the default procedure, respondents hear ten words from the laptop of the interviewer. Nevertheless, for about 20% of respondents in our sample the interviewer instead reads the words herself, leading to the cognitive test being effectively administered in two distinct modes.

We aim to answer three questions. Firstly, we ask which determinants drive imperfect compliance with the study protocol. Next, we turn to the question of what drives the difference in performance between the two modes. Finally, we seek to answer whether administration via the computer successfully reduces interviewer effects.

We find that hearing the words from the interviewer is predicted by respondent characteristics, which have been associated with the propensity to use technologies.  Moreover, the share of respondents who hear the words from the interviewer varies greatly across interviewers. Selection effects appear to be the main driver of performance differences in our setting. Those respondents who are read the words to by the interviewer, perform significantly worse not only in the word recall test but also in other cognitive tests. Finally, our results suggest that the use of the laptop does indeed reduce interviewer effects. Therefore, the use of laptops in administration of the word recall test may be preferable, despite the problem of non-compliance.


Latest tweets from @MOLS2Essex. Follow the conversation at #MOLS2

Today is the last day for submitting abstracts on longitudinal survey methods for the special edition of Longitudin… https://t.co/MYNRQh3bpE
2 months 3 weeks ago
4 days left! If you want to submit a paper for the Special Issue of Longitudinal and Life Course Studies you have u… https://t.co/F35Nzqp7U1
2 months 3 weeks ago
If you want to submit a paper for the Special Issue of Longitudinal and Life Course Studies you have until 24 Sept.… https://t.co/y0chY6856z
3 months 2 days ago
RT : Call for papers: Understanding social dynamics: 20 years of the Swiss Household Panel. The Swiss Journal of Sociolo… https://t.co/k2DJvFDA49
3 months 1 week ago