Methodology of Longitudinal Surveys II

This site is now archived.

Does the laptop always help? Non-compliance and interviewer effects in cognitive tests

Type:Contributed Paper
Jul 25, 11:30
  • Nadine Geiger - Munich Graduate School of Economics

Individual performance in a cognitive test does not only reflect inherent ability but also contextual circumstances and differences in interviewers’ styles. One way to scale down the impact of interviewers is to shift administration of a cognitive test to a technical device. However, this approach can introduce mode effects, as some interviewers may deviate from the study protocol and administer the test without the help of the technical device for some but not all respondents.

In this work, we study test scores of the word recall test, collected in Understanding Society. Under the default procedure, respondents hear ten words from the laptop of the interviewer. Nevertheless, for about 20% of respondents in our sample the interviewer instead reads the words herself, leading to the cognitive test being effectively administered in two distinct modes.

We aim to answer three questions. Firstly, we ask which determinants drive imperfect compliance with the study protocol. Next, we turn to the question of what drives the difference in performance between the two modes. Finally, we seek to answer whether administration via the computer successfully reduces interviewer effects.

We find that hearing the words from the interviewer is predicted by respondent characteristics, which have been associated with the propensity to use technologies.  Moreover, the share of respondents who hear the words from the interviewer varies greatly across interviewers. Selection effects appear to be the main driver of performance differences in our setting. Those respondents who are read the words to by the interviewer, perform significantly worse not only in the word recall test but also in other cognitive tests. Finally, our results suggest that the use of the laptop does indeed reduce interviewer effects. Therefore, the use of laptops in administration of the word recall test may be preferable, despite the problem of non-compliance.


Latest tweets from @MOLS2Essex. Follow the conversation at #MOLS2

Congratulations & happy new year 🌹
15 hours 12 min ago
Quick off the mark! I haven't finished dig… https://t.co/eVfyoY8Gxy
18 hours 13 min ago
My first publication of 2019!! Assessing the reliability of longitudinal study data Findings from… https://t.co/UZnHihQjlG
18 hours 18 min ago
Today is the last day for submitting abstracts on longitudinal survey methods for the special edition of Longitudin… https://t.co/MYNRQh3bpE
3 months 1 week ago