Methodology of Longitudinal Surveys II

This site is now archived.

Measuring cognition in a multi-mode context

Type:Monograph Paper
Jul 25, 13:45
  • Mary Beth Ofstedal - Survey Research Center, University of Michigan
  • Colleen McClain - Survey Research Center, University of Michigan
  • Mick P. Couper - Survey Research Center, University of Michigan

As longitudinal surveys increasingly integrate alternate modes (telephone, web, mail) to reduce costs, maintaining comparability of measures across modes presents operational and substantive challenges.  The measurement of cognitive ability, common in longitudinal surveys of the general population, as well as specific studies of aging, is particularly challenging for a multi-mode design.  Some of the standard tests that are designed for self-administration (e.g., tests that use visual stimuli) may be difficult to adapt for telephone administration and those used in interviewer-administered surveys (e.g., immediate and delayed word recall tests) can be difficult to adapt for web.  Mode differences may be exacerbated for older respondents who are less technologically sophisticated or have cognitive impairments that could affect the quality and completeness of the data.  Despite these challenges and the movement of many longitudinal studies to alternate modes, few multi-mode comparisons of cognition exist in the current literature.

To address this gap, we will first provide a review of the literature and current practice relating to the measurement of cognition in longitudinal surveys.  We’ll then present empirical results from mode comparisons of cognitive performance using data from the Health and Retirement Study (HRS).  The biennial core HRS interview has been dual mode (telephone and face-to-face) for most of its history, and plans to offer web as a third mode in the 2018 wave.  HRS has also conducted periodic web surveys between core waves with some overlap of content. We investigate mode differences for several cognition measures (word recall/recognition, verbal analogies, number series, Serial 7s, and self-rated memory) that were included in the 2013 HRS Internet Survey and either/both of the adjacent core waves. We compare levels of cognitive performance and associations between tests, known predictors, and outcomes across modes (telephone, face-to-face, web); assess differences in data quality using paradata; and identify tests for which mixed-mode approaches may be problematic. 

Preliminary results show systematic differences in levels of cognitive performance across all measures, with respondents performing better on the web than either telephone or face-to-face modes and slightly better on telephone than face-to-face.  In addition, we also find mode differences in associations between cognition and certain known predictors and outcomes.  This raises a concern that substantive conclusions about cognition may be sensitive to survey mode.  Results of this study will help inform the development of cognition measures for the HRS moving forward and provide guidance for design and research on cognition in mixed-mode contexts. 


Latest tweets from @MOLS2Essex. Follow the conversation at #MOLS2

Congratulations & happy new year 🌹
15 hours 12 min ago
Quick off the mark! I haven't finished dig… https://t.co/eVfyoY8Gxy
18 hours 13 min ago
My first publication of 2019!! Assessing the reliability of longitudinal study data Findings from… https://t.co/UZnHihQjlG
18 hours 18 min ago
Today is the last day for submitting abstracts on longitudinal survey methods for the special edition of Longitudin… https://t.co/MYNRQh3bpE
3 months 1 week ago