Editable summary screens for improving income measurement in household surveys
Jul 27, 09:00
Income data collected as part of household surveys is critical for the study of material living standards. Survey respondents are known to misreport their income; although the types of error are not well documented. Against this backdrop, we experiment with the use of an Editable Summary Screen (ESS) during data collection to improve income data quality. We test two version of the ESS in a large scale panel survey - the Understanding Society Innovation panel. In both versions respondents report on a detailed set of income questions in an individual interview. In version A, respondents get an individual ESS (IESS) at the end of the interview. In version B, respondents are invited with their partner to take part in an additional “Benefit Unit” interview where they review their reports from their individual interviews together (BUESS).
We find that respondents revise their income in both versions of the ESS and across a wide range of income sources and in different directions. The extent and magnitude of the revisions are non-trivial. For example, nearly 17% of benefit units corrected their income in version A and the mean absolute correction was £830 per month. Moreover, the revisions are large enough to induce changes in some measures of income inequality. Thus we conclude that an ESS is an effective tool for reducing misreporting of income.
We also shed light on the types of reporting errors made by survey participants by classifying the revisions. We find that some errors that researchers worry about - slipped decimal places, joint receipt - are not very common. But simple mis-reporting of amounts and period codes is prevalent.
We do not find that one version of the ESS is clearly better than the other. The main advantage of the BUESS is that it can identify joint receipt problems, but our results indicate they are a relatively uncommon error. While the BUESS has a higher correction rate conditional on participants seeing the summary screen (22% vs 16%), it achieves a slightly lower unconditional correction rate (13% vs. 15%). This is because many respondents do not consent to take part in a joint interview with their partner. Thus the IESS improves data quality for a large share of the sample, whereas the quality improvements of the BUESS are arguably larger, but for a select group of consenters.