Assessing discontinuities and rotation group bias in rotating panel and longitudinal designs
Jul 25, 15:45
An important quality aspect of longitudinal and rotating panel surveys is that they produce consistent series giving accurate estimates for period-to-period changes and flows. Periodically, however, elements of the survey process must be updated. These redesigns change the balance of the different non-sampling errors and generally result in systematic differences in the survey outcomes, called discontinuities (Van den Brakel et al. 2008). We review approaches to modelling and adjusting for discontinuities generally, and specifically use multivariate state-space models in rotating panel Labour Force Survey designs to produce consistent series.
Most Labour Force Surveys use rotating panels, with samples observed multiple times using a fixed rotation pattern. For example in the Netherlands and the UK monthly samples are observed five times at quarterly intervals. We discuss two different sources of measurement bias in rotating panel surveys. The first is known as rotation group bias (RGB), and arises when there are systematic differences in the outcomes for the same period depending on how many times sampled units have previously been interviewed. In the Netherlands for example total unemployment is estimated systematically higher in the first wave. RGB is the net effect of different non sampling errors, induced by differences in data collection modes questionnaires, panel effects, selective non-response and panel attrition.
The second source of measurement bias comes from survey process redesigns resulting in discontinuities. They interrupt consistent series of sample estimates, so national statistical institutes (NSIs) are reluctant to modify survey processes. Periodic redesigns are nevertheless inevitable and to avoid confounding their effects with real period-to-period change it is important to quantify discontinuities, for example by conducting the old and new approach in parallel for a period or by using time series models.
We describe a multivariate state-space model (based on Pfeffermann 1991) that uses direct estimates for each wave separately as inputs. It accounts for RGB by modelling systematic differences between the series of direct estimates, and is extended to account for discontinuities due to survey process redesigns. Different possibilities to plan and implement a new process in a rotating panel or longitudinal survey are proposed, aiming to minimize disturbance to the regular time series. We also show how parallel run information can be combined with the state-space modelling approach. Results are illustrated using redesigns in the Dutch and UK LFSs.