Investigation of Ways to Handle Sampling Weights for Multilevel Model Analyses

Citation

Cai, Tianji (2013). Investigation of Ways to Handle Sampling Weights for Multilevel Model Analyses. Sociological Methodology. vol. 43 (1) pp. 178-219

Abstract

When analysts estimate a multilevel model using survey data, they often use weighted procedures using multilevel sampling weights to correct the effect of unequal probabilities of selection. This study addresses the impacts of including sampling weights and the consequences of ignoring them by assessing the performance of four approaches: the multilevel pseudo–maximum likelihood (MPML), the probability-weighted iterative generalized least squares (PWIGLS), the naive (ignoring sampling weights), and the sample distribution methods for a linear random-intercept model under a two-stage clustering sampling design. When inclusion probabilities are correlated with the values of outcome variable conditioning on the model covariates, the sampling design becomes informative. The results show that whether a sampling design is informative and at which stage of the sampling design it is informative have substantial impacts on the estimation. The results also show that the level of variation of sampling weights is correlated with the bias of estimates. A higher level of variation of sampling weights is associated with a higher level of bias when a sampling design is informative; however, under a noninformative design, the level of variation of sampling weights may not necessarily associate with biased results. Ignoring an informative sampling design at the first stage will result in biased estimates on the intercept and variance of random effect, whereas ignoring an informative sampling design at the second stage will lead to slightly underestimated fixed effects and residual variance, in addition to the biased estimates on the intercept and variance of random effect. Including the sampling weights as the hybrid methods (MPML and PWIGLS) may still produce biased estimates on the intercept and variance of random effect and slightly underestimated fixed effects and residual variance. The sample distribution method may give unbiased estimates, but it depends on the correct specification of the sampling process.

URL

http://smx.sagepub.com/content/43/1/178.abstract

Reference Type

Journal Article

Journal Title

Sociological Methodology

Author(s)

Cai, Tianji

Year Published

2013

Volume Number

43

Issue Number

1

Pages

178-219

DOI

10.1177/0081175012460221

Reference ID

4628