Real World Health Care Data Analysis. Uwe Siebert

Optional additional examination: compute the expected distribution of standardized differences and variance ratios under the assumption of balance (sdm = 0, variance ratio = 1) and assess the observed values in relation to the expected distribution.

      3. Repeat steps 1 and 2 to compute and assess the standardized mean differences and variance ratios for 2-way interactions.

      4. As a final check, graphically assess differences in the full distribution of each covariates between treatments using displays such as a Q-Q plot.

      Of course, one could follow these instructions and substitute different statistics in each step – such as a formal KM test to compare distributions of the covariates instead of the graphical approach – or supplement the Q-Q plots with statistics for mean and max deviation from a 45-degree line as Ho et al. (2007) suggest. However, the goal is clear. A thorough check to confirm that the covariate distributions are similar between the treatment groups is necessary for quality comparative analysis from observational data.

      In practice, the above steps may indicate some imbalance on select covariates and the balance assessment might become an iterative process. Imbalance on some covariates – such as those known to be strongly predictive of the outcome measure – may be more critical to address than imbalance on others. If imbalance is observed, then researchers have several options including revising the propensity model, using exact matching or stratification on a critical covariate, trimming the population, and modifying the analysis plan to incorporate the covariates with imbalance into the analysis phase to address the residual imbalance.