balanced. Note that we do not advocate statistical tests of baseline balance.108
4.6.2 Deviations from the Intended Interventions
While robust randomisation procedures should ensure the unbiased assignment to, and comparison of, participants between treatment groups, this can only be guaranteed if all participants are analysed according to the treatments initially assigned: an intention‐to‐treat approach.109–111 Even if a trial has not been analysed appropriately, as long as the IPD have been provided with the original treatment allocation recorded, then participants can be grouped according to this treatment allocation, rather than the treatment they received, enabling an intention‐to‐treat analysis of effectiveness.109–111 However, there may be value in conducting certain analyses based on a subset of participants randomised, such as an analysis of toxicity in just those who received most of their allocated treatment, or sensitivity analyses according to treatment received, to explain differences between published trial results and those used in the meta‐analysis.
There may be sufficient detail in the trial dataset to check whether participants who deviated from intended interventions did so for pre‐specified or otherwise rational reasons. For example, if the data indicate that a participant had experienced an adverse event, this might explain why treatment was stopped early, and would also need to be considered in any analysis of adverse outcomes. It may also be possible to assess whether deviations from planned treatment are similar, and for comparable reasons across treatment groups (more so than with aggregate data).
If a treatment is a major procedure, such as surgery, or particularly toxic, then those delivering treatments and participants will usually be aware of the assigned treatment. Provided that outcomes are objectively measured or ‘hard’, such as mortality, this is unlikely to introduce bias. However, a carer might inadvertently or otherwise deliver a treatment or measure a more subjective outcome, such as an adverse effect, differently if they are aware of which treatment a participant received. Similarly, if a participant is aware of their assigned treatment, it might influence a patient‐reported outcome, such as pain or quality of life. Therefore, in this scenario, there is a potential for bias in this domain, which cannot be alleviated by the collection of IPD. However, the contact with trial teams, that is intrinsic to collaborative IPD meta‐analyses, can provide useful clarification of the methods used to blind participants, carers or outcome assessors, to help determine whether these are appropriate, and therefore allow the risk of bias to be judged with more accuracy.
Figure 4.10 Days of the week participants were allocated to treatment and control groups in a trial included in (a) an IPD meta‐analysis of pre‐operative chemotherapy for non‐small cell lung cancer,88 and (b) an IPD meta‐analysis of post‐operative radiotherapy for non‐small cell lung cancer.107
Source: (a) Based on NSCLC Meta-Analysis Collaborative Group. Preoperative chemotherapy for non-small cell lung cancer: a systematic review and meta-analysis of individual participant data. Lancet 2014;383:1561–71. (b) Based on PORT Meta-analysis Trialists Group. Postoperative radiotherapy in non-small-cell lung cancer: systematic review and meta-analysis of individual patient data from nine randomised controlled trials. The Lancet 1998;352(9124):257–63.
4.6.3 Missing Outcome Data
If participants drop out or are actively excluded from the analysis of a trial in substantial numbers and/or disproportionately by group, this could lead to quite considerable imbalances between intervention and control groups. More importantly, it could lead to incomplete outcome data and potentially attrition bias. For example, an examination of 14 cancer IPD meta‐analyses, incorporating 133 trials, found that between 0% and 38% of randomised participants were excluded from the original trial survival analyses, with the largest proportion often being excluded from the treatment groups (Figure 4.11). In one of these projects examining the effects of chemotherapy for soft tissue sarcoma,112 the evidence for a benefit of chemotherapy on survival was stronger when it was based just on participants included in the original trial analyses (hazard ratio = 0.85; 95% CI: 0.72 to 1.00, p = 0.06), compared to when all possible participants were included (hazard ratio = 0.90, 95% CI: 0.77 to 1.04, p = 0.16).
Figure 4.11 Percentage of participants excluded from the original analyses of trials included in 14 IPD meta‐analysis projects in the cancer field.112
Source: Jayne Tierney, reproduced with permission from Tierney and Stewart.112
Therefore, it is important to check that data on all, or as many as possible, participants recruited to a trial are included in an IPD meta‐analysis (assuming that all the individuals meet the IPD meta‐analysis inclusion criteria for population and setting). If good records are maintained for a trial, it is possible to recover data on participants who were excluded from the original trial analyses, as part of the IPD collection process, and incorporate them into the meta‐analysis. For example, in the same 14 cancer meta‐analyses described previously, when IPD were collected, approximately 1,800 participants who had been excluded from the original trial analyses were re‐instated, and without them most meta‐analysis results would have been biased towards the research intervention, albeit to a small degree in most cases.112
A major advantage of IPD meta‐analysis is the ability to include all outcomes of relevance to the meta‐analysis, irrespective of whether they have been published or not, thereby overcoming the potential biases associated with differential reporting of outcomes,113 and providing a more balanced view of benefits and harms. For example, in a systematic review of laparoscopic versus open surgery for the repair of inguinal hernia, based on the available published aggregate data from three trials, the risk of persistent pain was found to be significantly greater with laparoscopic repair (odds ratio = 2·03, 95% CI: 1·03 to 4·01).114 However, when IPD were collected, data were available for a further 17 trials that had not published results for this outcome, and the combined meta‐analysis results showed that the risk of persistent pain was actually lower with laparoscopic repair (odds ratio = 0·54, 95% CI: 0·46 to 0·64). Recognising that some outcomes measured in trials may not be reported, it is always worth checking trial protocols, registry entries and with trial investigators to firmly establish which outcomes can be made available when IPD are provided.115
4.6.4 Measurement of the Outcome
Direct access to IPD does not usually allow the assessment of whether the measurement or ascertainment of outcomes differed between intervention groups. However, contact with trial investigators can provide useful clarification of the methods used, to help determine whether these are appropriate, and therefore allow the risk of bias associated with, for example, differential outcome assessments to be judged with more certainty. If the IPD are sufficiently detailed, outcomes may be defined more appropriately or consistently across trials. For example, a new standardised composite outcome might be constructed from a series of component variables.
For