or late recurrence of disease. For example, in an IPD meta‐analysis evaluating the addition of chemotherapy local treatment for soft tissue sarcoma,102 the median follow‐up for survival was reported for seven of the included trials as being between 16 and 64 months,46 which is rather short for this type of cancer. Trial investigators supplying IPD were asked to provide updated information, which extended the median follow‐up for these trials to between 74 and 204 months,46 and thus allowed a more reliable examination of the effects of chemotherapy in the long term (Figure 4.5).102 Such updating may not be necessary if most or all events have already occurred, and may not be feasible if, for example, studies are very old, no longer obtaining follow‐up information, or resources are limited.
Figure 4.5 Median follow‐up based on published aggregate data compared to updated obtained IPD for seven trials included in an IPD meta‐analysis of adjuvant chemotherapy for soft tissue sarcoma.46
Source: Stewart et al.46, © 2006, John Wiley & Sons.
While ideally these checks should make use of database or statistical code, the value of scanning the data by eye should not be underestimated, as it can help members of the central research team get a feel for the trial as a whole, and even highlight unusual patterns or peculiarities.
4.5.3 Harmonising IPD across Trials
If data providers have followed the supplied data dictionary closely when preparing their IPD, much of the data harmonisation will have been done already, and minor adjustments may be all that are required. If trial investigators are unable or unwilling to prepare data according to suggested pre‐specified formats, the central research team should accept data in whichever format is most convenient, and recode it as necessary.
Beyond simply aligning trial IPD to the data dictionary, there is also the opportunity to standardise definitions of outcomes or participant‐level variables,7,43 such as scoring or staging systems. For example, in an IPD meta‐analysis examining the effects of chemotherapy for soft tissue sarcoma,102 different definitions of histological grade were used in the included trials, but with input from trial investigators, it was possible to translate each of these into a high‐ or low‐grade disease category, allowing exploration of treatment effectiveness according to grade.43 It may also be necessary to construct new standardised variables for use in analyses. For example, in an IPD meta‐analysis of the effects of antenatal diet and physical activity on maternal and foetal outcomes, the research team collected data on each woman’s height, baseline weight and parity, as well as the gestational age at birth and foetal birthweight for each baby. This allowed researchers to generate a standardised meta‐analysis definition of ‘small for gestational age’ (< 10th centile), using a bulk birthweight centile calculator.103
4.5.4 Checking the Validity, Range and Consistency of Variables
IPD should be checked for invalid, outlying or implausible values,7,9,43 as these are the sorts of errors that can occasionally be missed during data input or checking of the original trials, or occasionally could arise as a result of variable recoding by the IPD meta‐analysis project team. Such checks might highlight, for example, records of unusually old or young patients, or those with abnormally high or low levels of important biomarkers, which need to be queried with trial teams. Of course, such apparently implausible values may turn out to be accurate. For example, in a trial included in an IPD meta‐analysis of neoadjuvant chemotherapy for cervical cancer,104 some invalid codes were used for the brachytherapy (internal radiotherapy) variable (Figure 4.6), which were then queried and clarified as representing not applicable. When variables are related, checks should be carried out to ensure that they are coherent and consistent. For example, in a randomised trial, the recorded date of any event should follow the recorded date of randomisation and occur on or before the recorded date of last follow‐up.
At this stage, it is also useful to perform a simple descriptive analysis of the IPD from each trial to provide, for example, the number of participants, distribution of baseline characteristics by treatment group and overall results for the main outcome(s). These can then be checked for concordance with relevant publications or, for unpublished trials, with any results that have been deposited in trial registers (e.g. ClinicalTrials.gov). However, it should be borne in mind that inconsistencies can arise if, for example, follow‐up in a trial’s IPD has been extended beyond that used to derive the reported results, or if the meta‐analysis employs a different approach compared to the original trial analyses. When unexplained differences do arise, it is crucial to work with the original trial investigators to understand how and why they differ, and therefore, be in a position to report and explain any important discrepancies.
Figure 4.6 Summary of the data validity, range and consistency checks on IPD from a trial included in an IPD meta‐analysis of neoadjuvant chemotherapy for cervical cancer104 (N.B. The participant references are pseudonymised).
Source: Based on Neoadjuvant Chemotherapy for Locally Advanced Cervical Cancer Meta-analysis C. Neoadjuvant chemotherapy for locally advanced cervical cancer: a systematic review and meta-analysis of individual patient data from 21 randomised trials. European journal of cancer 2003;39(17):2470–86.
4.6 Checking the IPD to Inform Risk of Bias Assessments
Similar to conventional aggregate data reviews, assessing the reliability (quality) of included trials is also an important feature of the checking phase of IPD meta‐analysis projects. In such reviews, this is usually based on the risk of bias, a term that refers to the likelihood that included trials will generate biased results. In particular, the risk of bias assessment tool (RoB 2) can be used to evaluate potential bias in estimates of intervention effects from randomised trials.91 It includes five domains to be considered for each eligible trial: the randomisation process; deviations from intended interventions; missing outcome data; measurement of the outcome; and selection of the reported result. Within each domain, assessments are guided by multiple signalling questions (with answers: yes, probably yes, probably no, no, or no information), allowing a risk of bias classification for that domain (low, high, or some concerns). Finally, an overall risk of bias judgement can be made (low, high, or some concerns) based on all domains (Section 4.7).
In aggregate data reviews, assessment of risk of bias is usually based on the information available in trial publications and other publicly accessible documents, such as trial registration entries or published protocols, sometimes supplemented by information requested from trial investigators. In an IPD meta‐analysis project, it is common to obtain additional information from protocols, codebooks and forms, or direct from trial investigators, which can increase the clarity of risk of bias assessments compared to those based on trial reports alone.48,105 As discussed in Sections