href="#ulink_5a407408-330e-5fe7-806d-f0ddca72e90a">Section 4.6),48 and if potential biases or errors are identified, they may be able to supply additional data to resolve or minimise these (Section 4.5.4).7,9,43 Bringing together a group of international and multi‐disciplinary collaborators can also facilitate wider discussion and interpretation of results, and aid dissemination of key findings (Chapter 10).
The advantages shown in Table 2.1 focus on the synthesis of randomised trials to evaluate treatment effects, but IPD meta‐analyses have further advantages for non‐efficacy questions. In particular, they are pivotal in guiding and tailoring diagnostic strategies (Chapter 15), identifying risk and prognostic factors (Chapter 16), and individualising risk prediction (Chapter 17) to guide healthcare policy and practice.32
2.4 What Are the Potential Challenges of an IPD Meta‐Analysis Project?
The processes for collecting, checking and analysing IPD are more involved and complex than for conventional aggregate data reviews of the same topic, and therefore, more time and resources are required (Chapter 3).7,9,43 Thus, before embarking on an IPD project, careful consideration about whether it is an appropriate course of action is needed (Section 2.6).47,49
As most IPD projects rely on collaboration with the teams responsible for the included trials, negotiating and maintaining relationships with investigators from different countries, settings and disciplines can take considerable time, effort, diplomacy and careful management (Section 3.2.1).43,44 In an era in which the value of clinical data sharing is more widely appreciated, persuading trial investigators of the value of participating has perhaps become easier. That said, IPD meta‐analysis research teams and those providing IPD are now faced with additional tasks, such as seeking ethical or other institutional approval for the exchange of IPD (Section 3.10), as well the development of detailed data‐sharing agreements (Section 3.11).
Over the years, advances in database and statistical software and electronic communication have greatly reduced the burden of labour required for the data exchange, management and analysis aspects of IPD projects.43 Even so, these remain the most time‐consuming and resource‐intensive phases (Section 3.8), and require skills and expertise beyond those needed for a conventional aggregate data review (Section 3.5). This is emphasised by a growing number of methodological articles relating to IPD meta‐analysis projects, covering issues such as data checking and harmonisation,7,28 statistical methodology,33,50–57 examining potential biases,46,58 dealing with unavailable IPD,59 reporting,60 and statistical software development,61 amongst others.
2.5 Empirical Evidence of Differences Between Results of IPD and Aggregate Data Meta‐Analysis Projects
There are many empirical comparisons of results produced by IPD meta‐analyses with results based on corresponding analyses of published aggregate data. An early example, in advanced ovarian cancer, found that results based on published aggregate data suggest a 7.5% absolute improvement in the percentage of women surviving at 30 months with platinum‐based chemotherapy, whereas the IPD meta‐analysis suggests a 2.5% improvement in the percentage surviving.8 This disparity, which could have led to different clinical conclusions, seemed to be driven by the IPD meta‐analysis project including more trials, participants and follow‐up, as well as including all of the events in a time‐to‐event analysis, rather than calculating a risk ratio from events observed at a fixed time‐point.
In contrast, in a different example, there was no clear evidence of an effect of ovarian ablation on survival of women with early breast cancer, based on the published aggregate data, but a 10% absolute increase in the percentage surviving at 15 years based on IPD.62 In this case, the IPD for the included trials incorporated much greater follow‐up, leading to a near doubling of events. This, and a more appropriate time‐to‐event analysis, were likely to be the key drivers for the discrepancy with the aggregate data findings. Many other comparisons of results from IPD and aggregate data meta‐analysis projects have been carried out, not only in cancer and cardiovascular disease where IPD meta‐analysis first gained traction, but also in other healthcare areas such as infectious diseases, neurology, nephrology and critical care. The differences shown between IPD and aggregate data findings are variable and seem context specific (e.g. depending on the research question; Sections 2.2.1 and 2.6.1).63
A large systematic review that brought together published comparisons of treatment effects from IPD and aggregate data meta‐analyses found that many pairs of IPD and aggregate data analyses agreed in terms of the statistical significance of the overall results for the main outcomes. However, the disagreement observed in 20% of cases could have led to different clinical conclusions.63 The discrepancies did not seem to be clearly associated with variation in the number of trials, number of participants or length of follow‐up.63 Importantly, discrepancies are likely to be more pronounced when going beyond overall treatment effects, which is often a key aim of an IPD meta‐analysis project, such as when examining treatment‐covariate interactions at the participant level (Chapter 7).33
Evidence from a large cohort of systematic reviews of the effects of cancer therapies on survival showed that, on average, meta‐analysis results for the overall treatment effect derived from published aggregate data (based on hazard ratios) were slightly more in favour of the research treatment than those from IPD.47 Although most results were similar between aggregate data and IPD meta‐analyses, those discrepancies that did occur were often substantial.47 Importantly, results from aggregate data were most likely to agree with those from IPD when the number of participants or events (absolute information size) and the proportion of participants or events available from the aggregate data relative to the IPD (relative information size) were large. This emphasises that assessing the amount of information provided by the available aggregate data, and what the obtainable IPD might add for a particular research question, is an important step in determining when IPD will bring the greatest value (Section 2.6.3).
2.6 Guidance for Deciding When IPD Meta‐Analysis Projects Are Needed to Evaluate Treatment Effects from Randomised Trials
Based on practical experience7,43 and the empirical evidence summarised in Section 2.5,47,63 any decision about undertaking an IPD meta‐analysis project should be based on the nature of the specific research question (Section 2.6.1), the completeness and uniformity of the available aggregate data (