2.6.2), the information size (Section 2.6.3), and the data and analyses required to address the research question reliably (Section 2.6.4). A checklist of questions to aid in this decision‐making is provided in Table 2.2, and is explained in more detail in the following subsections. The focus is on the evaluation of treatment effects from randomised trials. If the answer to one or more of the questions is “yes”, then an IPD project is likely to add considerable value compared to a conventional systematic review and meta‐analysis of aggregate data.
2.6.1 Are IPD Needed to Tackle the Research Question?
In certain circumstances, the rationale for obtaining and analysing IPD is immediately very strong, such as when the research question is focused on participant‐level relationships to inform more stratified or personalised approaches to treatment. In particular, if the desire is to evaluate whether particular types of participants benefit more or less from an intervention than others (Chapter 7), IPD will almost always be needed to obtain a reliable assessment of treatment‐covariate interactions.33 This is also true for situations where participant‐level diagnosis, prognosis and prediction are of interest (Chapters 15 to 17). Another powerful motivation for seeking IPD is in controversial areas, where independent scrutiny of the trial IPD may improve credibility and increase transparency, which may be sufficient justification in itself.20,65 On the other hand, if it is clear that a conventional aggregate data meta‐analysis could provide reliable (albeit less detailed) results, then an IPD project may be of less value, or may not be considered a priority. For example, if the sole research objective is to investigate the overall effect of a treatment, then it may be reasonable to rely on aggregate data from existing trials (reported or otherwise available) that summarise these effects (Section 2.6.2).
Table 2.2 Signalling questions to help decide when aggregate data are insufficient for meta‐analysis and IPD are needed, focusing on the evaluation of treatment effects from randomised trials.
Source: Jayne Tierney, adapting the figure presented by Tudur Smith et al.,64 with permission.
| Why IPD might be required | Signalling questions to help consider whether IPD are needed | Yes/No |
|---|---|---|
| To address the specific research question | Is going beyond overall treatment effects an aim of the project? (e.g. to examine treatment effects in relation to particular participant characteristics) | |
| Is independent scrutiny of one or more eligible trials required? (e.g. if some trial results are controversial or all trials arise from a single sponsor) | ||
| Is it reasonable to wait some time for the research question to be addressed?(e.g. if a good‐quality aggregate data meta‐analysis already exists, but IPD are needed for a more thorough or up‐to‐date analysis) | ||
| To improve the completeness and uniformity of the information | Are suitable aggregate data lacking for key outcomes of the trials? (e.g. publications do not provide a risk ratio, mean difference or hazard ratio for the overall effect; a treatment‐covariate interaction for each participant‐level covariate of interest, or the data to calculate these) | |
| Are outcome definitions corresponding to the aggregate data unsuitable or do they lack uniformity across trials? (This could make synthesis or interpretation of outcome effects using aggregate data difficult) | ||
| Are participant‐level covariate definitions corresponding to the aggregate data unsuitable or do they lack uniformity across trials?(This could make synthesis or interpretation of treatment‐covariate interactions using aggregate data difficult) | ||
| To improve the information size | Is the absolute information size represented by the aggregate data too small to detect realistic effects of treatment on the main outcomes? (i.e. the total number of participants, and total events if applicable) | |
| Is the relative information size represented by the aggregate data ‘low’ or potentially unrepresentative?(i.e. the proportion of all potentially eligible participants or events, if applicable) | ||
| For time‐to‐event outcomes, is the duration of follow‐up captured by the aggregate data too short? | ||
| To improve the quality of the analyses | Are the statistical analysis methods and assumptions used by the trials to produce the aggregate data inappropriate? | |
| Are the statistical analyses used by across the trials to produce the aggregate data incompatible? | ||
| Are continuous outcomes and variables handled inappropriately in the trial analyses that produced the aggregate data? | ||
| Answer to key questions “YES” = IPD may add considerable value |
It will not always be possible to complete an IPD meta‐analysis project in a suitably timely manner for the question of interest, because, for example, trial investigators are focused on completion of their individual trials, trial data are embargoed for a period, or it will take too long to set up data‐sharing agreements. Therefore, if a therapeutic area is moving very quickly, there is an urgent policy need, or results are required to inform an ongoing trial of the same treatment(s), a prospective aggregate data meta‐analysis,66, 67 perhaps as part of a living systematic review,67, 68 may be more suitable for delivering results in the shortest time frame. However, ultimately the merits of any aggregate data synthesis need to be balanced against the benefits that a good‐quality IPD project could bring, and for an important question, the extra time needed for IPD meta‐analysis may be justified to better inform decision‐making in the longer term. Instead, researchers might choose to complete a conventional aggregate data meta‐analysis first, with the intention of conducting an IPD meta‐analysis project in a subsequent stage, if more reliable and nuanced results are needed.
2.6.2 Are IPD Needed to Improve the Completeness and Uniformity of Outcomes and Participant‐level Covariates?
Before deciding whether to collect IPD for meta‐analysis, an important step is to assess the completeness and uniformity of the available aggregate data, either based on a pre‐existing systematic review that addresses a similar research question, or by conducting a scoping or systematic review of existing trials of interest. If outcomes and participant‐level covariates have been collected in the eligible trials, but are not (adequately) described in the associated trial reports (such as side effects of treatment, multiple time‐points or continuous values of prognostic