now be less of a factor in pharmacovigilance than before, but is worth accounting for it’s potential effect when using spontaneous reported data in quantitative or numerical analyses.
(2): Secular effects are much less well documented in the literature. This refers to the use of media awareness, celebrity endorsements or critical comments or some other external event or media comment on a drug.
Drug safety officers live in dread of reports of celebrities or politicians using a particular product, especially if an AE is reported or if spectacular efficacy or harm is anecdotally reported. This phenomenon is also called temporal bias and reflects an increase in AE reporting for a drug or class of drugs after increased media attention or rumor, use of a medication by a celebrity, a warning from a health agency, social media comments, and so on. There are many multipliers of this effect, particularly if the event is widely reported or goes “viral” on social media. “Overall adverse drug reaction reporting rates can be increased several times by external factors such as a change in a reporting system or an increased level of publicity attending a given drug or adverse reaction”.6
The best known example of this is complicated and involves the still on-going (by some) discussion of vaccines increasing the risk for autism. Ultimately, it was concluded by the US CDC and others that there was no evidence to suggest a link between vaccines (in particular those containing thimerosal as preservative) and the risk of autism.
Wakefield and 10 of the original 12 authors retracted the interpretation of the original data. According to the retraction, “no causal link was established between MMR vaccine and autism as the data were insufficient”. Also, Wakefield did not disclose certain financial conflicts, i.e., he was funded by lawyers suing vaccine companies.7 The medical and media reports and comments on this are extensive. A good place to start is the comment by the CDC “Vaccines Do Not Cause Autism” (https://www.cdc.gov/vaccinesafety/concerns/autism.html).
Perhaps the biggest limitation of spontaneous and stimulated ADR reporting, inherent to its nature of surveillance, is incomplete data which prevent the accurate quantitative estimation of risk. Ideally, one would like to calculate the probability for the occurrence for a particular AE in the patients receiving a particular drug, or risk, where
Numerator data are almost always incomplete. The number of AE reported is less than the true number of AE occurrences, because not all AEs are reported nor do we know what percent of AEs that occur are not reported.
There are many publications from around the world on underreporting but few actually are able to measure the true rate of underreporting (“a known unknown”).
It was estimated that in the United Kingdom only 10% of serious ADRs and 2–4% of non-serious ADRs that occur are reported.8 In the United States, the FDA estimated that only 1% of serious suspected ADRs were reported.9 In France, it was estimated that “no more than 5% of serious ADRs” are actually reported.10
In 2006, a report in the UK, Hazell and Shakir looked at the phenomenon and noted that, “In total, 37 studies using a wide variety of surveillance methods were identified from 12 countries. These generated 43 numerical estimates of under-reporting. The median under-reporting rate across the 37 studies was 94% (interquartile range 82–98%).” They reported that, “Factors associated with under-reporting included ignorance, diffidence and lethargy.”11
In a study in Swedish hospitals, an underreporting rate of 86% for 10 selected diagnoses was found in one study (Underreporting of serious adverse drug reactions in Sweden.)12
A more recent report from Canada (https://uottawa.scholarsportal.info/ojs/index.php/RISS-IJHS/article/download/1448/1422) contained a good review of the situation as of 2014 and estimates that only 5% of AEs in Canada are reported.
The number of unreported AEs is, therefore, quite variable. This renders the numerator in the proportion quite suspect.
Denominator data are also incomplete.
The denominator, patient exposure, is often unknown in spontaneous reporting systems. Although one can obtain prescription data and one can know how many tablets, capsules, or tubes were sold, it is hard to know how many consumers actually took the product in the manner and for the length of time prescribed. Should one count the patient who took one tablet once in the same way as someone who took three tablets a day for a month or a year?
Ideally, denominator data would be reported in a number of ways including patients exposed (for risk estimates), patient-time exposed (number of patients times the length of time each patient took the drug such as patient-months or patient-years) for incidence rates estimates. But instead of accurate denominator figures, the only known data are total number of tablets sold, kilograms manufactured, prescriptions written or dispensed and so on.
Nonetheless, by obtaining these data, crude estimates of the number of ADR reports divided by a quantitative denominator, or reporting rates can be made. However, the numbers are often not terribly informative. One of the authors recalls one widely used antihistamine for which 12 cardiac arrhythmia reports were received for what was calculated to be 9 billion (9,000,000,000) patient-years of exposure. This works out to 12/9,000,000,000 or a reporting rate of 0.0000000013 cardiac arrhythmia events/patient-years of exposure, which is not a meaningful number. In fact, this number is so low that it is below the naturally occurring incidence of cardiac arrhythmias in the population, meaning marked under-reporting of the AEs. Using a different interpretation of these data, one could argue that the drug in question actually prevents cardiac arrhythmias, which is clearly not the case. This number is useless to a clinician who needs to make a decision on whether a particular cardiac patient should be given this drug.
The manufacturing data (kilograms manufactured) are available from the company producing the product, and the prescription information and the patient exposure data are obtained from various private companies that track such things. Confounders include generic products in which the denominator is not included in the calculation but in which the company reporting on the branded drug receives and includes AEs (numerator cases) as well as counterfeit drugs where the denominator and numerator are compromised. Nonetheless, drug utilization data can be broken down by gender, age, and other demographic characteristics. Trends over time in usage can be observed.
In summary, as has been said, “The numerator is bad, the denominator is worse, and the ratio is meaningless.” Hence, one cannot calculate quantitative measures of risk for a particular AE based on spontaneous data, only reporting frequencies. Period.
However, the large repositories of spontaneous data can be somewhat used to monitor unexpected patterns of reporting through innovative quantitative techniques looking at proportional reporting. These include proportional reporting ratios (PRR), gamma poisson shrinker (GPS), urn-model algorithm, reporting odds ratio (ROR), Bayesian confidence propagation neural network–information component (BCPNN-IC), and adjusted residual score (ARS), sequential probability ratio test (max-SPRT). The PRR is used as an example of the application of quantitative disproportionality method, and is described below.13
Disproportionality (proportional reporting ratio (PRR) as an example):