Identify scientific studies that evaluate the substance–disease relationship. Identify surrogate endpoints of disease risk. Evaluate human studies. Assess the methodological quality of the scientific studies. Evaluate the totality of the scientific evidence. Assess significant scientific agreement.
As a preliminary matter before evaluating the science, to determine what information is needed to substantiate a claim, one must first identify and understand the meaning of the claim, expressed and implied. Clearly understanding a claim’s meaning is crucial in identifying the appropriate study hypotheses and measurable endpoints needed to substantiate the claim.
Next, a threshold review of the studies and other evidence is needed to determine whether they have a relationship to the claim. Did the study specify and measure the ingredient that is the subject of the claim? For example, a study only measuring carrot consumption will not support a claim about carotene consumption. Did the study specify and measure the affect that is the subject of the claim? That is, an appropriate endpoint needs to be measured to evaluate the effect.
Then the scientific quality of the evidence must be determined. The criteria of scientific quality include the study type, the study population, the study design and conduct (e.g., presence of a placebo control), data collection, statistical analysis, and outcome measures. High‐quality scientific study adequately addresses all or most of the above criteria.
Finally, whether there is adequate evidence to substantiate a claim is based on the strength of the entire body of evidence. Ideally, the evidence has been replicated in independent studies and is supported by the surrounding body of evidence. However, there is no rule on the number of studies needed or what combination of evidence is sufficient to support a claim because of the diverse nature of various studies. The quality and quantity of the studies and their consistency and relevancy matter. Conflicting or inconsistent results raise serious questions as to whether a particular claim is substantiated. All of the evidence must be weighed in totality.
* * * * *
Guidance for Industry Evidence‐Based Review System for the Scientific Evaluation of Health Claims
CFSAN, FDA (Jan. 2009)
… This guidance document describes the evidence‐based review system that FDA intends to use to evaluate the publicly available scientific evidence for [significant scientific agreement] SSA health claims or qualified health claims on the relationship between a substance and a disease or health‐related condition. This guidance document explains the agency’s current thinking on the scientific review approach FDA should use and is intended to provide guidance to health claim petitioners.
The specific topics addressed in this guidance document are: (1) identifying studies that evaluate the substance/disease relationship, (2) identifying surrogate endpoints for disease risk, (3) evaluating the human studies to determine whether scientific conclusions can be drawn from them about the substance/disease relationship, (4) assessing the methodological quality of each human study from which scientific conclusions about the substance/disease relationship can be drawn, (5) evaluating the totality of scientific evidence, (6) assessing significant scientific agreement, (7) specificity of claim language for qualified health claims, and (8) reevaluation of existing SSA or qualified health claims….
An evidence‐based review system is a systematic science‐based evaluation of the strength of the evidence to support a statement. In the case of health claims, it evaluates the strength of the scientific evidence to support a proposed claim about a substance/disease relationship. The evaluation process involves a series of steps to assess scientific studies and other data, eliminate those from which no conclusions about the substance/disease relationship can be drawn, rate the remaining studies for methodological quality and evaluate the strength of the totality of scientific evidence by considering study types, methodological quality, quantity of evidence for and against the claim (taking into account the numbers of various types of studies and study sample sizes), relevance to the U.S. population or target subgroup, replication of study results supporting the proposed claim, and overall consistency of the evidence. After assessing the totality of the scientific evidence, FDA determines whether there is SSA to support an authorized health claim, or credible evidence to support a qualified health claim….
Before the strength of the evidence for a substance/disease relationship can be assessed, FDA separates individual relevant articles on human studies from other types of data and information. FDA intends to focus its review primarily on articles reporting human intervention and observational studies because only such studies can provide evidence from which scientific conclusions can be drawn about the substance/disease relationship in humans. Next, the agency considers a number of threshold questions in the review of the scientific evidence:
Have the studies specified and measured the substance that is the subject of the claim? Studies should identify a substance that is measurable… .
Have the studies appropriately specified and measured the specific disease or health‐related condition that is the subject of the claim? “Disease or health‐related condition” is defined as damage to an organ, part, structure, or system of the body such that it does not function properly (e.g., cardiovascular disease), or a state of health leading to such dysfunctioning (e.g., hypertension). 21 C.F.R. 101.14(a) (5). Studies should identify a specific measurable disease or health‐related condition by either measuring incidence, associated mortality, or validated surrogate endpoints that predict risk of a specific disease.
For example, cancer is a constellation of more than 100 different diseases, each characterized by the uncontrolled growth and spread of abnormal cells. Cancer is categorized into different types of diseases based on the organ and tissue sites. Cancers at different organ sites have different risk factors, treatment modalities, and mortality risk… . Since each form of cancer is a unique disease based on organ site, risk factors, treatment options, and mortality risk, FDA’s current approach is to evaluate each form of cancer individually in a health claim or qualified health claim petition to determine whether the scientific evidence supports the potential substance–disease relationship for that type of cancer, which would constitute a disease under 21 C.F.R. 101.14(a)(5)….
After considering these threshold issues, FDA categorizes the studies by type.
Intervention Studies
In an intervention study, subjects are provided the substance (food or food component) of interest (intervention group), typically either in the form of a conventional food or dietary supplement. The quality and quantity of the substance should be controlled for. In randomized controlled trials, subjects are assigned to an intervention group by chance. Individual subjects may not be similar to each other, but the intervention and control groups should be similar after randomization. Randomized controlled trials offer the best assessment of a causal relationship between a substance and a disease because they control for known confounders of results (i.e., other factors that could affect risk of disease). Through random assignment of subjects to the intervention and control groups, these studies avoid selection bias—that is, the possibility that those subjects most likely to have a favorable outcome, independent of an intervention, are preferentially selected to receive the intervention. Potential bias is also reduced by “blinding” the study so that the subjects do not know whether they are receiving the intervention, or “double blinding,” in which neither the subjects nor the researcher who assesses the outcome knows who is in the intervention group and who is in the control group. By controlling the test environment, including the amount and composition of substance consumed and all other dietary factors, these studies also can minimize the effects of variables or confounders on the results. Therefore, randomized, controlled intervention studies provide the strongest evidence of whether or not there is a relationship between a substance and a disease.
Furthermore, such studies can provide