that would tend to indicate a stronger relationship between a substance that is the subject of a study and the substance that is the subject of the dietary supplement claim includes similarities in formulation, serving size, route of administration, total length of exposure, and frequency of exposure. Manufacturers should be aware that other substances involved in the study or included in the dietary supplement product itself might also affect the dietary supplement’s performance or the study results.
Example 5:
To illustrate this issue, assume that a firm has high‐quality studies that are also consistent with the totality of the scientific evidence. The firm would like to use these studies to substantiate a claim that its dietary supplement has a particular effect on the human body, but the studies involved the impact of a specific ingredient in foods on the human body, and did not involve the dietary supplement product itself. In this instance, although the studies might be of high quality, the results of these studies of conventional foods are not applicable to the specific dietary supplement product.64 …
Were the studies based on a population that is similar to that which will be consuming the dietary supplement product? For example, if the study involved young adults, but the product’s claims involve conditions seen only in the elderly, the study might not be applicable to the claims.
Does the claim accurately convey to consumers the extent, nature, or permanence of the effect achieved in the relevant studies and the level of scientific certainty for that effect? …
Example 10:
A firm has developed a product to improve memory and cognitive ability and intends to market the product to parents for their school‐aged children. The firm has several high quality clinical studies that examined the ingredient’s effect in elderly people with diagnosed, age‐related memory problems. These studies alone would likely not be adequate substantiation for a claim about memory improvement in young children because the patient population (elderly people with memory problems) is completely different from the intended population (children) in the claim.
….
In deciding whether studies substantiate a claim, an important consideration is the scientific quality of studies. Scientific quality is based on several criteria including study population, study design and conduct (e.g., presence of a placebo control), data collection (e.g., dietary assessment method), statistical analysis, and outcome measures. For example, if the scientific study adequately addressed all or most of the above criteria, it could be considered of high quality. Generally accepted scientific and statistical principles should be used to determine the quality of the studies used as evidence to substantiate a claim. The “gold” standard is randomized, double blind, placebo‐controlled trial design. However, trials of this type may not always be possible, practical, or ethical. There are several systems available to rate scientific information. Firms making claims are encouraged to refer to these systems when developing substantiation for claims or relying on existing information….
Human studies can be divided into two types: intervention studies and observational studies. Of these types of studies, intervention studies can provide causal evidence to substantiate the effect of a dietary supplement in humans because they can evaluate the product’s direct effect in the human body. Observational studies have a more limited ability than intervention studies to distinguish relationships between a substance and the outcomes being evaluated and cannot provide causal evidence….
The following additional types of information would generally be considered background information, but alone may not be adequate to substantiate a claim.
Animal studies: Animal studies may provide useful background on the biological effects of a substance. However, they often have limited or unknown value in predicting the effect of the substance in humans… . Without any data from human studies, the results of animal studies alone are not sufficient to substantiate a claim.
In vitro studies are studies that are done outside a living body. For example, such studies might examine a product’s effect on isolated cells or tissues. These studies are of limited value in predicting the effect of a substance when consumed by humans… . this evidence alone would not substantiate a claim.
Testimonials and other anecdotal evidence: This type of evidence includes descriptions of experiences of individuals using a dietary supplement product or ingredient. It might also include descriptions of the use of the product or ingredient by others, for example, by other cultures in the past or present. It might consist of an opinion or statement of an expert or someone who endorses the product. Anecdotal evidence generally would not be sufficient to substantiate claims regarding a dietary supplement’s effect because each individual’s experience might be attributable to factors other than the dietary supplement itself… .
Review articles summarize the findings of primary reports. Review articles may identify relevant primary reports, which may provide substantiation for the claim. Review articles may also provide background information that is useful to understand the scientific issues about the relationship between the substance and the claimed effect… .
Multiple factors should be considered in study design. These include, but are not limited to:
Bias, confounders, and other limitations: Potential sources of bias include lack of appropriate randomization and blinding, the number of subjects called for in the protocol vs. the number of subjects who actually participated in the trial, demographics, adequacy of primary variables, compliance, control agent, drop‐outs, statistical procedures, subgroup analysis, safety issues, and reproducibility of results. Confounders are factors that are associated with the outcome in question and the intervention and prevent the measured outcome from being attributed unequivocally to the intervention. Potential confounders include variability in the quantity of the dietary supplement being administered or the presence of other dietary ingredients that may have their own independent effects. These factors can limit the reliability of the study… .
Peer Review: The nature and quality of the written report of the research are also important. Although studies or evidence used to substantiate a claim do not have to be published in a peer‐reviewed journal or publication, such publications do give some level of assurance that qualified experts have reviewed the research and found it to be of sufficient quality and validity to merit publication. In contrast, an abstract or informal summary of an article is less reliable, because such documents usually do not give the reader enough insight into how the research was conducted or how the data were analyzed to objectively evaluate the quality of the research data and the conclusions drawn by the authors. Moreover, the mere fact that the study was published does not necessarily mean that the research is competent and reliable evidence adequate to substantiate a particular claim.
E. Consider the Totality of the Evidence …
In determining whether there is adequate evidence to substantiate a claim, one should consider the strength of the entire body of evidence, including criteria such as quality, quantity (number of various types of studies and sample sizes), relevance of exposure, and consistency and replication of the findings.
To determine whether the available scientific evidence is adequate to substantiate a claim, it is important to consider all relevant research, both favorable and unfavorable….
There is no general rule for how many studies, or what combination of types of evidence, is sufficient to support a claim. However, the replication of research results in independently conducted studies makes it more likely that the totality of the evidence will support a claim.
Although the quality of individual pieces of evidence is important, each piece should be considered in the context of all available information; that is, the strength of the total body of scientific evidence is the critical factor in assessing whether a claim is substantiated….
F. Conclusion