EPA's Proposed Cancer Risk AssessmentGuidelines:

Has the Statistical Significance Requirement BeenDeleted?

BACKGROUND

The U.S. Environmental Protection Agency (EPA) recently proposednew guidelines forcarcinogen risk assessment (1). The proposed guidelines areintended to replace the existingguidelines that were issued in 1986 (2). The 1986 guidelinesrequire, among other things, thatepidemiologic studies be statistically significant where they areused to establish cause-and-effectrelationships between exposures of concern and cancer. Thisrequirement is not included in theproposed guidelines.

On May 6, 1996, EPA held a forum on the proposed guidelines. Atthis forum, EPA staff wasasked why statistical significance was not included as arequirement for determining cause-andeffect relationships. EPA staff denied that the statisticalsignificance requirement had beenomitted.

ISSUE

Do the proposed guidelines require that epidemiologic studies bestatistically significant beforethey may be used to support conclusions that cause-and-effectrelationships exist betweenexposures of concern and cancer.

WHAT IS STATISTICAL SIGNIFICANCE AND WHY IS IT SIGNIFICANT?

In its 1991 report entitled Environmental Epidemiology: PublicHealth and HazardousWastes , the National Academy of Sciences stated that(3)
Historically, discussions on causality [based on epidemiology]have proceeded once astatistically significant relationship between a potential casualfactor and a disease has beenfound, as is discussed below...The requirement that a finding bestatistically significant has beena convention of epidemiologic research. If results have alikelihood of only 5 percent or less ofoccurring by chance, then they are usually considered statisticallysignificant, as measured by anumber of customary tests, such as p and t values.

Thus, statistical significance is a traditional and standard toolto rule out luck or chance as beingthe factor that has caused the observed results of an epidemiologicstudy.

STATISTICAL SIGNIFICANCE IS REQUIRED BY THE 1986GUIDELINES.

According to the 1986 guidelines:
Three criteria must be met before a causal association can beinferred between exposureand cancer in humans:
1. There is no identified bias that could explain theassociation.
2. The possibility of confounding has been considered and ruled outas explaining theassociation.
3. The association is unlikely to be due tochance.

The latter criteria is statistical significance.

Statistical significance is not required by the 1996 proposedguidelines.
In relevant part, the 1996 guidelines propose that(4)
2.2.1.3. Criteria for Causality. A causal interpretation isenhanced for studies to the extentthat they meet the criteria described below. None of the criteriais conclusive by itself, and theonly criterion that is essential is the temporalrelationship...

Temporal relationship: The development ofcancers requirecertain latency periods, and while latency periods vary, existenceof such periods is generallyacknowledged. Thus, the disease has to occur within a biologicallyreasonable time after initialexposure. This feature must be present if causality is to beconsidered.

Consistency: Associations occur in severalindependentstudies of a similar exposure in different populations, orassociations occur consistently fordifferent subgroups in the same study. This feature usuallyconstitutes strong evidence for acausal interpretation when the same bias or confounding is not alsoduplicated acrossstudies.

Magnitude of the association: A causalrelationship is morecredible when the risk estimate is large and precise (narrowconfidence intervals).

Biological gradient: The risk ratio (i.e.,the ratio of the riskof disease or death among the exposed to the risk of the unexposed)increases with increasingexposure or dose. A strong dose response relationship acrossseveral categories of exposure,latency, and duration is supportive for causality given thatconfounding is unlikely to becorrelated with exposure. The absence of a dose responserelationship, however, is not by itselfevidence against a causal relationship.

Specificity of the association: Thelikelihood of a causalinterpretation is increased if an exposure produces a specificeffect (one or more tumor types alsofound in other studies) or if a given effect has a uniqueexposure.

Biological plausibility: The associationmakes sense interms of biological knowledge. Information is considered fromanimal toxicology, toxicokinetics,structure-activity relationship analysis, and short-term studies ofthe agent's influence on eventsin the carcinogenic process considered.

Coherence: The cause-and-effectinterpretation is in logicalagreement with what is known about the natural history and biologyof the disease, i.e., theentire body of knowledge about the agent.

Although the number of criteria have increased from three in the1986 guidelines to seven in theproposed guidelines, the requirement of statistical significance isnot included in the proposedguidelines.

THE PROPOSED GUIDELINES MISLEAD THE PUBLIC ABOUT THE STATISTICALSIGNIFICANCE REQUIREMENT.

The proposed guidelines contain language in several places thatgives the misleading impressionthat statistical significance is included in the guidelines.Proposed section 2.2.1.2. Criteriafor Assessing Adequacy of Epidemiologic Studies , states(5)
Statistical Considerations. The analysis applies appropriatestatistical methods to ascertainwhether or not there is any significant association betweenexposure and effects. A descriptionof the method or methods should include the reasons for theirselection. Statistical analyses ofthe potential effects of bias or confounding factors are part ofaddressing the significance of anassociation, or lack of one, and whether a study is able to detectany effect.

The analysis augments examination of the results for the wholepopulation with exploration ofthe results for groups with comparatively greater exposure or timesince first exposure. This maysupport identifying an association or establishing a dose responsetrend. When studies show noassociation, such exploration may apply to determining an upperlimit on potential human riskfor consideration alongside results of animal tumor effectsstudies.

This language mentions doing statistical analyses only todetermine whetherstatistical significance exists. This language does not requirethat statistical significance existbefore concluding that a causal relationship exists.

Also, section 2.6.1 Weight of Evidence Evaluation for PotentialHuman Carcinogenicity, states(6)
Human Evidence. Analyzing the contribution of evidence from abody of human datarequires examining available studies and weighing them in thecontext of well-accepted criteriafor causation (see section 2.2.1). A judgment is made about howclosely they satisfy thesecriteria, individually and jointly, and how far they deviate fromthem. Existence of temporalrelationships, consistent results in independent studies, strongassociation, reliable exposure data,presence of dose-related responses, freedom from biases andconfounding factors, and high levelof statistical significance are among the factors leading toincreased confidence in a conclusionof causality.

Although statistical significance is mentioned as a factor inincreasing confidence in a conclusionof causality, this language does not make statistical significancea basic requirement fordetermining causality.

Finally, the proposed guidelines include Figure 2-1 entitled Human Evidence Factors is presented to summarize theabove-quoted section 2.6.1.(7).

Human Evidence Factors

Increase Weight
Number of independent studies with consistent results
Most causal criteria satisfied:
Temporal relationship
Strong association
Reliable exposure data
Dose response relationship
Freedom from bias and confounding
Biological plausibility
High statistical significance

Decrease Weight
Few studies
Equally well designed and conducted studies with null results
Few causal crieria satisfied
Figure 2-1. Factors for Weighing Human Evidence

According to the proposedguidelines, factors that increase theweight of evidence for a conclusion of causality include (1) the number ofindependent studies withconsistent results and (2) satisfying most causal criteria,including temporal relationship, strongassociation, reliable exposure data, dose repsonse relationship,freedom from bias andconfounding, biological pausibility, and high statisticalsignificance . Factors that decrease theweight of evidence determination, include few studies withconsistent results, equally welldesigned studies with null or negative results, few causalcriteria.

Figure 2-1 is misleading because listing "high statisticalsignificance" as increasing the weightof evidence determination may give the impression that statisticalsignificance is a requirement forcausality when, in fact, it is not. The figure is consistent with the text of theproposed guidelines which says only that"high statistical significance" increases confidence in aconclusion of causality. The text "fewcausal criteria satisfied" under the column "Decrease Weight"further indicates that statisticalsignificance is not a specific requirement for a conclusion ofcausality. That is, the absence ofstatistical significance does not necessarily hinder a conclusion ofcausality.

CONCLUSION

Contrary to the denial of EPA staff at the May 6, 1996 forum, theproposed guidelines do notinclude statistical significance as a requirement for determiningcausality based on epidemiologyin cancer risk assessment. The proposed guidelines also containlanguage about statisticalsignificance that misleads unsophisticated readers.

REFERENCES

1. 61 FR 17960 (April 23, 1996).
2. 51 FR 33992 (September 24, 1986).
3. NATIONAL ACADEMY OF SCIENCES. Environmental Epidemiology: PublicHealth and Hazardous Wastes. 1991. National Academy Press: Washington, D.C.
4. 61 FR, at 17974.
5. 61 FR, at 17974.
6. 61 FR, at 17981.
7. 61 FR, at 17982.

Material presented on this home page constitutes opinion of theauthor.

Copyright © 1996 StevenJ. Milloy. All rights reserved. Site developed and hosted by WestLakeSolutions, Inc.