EPA Lacks the Authority and Supporting Data to Propose Transparency Requirements for Science

Acting Administrator Andrew Wheeler
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, D.C. 20460

Submitted to: http://www.regulations.gov

Subject: Comments on Proposed Rule, “Strengthening Transparency in Regulatory Science,” 83 Fed. Reg. 18768 (April 30, 2018), Docket ID No. EPA-HQ-OA-2018-0259

Dear Acting Administrator Wheeler:

The Project On Government Oversight (POGO) provides the following public comment about the Environmental Protection Agency’s (EPA) proposed rule, “Strengthening Transparency in Regulatory Science,” published on April 30, 2018.¹ As an independent nonprofit organization committed to achieving a more effective, ethical, and accountable federal government, POGO has an interest in ensuring that the EPA follows its legal obligations for the use of scientific evidence in rulemaking, adheres to all appropriate steps of the rulemaking process, and continues to issue and strengthen sound public protections under its statutory obligations. Because this rule fails in each of these regards and would cause the EPA to fail in many future rulemakings going forward if put into effect, POGO expresses its strong objections to the proposed rule and urges the EPA to withdraw it.

The proposed rule notes that “the best available science must serve as the foundation of EPA’s regulatory actions” and uses the words “transparency” and “reproducibility” to project lofty goals. But, instead of making scientific evidence more available or easier to use, the rule will often mean the best available science is off limits to the Agency. Its real effect will be to undermine the way that the EPA is able to rely on and even-handedly assess scientific studies for use in the rulemaking process.

The rule lacks a purpose and scientific basis

This proposed rule presents no clear explanation or examples of the types of problems it is seeking to solve

This rule lacks a fundamental statement of its purpose or of the problems that it purports to address, the central element of any proposed rule. In addition to offering no clear explanation of any problem, the proposal provides no supporting evidence, no studies establishing that the EPA has an information problem, nor any citations that the proposed standard has ever been used before or that the EPA understands what its impact will be when implemented. This lack of a statement of purpose reflects the wholly insufficient development process that produced this rule, which, as is described below, originated without input from key stakeholders inside and outside of the EPA.

If the EPA does believe there is a real problem, it should be able to provide some example of a scientific study that has been used during rulemaking that does somehow substantively lack transparency or fails some standard for reliability. Inclusion of such examples are necessary in a proposed rule so that commenters can debate those examples. By failing to include any past or present cases that might necessitate its proposed rule, we are left to conclude that there is no clear purpose for the EPA’s proposal.

There is no systematic analysis of the use of scientific studies in rulemaking that provides a basis for this rule

Proposing a rule that will fundamentally change what information can be used in future rulemakings is a major undertaking and requires a great deal of certainty and evidence. Given the complete lack of evidence provided in this case, this proposed rule is premature even if the Agency truly believes there is some deficiency in the policies and procedures governing use of information in rulemakings. Before proposing any rule, but especially one that is this foundational to future rulemaking, the Agency should start by conducting studies to better understand the scope of the problem, if there is one, and the best way to improve its use of scientific studies. Without such a study, the EPA has provided no evidence to support the claim that there is an issue with the “transparency of EPA regulatory science” or that there is a need for the public to be able to “replicate findings,” as the rule suggests.

This type of study should go hand-in-hand with an evaluation of the rule and its supporting evidence by the EPA’s Science Advisory Board (SAB). In this case, to appropriately assess the scientific claims being made, the SAB should be allowed to fully investigate and offer specific recommendations on the rule. In fact, the SAB itself has said that the rule “deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board.”²

In fact, scientific studies are already thoroughly evaluated under the current rulemaking process

As is described below, this rule’s implementation will place large portions of scientific research off-limits during EPA rulemaking. Instead of arbitrarily excluding broad types of studies from being cited in rulemaking, why not continue to give Agency scientists the ability, as they have had for decades, to comprehensively assess and compare the scientific evidence presented in a study and give weight to each study as a result of careful deliberation?

During the rulemaking process, EPA officials already decide if studies are unreliable or flawed based on the studies’ own merits—and sometimes even flawed studies can offer important insights that the EPA should benefit from. For each rule, the Agency is already required to fully explain its reasoning and the studies relied on, offer dockets of supporting information, and have a public comment period. This notice-and-comment process already allows outside stakeholders to raise concerns or problems with the science used or offer alternative studies. The Agency has to consider and respond to those comments, which commonly occurs in the form of an extensive explanation that accompanies the final rule in the Federal Register.

A letter from the chief editors of six of the major scientific journals explains this process of evaluating studies, even when data cannot be made public:

“The merits of studies relying on data that cannot be made publicly available can still be judged. Reviewers can have confidential access to key data and as a core skill, scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results.”³

The rule fails to explain its two key requirements for the use of studies in rulemaking

The rule fails to properly define the two key requirements that will have a major impact on how it is implemented: 1) how to anonymize sensitive data for public release and 2) the distinction between replicability and reproducibility and how either precisely applies to scientific studies.

Without knowing the details of how these transparency and replicability provisions, central to the rule, will be implemented, commenters can’t even begin to assess the wide-ranging outcomes of this rule. Even ignoring the fact that this rule provides no statement of purpose, as described above, or that it was created with significant procedural shortcomings, as described below, the fact alone that it is impossible to provide substantive comment is sufficient reason for this rule to be withdrawn.

The rule provides only a vague description of how to anonymize data

First, the rule states that data relied on in making regulations must be made publically available, but there are a variety of valid reasons researchers don’t publish all the underlying data—personally identifiable information and confidential business information being among the biggest concerns.

The scientific community itself acknowledges that not all data can be made public. The letter from the six chief editors explains the sharp limits on transparency, stating that “in not every case can all data be fully shared. Exceptional circumstances, where data cannot be shared openly with all, include data sets featuring personal identifiers.”⁴

Given the range of studies and information that would be affected by the proposed rule, the Agency would need numerous and complicated processes to ensure that data was properly anonymized. The EPA’s proposed rule claims there are ways to mask data to ensure privacy is protected, but fails to provide any details or specifics for how such a process would be implemented—this is not a simple issue of redacting a few data fields. But instead of providing specific steps for how this process would be handled so that commenters could provide input, the rule is all but silent on this issue.

Some scholars have explored ways to better anonymize data in scientific studies, but those efforts are not foolproof. Even when personal identifying information is removed from data, it can be possible to identify individuals in the right circumstances from a combination of simple data points.⁵ The most effective way to protect personal privacy, then, is to not publish the detailed data underlying these studies at all. In these cases, even though the studies have been conducted by reputable researchers at academic institutions, and peer reviewed to ensure validity, they would ultimately be unavailable to Agency officials as evidence in rulemakings.

The rule fails to differentiate meaningfully between reproducibility and replicability

The second key consideration that the proposed rule fails to address is a concrete definition for what it means for information that “includes the information necessary for the public to understand, assess, and replicate findings,” which is the standard the rule attempts to establish for information that is considered “publicly available in a manner sufficient for independent validation.” Besides a vague list containing items that may be included in this type of publically available and replicable information (“data,” “associated protocols,” “computer codes and models involved in the creation and analysis of such information,” “recorded factual materials,” and “detailed descriptions of how to access and use such information,”), no further description of what it means to “replicate findings” is given.

Confounding matters, while the statement of the rule itself refers to replicability of scientific findings, the background information supporting the rule focuses on scientific studies’ “reproducibility,” which has a wholly different meaning in a scientific context. While the definitions of these terms continue to be debated by scientists, which further demonstrates the difficulty in how the EPA has used them, there is broad consensus:⁶ a study is commonly defined by scientists as replicable if its findings can be obtained again through conducting a new, independent study, whereas a study is typically defined as reproducible if reanalysis of data collected during that study, using the same or similar methods, produces the same findings.

The vast disparity in these definitions, and the fact that both terms are mentioned multiple times between the proposed rule and its supporting information, leaves us to guess what the intent of the rule really is, which means commenters simply can’t interpret how this rule will be implemented. But, because the rule itself says it must be possible to “replicate” studies’ findings, we should assume that the rule may intend the strongest possible meaning: that it must genuinely be possible to conduct all studies used in rulemaking again, from scratch, and obtain the same findings. As we explain below, this then establishes a standard that would preclude an enormous quantity of studies form being used in the rulemaking process.

The rule will undermine the use of scientific evidence in rulemaking

Scientific studies that could inform rulemaking will be thrown out

Essentially, the proposed rule would require that the Agency only use studies for which the underlying data is fully public or whose findings can be replicated in their entirety. So it’s reasonable to conclude that, if the rule goes into effect, the EPA will no longer be able to use a large portion of the studies that it currently relies on, including important longitudinal human health studies, to craft public safeguards. Major health studies often collect large amounts of information about the people who agree to participate and there are laws, like the Health Insurance Portability and Accountability Act of 1996,⁷ that strictly prohibit sharing a person’s medical information.

In the letter from the six major scientific journals,⁸ after the editors raise concerns about limiting scientific evidence, they also conclude that “excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes.”

The Agency also uses many studies, such as those that link living in proximity to an airport to toxic blood lead levels in children⁹ or studies that found a link between fine particulate air pollution and premature deaths,¹⁰ that cannot be repeated, because they were based on environmental disasters or major exposures to toxic substances. Just because they can’t—or shouldn’t—be repeated, however, doesn’t mean we should ignore the vital insights they provide. The knowledge we have gained from these tragedies can and should be used to help safeguard the public in the future.

Instead, banned from being allowed to make use of the vast wealth of scientific evidence based on human subjects, Agency officials will be left with studies that don’t have any personal privacy concerns, such as industry studies that often rely on animal test subjects.¹¹

The rule will put the EPA in the position of setting standards for studies, significantly reducing the number of studies the EPA can rely on

The rule’s constraints on the use of scientific studies mean that even the use of studies that don’t end up being haphazardly tossed out by this rule will be hindered substantially.

The rule also puts the Agency in a position in which it’s forced to serve as an independent reviewer of all scientific data underlying studies it uses, effectively having to peer-review these studies, which will severely hamstring Agency scientists, who already have limited resources. When the EPA was sued over air quality standards for particulate matter and ozone during the George W. Bush administration, the U.S. Court of Appeals for the District of Columbia Circuit said a requirement to make public the underlying data for the key studies used in the rulemaking process would be “impractical and unnecessary.”¹²

The three judge panel concluded that, “if EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment ...”

The Congressional Budget Office (CBO), in response to the HONEST Act of 2017,¹³ a piece of legislation with very similar provisions to the proposed rule, has said that this type of policy, without a major funding commitment, would significantly reduce the number of studies that the EPA is able to rely on when proposing rules.¹⁴

If the EPA wants to address the accessibility of scientific studies and data, an important issue to scientists as well as members of the public, it should acknowledge that those efforts, which might include building a new public-facing platform or carefully considering certain types of standards, will amount to a years-long process and will require an enormous investment of Agency time and funding. That type of proposal shouldn’t be made in a brief proposed rule, however, and should only be made, as described above, if extensive studies demonstrate that there is a real need for an update to how scientific studies are used in Agency rulemaking.

The process for creating this rule was severely flawed and will result in procedural issues for future rules

There is no statutory authority for this rule

The EPA is proposing this rule without any clear statutory authority from Congress. Agencies are not permitted to create new laws or requirements unless duly authorized by Congress. While an agency has authority in its given issue area, which, in the case of the EPA, is protecting the environment, that authority is not absolute.

The EPA claims that its authority for this rule stems from “provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions” under a number of environmental laws. This is a grave misinterpretation of the Agency’s authority under these laws, as none of these laws require or mention transparency requirements for scientific studies. Agencies do offer new regulations or update existing ones under the authority of long-standing statutes, but these are done because of changes in technology, science, or law that then require new rules to properly enforce the original intent of the statute. But this proposal to regulate what counts as usable science during rulemaking is far removed from the intent Congress had in passing laws about keeping our air and water clean and protecting the public from hazardous chemicals.

In fact, this proposal would directly contradict requirements in several of the laws cited by the Agency that instruct the EPA to consider available science in rulemakings. For instance, the Safe Drinking Water Act directs the EPA to base its determination about whether to regulate any particular contaminant “on the best available public health information.”¹⁵ Additionally, the Toxic Substances Control Act requires the EPA to take regulatory action “consistent with the best available science.”¹⁶

The rule violates the Administrative Procedure Act¹⁷

The Agency also seems to claim it derives some authority from “requirements in the Administrative Procedure Act (APA) to ensure public participation in the rulemaking process.” However, that is again an overly broad interpretation. Federal agencies have overseen public participation in rulemakings for years. The proposed rule would not improve the key public participation components such as rulemaking disclosures or the notice and comment process.

If anything, the rule is in violation of the APA, which makes it clear that an agency can not engage in arbitrary and capricious actions or decisions in rulemakings. The Agency must have clear and strong justification for actions taken in a rulemaking. Given the lack of supporting evidence or statutory requirement for this policy, the EPA will be hard pressed to prove that this untested standard for scientific transparency is not arbitrary.

In fact, if the rule were put into effect, it could undermine future rulemakings by the EPA. Many of the proposed rules using these standards could be challenged in court and deemed “arbitrary and capricious” because they exclude relevant data and studies for failing to meet poorly established data transparency requirements. Additionally, if a commenter referred substantively to a study that the EPA was unable to use because of the requirements from this proposed rule, the Agency’s failure to fully consider the comment and the referenced study could also cause the rule to be deemed “arbitrary and capricious.”

The proposed rule gives the Administrator alone discretion to exempt future rulemakings from this rule “on a case-by-case basis if he or she determines that compliance is impracticable,” either because scientific data underlying the rule cannot be made appropriately publicly available or because a review of the science cannot be conducted in accordance with cited guidance from the Office of Management and Budget. Because the rule does not provide any mechanism for evaluating if studies should be exempted from the rule’s requirements, however, there is no reason to conclude that the Administrator will make case-by-case exemptions appropriately and there is no way to prevent exemptions from be granted arbitrarily.

The rule should be withdrawn

In conclusion, POGO finds the EPA to be without sufficient authority to propose this rule and the proposed rule itself to be incomplete, ill-considered, and contrary to the Agency’s mission to protect the public and environment. Therefore, we again urge the EPA to withdraw this rule.

We appreciate your consideration and attention to this matter. If you have questions or need additional information, please contact us at 202-347-1122 or [email protected].

Sincerely,

Sean Moulton
Senior Policy Analyst

Andrew Bergman
Special Environmental Advisor