Overview
What is Regulatory Science?
The origin of the term “regulatory science” is unknown. It was probably coined in the 1970s when the newly formed U.S. Environmental Protection Agency was forced to make decisions based on incomplete and sometimes non-existing scientific information. There were those who argued that there is no reason to coin a new phrase because “science is science” regardless of how it is applied. However, others argued that regulatory science is a scientific discipline much like many other applied scientific disciplines. During that period, there was increasing controversy dealing with nuclear power, pesticides, waste disposal, mine safety, chemical emissions, and numerous other issues. In order to address these issues, the need for a predictive process became apparent to evaluate options for new laws; development of implementing regulations; and dealing with numerous court cases.
Regulatory science is defined as a scientific discipline consisting of the development and application of scientific methods, tools, approaches, and other relevant processes derived from various scientific disciplines used in regulatory and other policy decisions. It is both interdisciplinary and multidisciplinary and relies upon virtually all scientific disciplines. A key characteristic of much of regulatory science is an attempt to predict future events. Laws and regulations dealing with food, drugs, environment, safety, economy, and many other human activities attempt to predict potential events and ensure that adverse consequences are avoided, or favorable conditions are promoted. Regulatory science also includes assessing the validity of scientific claims in courts and assists in numerous other policy decisions.
BEST AVAILABLE REGULATORY SCIENCE:
METRICS FOR EVALUATION OF REGULATORY SCIENCE CLAIMS
For several decades, the staff of the Institute for Regulatory Science has been involved in the development of a process to evaluate the validity of scientific claims. We chose the term Best Available Regulatory Science (BARS) to generically sort out the validity of claims and counter claims. These efforts led to the development of Metrics for Evaluation of Regulatory Science Claims (MERSC). The BARS concept uses five Principles for evaluation a scientific claim leading to three pillars.
BARS PRINCIPLES
Those who attempt to search the relevant literature would benefit from the availability of principles that provide fundamental guidance for the search process.
Open-mindedness and Skepticism: The Open-mindedness Principle complements the Skepticism Principle. Whereas the Open-mindedness Principle implies the willingness to consider new knowledge, the Skepticism Principle requires that those who make a scientific claim must provide sufficient evidence supporting their claim.
Ethical Rules: The key elements of this principle consist of Truthfulness, Transparency, Communicability, and Morality. One of the primary reasons for controversies during the regulatory process is the assumption of policy makers— including regulators—that the public is incapable of comprehending the complexities of various branches of science. If a scientific issue is complex, it is the responsibility of regulatory scientists to explain the subject to the public in a language that is understandable to its recipients.
Outside the Purview of Science: The concept of BARS deals with classification of scientific information and evaluation of scientific assertions. It specifically excludes areas that are outside of science. The inclusion of faith, ideology, societal objectives, beliefs, or any other non-scientific objectives in assessing the validity of scientific information is inconsistent with the foundation of BARS.
Reproducibility Principle: The ultimate proof of the validity of a scientific claim that it is reproducible by those who have the necessary competency and the needed equipment and facilities. This principle separates undisputed areas of science from those that include assumptions, interpretations, and in some cases, the inclusion of ideological and societal objectives in a scientific assertion.
PILLAR: CLASSIFICATION OF SCIENCE
This pillar identifies the level of maturity of Science published in scientific journals; reports prepared by government agencies; pronouncements of professional organizations; opinions expressed by advocacy groups; or any other scientific expression.
Proven Science
This group consists of scientific laws (sometimes called scientific principles); application of scientific laws; and virtually proven science.
Scientific Laws: Scientific laws are unequivocally confirmed and generally accepted. The cornerstone of this class is that any investigator who has the necessary skills and proper equipment can reproduce it. Similarly, this class of science does not require assumptions or any other prerequisite for its proof.
Applied Science: This class consists of application of scientific laws to various branches of commerce and industry. The only prerequisite for a scientific claim to be placed into this class is the correct interpretation of scientific laws. Much of the Engineering disciplines and a large segment of other applied sciences fall into this class.
Virtually Proven Science: This class consists of scientific information whose reliability has been uncontested yet there is insufficient proof to be placed in the proven science class. The process for placing science into this class consists of: 1) ensuring that no reliable observation contradicts information in this class and 2) there is no alternative system that describes the observed phenomena. Possible examples of this class include both general and special relativity theories. These theories attempt to describe the universe, forces operating in the universe, and their interrelationship.
Evolving Regulatory Science
The overwhelming majority of scientific advances in virtually all disciplines are evolving science.
Reproducible Evolving Regulatory Science: Reliable scientific assertions dealing with a subject that is not completely understood constitutes the core of this class. The key factor in placing a scientific assertion into this category is reproducibility. Although the observations that are the foundation of this class may not necessarily have general applicability, they are clearly and unambiguously reproducible by those with appropriate skills and equipment. Advancements in virtually all branches of science including physics, chemistry, biology, and many other scientific disciplines are based on the desire of investigators to improve knowledge in this class.
Partially Reproducible Evolving Regulatory Science: The scientific foundation of scientific claims in this class is derived from proven or reproducible evolving regulatory science. However, it uses assumptions, extrapolations, and other similar processes in deriving its results and conclusions. A characteristic of this class is that the investigator who is trying to reproduce this class must not only have proper skills and the necessary equipment, but must also accept the asserted scientific foundation, assumptions, choice of mathematical processes, default data, and numerous other prerequisites. Most predictive models, including mathematical models and a large segment of contested scientific information, fall into this class.
Association-Based Evolving Regulatory Science: Sometimes called correlation or observation studies, this class is not based on Proven or Reproducible Evolving Science. Instead, often an investigator attempts to correlate two groups such as two populations with different characteristics hoping to correlate an observed difference between the two populations to a cause. Consequently, the level of reproducibility of information in this class ranges from unknown to reasonable.
Hypothesized Regulatory Science: This class consists of an organized response to an observation, an idea, or any other initiating thought process. Experience shows that although many great scientific discoveries started with this class, there is also a long list of hypothesized claims that have proven to be either wrong or not worth pursuing.
Borderline Regulatory Science
As the title implies, Borderline Regulatory Science is exactly that and should be used accordingly.
Judgment: On occasion, information is provided to the society that lacks scientific foundation. Similarly, often decisions must be made without having the needed scientific information including basic principles, the necessary data, and other scientific requirements. The methodology for expert judgment is reasonably well-developed and consists of asking a number of presumably knowledgeable individuals to give answers to specific questions and statistically assess the results. However, this class is often an educated guess.
Speculation: This class consists of information that cannot meet standards described in any of the above classes. It is often based on the intuition of an individual who wants to stimulate a discussion or initiate a research project.
Fallacious Information
Information that clearly falls into the purview of science but is inconsistent with the three groups of classes identified above is fallacious information. This information is often called “pseudoscience”, “junk science”, or “politically-processed science”.
PILLAR: CATEGORIZATION OF RELIABILITY OF REGULATORY SCIENCE
There are rational and reasonable uncontested methods to resolve scientific controversies. Briefly, scientific claims or assertions are divided into four distinct categories. For the sake of simplicity, in the following the description of each group starts from the bottom of the figure.
Personal Opinions: Expression of views by individuals regardless of their training, experience, and social agenda are included in this group. Personal opinions are seldom, if ever, BARS. At best, this category can be used to initiate the study of a scientific issue. Note that the standard process of the news media is reliance upon this category in its reporting of scientific issues.
Gray Literature: This is the favorite category of government agencies, advocacy groups, and individuals who want to promote an idea. In fact, this category is the more organized and written form of personal opinions. Again here, at best, this category should be used to initiate a study. Experience shows that in most cases this category does not meet the requirements of scientific acceptability.
Peer Reviewed Regulatory Science: A scientific claim subjected to an independent peer-review constitutes this category. Peer review is the foundation of scientific acceptability. There are numerous requirements for the acceptability of peer review. Briefly, the individual who is chosen as a reviewer must be a “peer” to the author of the study and must have no conflict of interest. In addition, the author of the study must respond to criticism by the peer to the satisfaction of an uninvolved person or organization.
Verification Processed Regulatory Science: This category addresses the reproducibility of peer-reviewed claims. The peer-review process, as used in scientific publishing, is not perfect and a reevaluation of certain peer-reviewed studies has identified a lack of reproducibility. The verification process attempts to ensure the reproducibility of a peer-reviewed regulatory science.
Consensus-Processed Regulatory Science: This category uses the consensus process providing an opportunity to resolve scientific disputes. The prerequisite for this process is identical to the process used for independent peer review.
Regulatory Science Processes and Tools
One of the reasons for the development of the concept of Best Available Regulatory Science (BARS) and Metrics for Evaluation of Regulatory Science Claims (MESRC) derived from BARS was to respond to the needs of regulatory science. Based on BARS/MERSC, regulatory science overwhelmingly relies upon Partially Reproducible Evolving Regulatory Science, Evidence-Based Regulatory Science, and Hypothesized Regulatory Science. On occasion, it also uses Scientific Judgment and Speculation.
An increasingly recognized and important tool of regulatory science is independent peer review. Due to the nature of regulatory science, it is imperative that both the scientific foundation and non-scientific inputs that are evaluated by reviewers who are qualified and have no conflict of interest—based on review criteria that are responsive to the needs of stakeholders of regulatory science. Furthermore, transparency is a key factor in the acceptability of the peer-review process.
Regulatory Science Ethics
There are ethical principles that members of all professions must follow. In addition, certain professions have ethical requirements that are unique. The unique regulatory science ethical requirements include the following:
Finality Principle: The media, certain policy makers, many advocacy organizations, and others claim that an issue is settled; the science has spoken; the toxicity of agent is scientifically proven; science has established the benefit of a medical procedure; and numerous others. The regulatory science ethics finality principle is as follows:
A scientific issue is only then settled if any investigator who has the necessary skills and, depending upon the nature of the issue, relevant equipment and facilities is capable to reproduce and confirm it.
Transparency Principle: Also known as Jeffersonian Principle is based on the notion that regulatory science relies upon assumptions, judgments, and default data. The authors of regulatory science must provide justification for using them and provide potential alternatives that were not used. The regulatory science ethics transparency principle is as follows:
Regulatory science information must include assumptions, judgments, and similar parts in a language understandable to a
knowledgeable non-specialist.
Confirmation Principle: As previously stated, there is a consensus within the scientific community that independent peer review is a prerequisite for acceptability of scientific claims. The regulatory science ethics confirmation principle requires that:
Regulatory science information is only then acceptable if it has been subjected to independent peer review and the review criteria (questions provided to peer-reviewers) include compliance with the Transparency Principle of regulatory science ethics.