2021-11637. Evidence-Building and Evaluation Policy Statement

AGENCY:

Nuclear Regulatory Commission.

ACTION:

Policy statement; issuance.

SUMMARY:

The U.S. Nuclear Regulatory Commission (NRC) is issuing an Evidence-Building and Evaluation Policy Statement that describes the general standards that guide the NRC's “evidence-building” activities, consistent with the Foundations for Evidence-Based Policymaking Act of 2018. The policy statement is intended to provide agency personnel and stakeholders with a clear understanding of the expectations related to the NRC's standards for evidence-building activities, which includes analyses, research, assessments, and evaluations performed by the agency for programmatic, operational, regulatory, and policy decision making. These standards include rigor, relevance and utility, transparency, collaboration, independence and objectivity, and ethics.

DATES:

This policy statement is effective on June 3, 2021.

ADDRESSES:

Please refer to Docket ID NRC-2020-0262 when contacting the NRC about the availability of information for this action. You may obtain publicly-available information related to this action by any of the following methods:

• Federal Rulemaking Website: Go to https://www.regulations.gov and search for Docket ID NRC-2020-0262. Address questions about NRC dockets to Dawn Forder; telephone: [phone]; email: [e-mail]. For technical questions contact the individual listed in the FOR FURTHER INFORMATION CONTACT section of this document.
• NRC's Agencywide Documents Access and Management System (ADAMS): You may obtain publicly-available documents online in the ADAMS Public Documents collection at https://www.nrc.gov/​reading-rm/​adams.html. To begin the search, select “Begin Web-based ADAMS Search.” For problems with ADAMS, please contact the NRC's Public Document Room (PDR) reference staff at 1-800-397-4209, at 301-415-4737, or by email to [e-mail]. The final Evidence-Building and Evaluation Policy Statement, in its entirety, is in the attachment to this document.
• Attention: The PDR, where you may examine and order copies of public documents is currently closed. You may submit your request to the PDR via email at [e-mail] or call 1-800-397-4209 between 8:00 a.m. and 4:00 p.m. (EST), Monday through Friday, except Federal holidays.

FOR FURTHER INFORMATION CONTACT:

Matthew Meyer, Office of the Executive Director for Operations, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, telephone: [phone], email: [e-mail].

SUPPLEMENTARY INFORMATION:

I. Background

The Foundations for Evidence-Based Policymaking Act of 2018 (“Evidence Act”) became law on January 14, 2019 (Pub. L. 115-435), to enhance evidence-building activities, make data more accessible, and strengthen privacy protections.^[1] “[T]he Evidence Act creates a new paradigm by calling on agencies to significantly rethink how they currently plan and organize evidence-building, data management, and data access functions to ensure an integrated and direct connection to data and evidence needs.” ^[2] The Evidence Act requires each agency to name an Evaluation Officer. At the NRC, the Director of the Office of Nuclear Regulatory Research holds this position and must “establish and implement an agency evaluation policy” to fulfill a primary function of this position.^[3] The agency evaluation policy “should guide the agency's activities throughout the evaluation lifecycle.” ^[4] The Office of Management and Budget (OMB) has provided guidance on establishing an agency evaluation policy based on “approaches that Federal agencies have found useful.” ^[5] This guidance includes “[e]nsuring that the agency evaluation policy incorporates the evaluation standards” recommended by OMB.^[6] OMB developed these evaluation standards through an interagency council that “reviewed an extensive list of source documents to identify widely accepted standards for evaluation.” ^[7] The interagency council identified the following evaluation standards: relevance and utility, rigor, independence and objectivity, transparency, and ethics.^[8]

The Evidence Act focuses on the importance of sound evidence-building, which includes evaluation, to make informed evidence-based decisions. The evaluation standards developed by the interagency council, including an additional standard developed by the NRC (collaboration), are applicable to all of the NRC's evidence-building activities.

Historically, the NRC has relied on high-quality evidence obtained from external entities and through its own capacity. In recent years the agency has begun evidence-building activities to support licensing new or novel nuclear technologies, including advanced, non-light water reactor designs; accident tolerant nuclear fuel; and digital instrumentation and controls.^[9] Additionally, the NRC has increasingly sought to rely on evidence-based metrics to improve internal agency performance including budgeting and financial management.^[10] The NRC has developed an evidence-building and evaluation policy statement to enhance its existing evidence-building activities through the activities directed in the Evidence Act. The NRC envisions that this approach will strengthen the agency's oversight of existing uses of nuclear technology, enhance the agency's readiness to license and regulate new and novel nuclear technologies, and further the NRC's ongoing efforts to improve its internal processes.

II. Public Comments

The NRC published the Proposed Evaluation Policy Statement in the Federal Register for a 30-day comment period on December 8, 2020 (85 FR 79042). The NRC received a total of nine public comments.

These comments were generally supportive of the policy statement and the NRC's commitment to ensuring that its regulatory decisions are supported with evidence and sound technical bases. However, commenters also requested that the NRC clarify the applicability of the policy statement to evidence-building activities other than “evaluation” as that term is defined in the Evidence Act (5 U.S.C. 311(3)), such as licensing, inspection, rulemaking, generic communication, and other regulatory activities (including backfitting analyses, and environmental reviews performed under the National Environmental Policy Act). The NRC agrees and has revised the proposed policy statement to clarify that the general standards articulated in the policy statement apply to all agency “evidence-building” activities. This includes not only “evaluations” conducted to review the effectiveness and efficiency of NRC programs, policies, and organizations, but other types of evidence-building such as regulatory analyses, compliance analyses, and performance assessments. A complete table of the comments received on the proposed policy statement and NRC staff responses to those comments is available in ADAMS under Accession No. ML21070A196.

III. Procedural Requirements

Congressional Review Act

This policy statement is not a rule as defined in the Congressional Review Act (5 U.S.C. 801-808).

Paperwork Reduction Act

This Policy Statement does not contain new or amended information collection requirements and, therefore, is not subject to the Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.).

The text of the Evidence-Building and Evaluation Policy statement is attached.

Dated: May 28, 2021.

For the Nuclear Regulatory Commission.

Wesley W. Held,

Acting Secretary of the Commission.

Note:

The following attachment will not appear in the Code of Federal Regulations:

Attachment—Evidence-Building and Evaluation Policy Statement

The purpose of this Evidence-Building and Evaluation Policy Statement is to describe the general standards that govern the NRC's planning and conduct of evidence-building. Evidence-building includes activities such as analysis, assessment, research, and program evaluation (evaluation).^[11] The Foundations for Evidence-Based Policymaking Act of 2018 requires an agency evaluation policy to guide the agency's evaluation activities throughout the evaluation lifecycle. The NRC is committed to using evidence and scientific methods when making evidence-based decisions.

The NRC is an evidence-based organization with a culture of continuous learning and improvement. The NRC's evidence-building activities use objective technical analyses and assessments to document decisions with explicitly stated rationale. Furthermore, the NRC commits to implementing the standards of rigor; relevance and utility; transparency; collaboration; independence and objectivity; and ethics in the conduct of its evidence-building activities. This policy statement describes these general standards.

The Commission, as a collegial body, formulates policies, develops regulations governing nuclear reactor and nuclear material safety, issues orders to licensees, and adjudicates legal matters. The collegial decision-making process results in actions reflecting the collective judgment of a group aided by professional and administrative staff and advisory committees, such as the Advisory Committee on Reactor Safeguards. Strict requirements govern the admission and consideration of “evidence” when the Commission acts in its adjudicatory capacity. This policy applies to the NRC's non-adjudicatory functions.^[12]

The NRC's Principles of Good Regulation, which include independence, efficiency, clarity, reliability, and openness, have guided the agency's regulatory activities and decisions using evidence and scientific methods. The principles focus on meeting the agency's important safety and security mission while appropriately considering the interests of stakeholders, including licensees; State, local, and Tribal governments; nongovernmental organizations; and the public. The agency's openness principle explicitly recognizes that the public must be informed about and have an opportunity to participate in the regulatory process.

Evidence-building is used to inform agency activities and actions, such as licensing, oversight, budgeting, program improvement, accountability, management, rulemaking, guidance development, and policy development. The emphasis on evidence is meant to support innovation, improvement, and learning. Examples of how the NRC carries out evidence-building include (1) identifying, evaluating, and resolving safety issues; (2) ensuring that an independent technical basis exists to review licensee submittals; (3) evaluating operating experience and results of risk assessments for safety implications; (4) supporting the development and use of risk-informed regulatory approaches; (5) conducting research with scientific integrity; and (6) ensuring that licensing and oversight findings are supported by evidence.

Evidence-Building Standards

The NRC uses the following standards when conducting evidence-building activities.

1. Rigor—The NRC is committed to using rigorous evidence-building methods by qualified staff with relevant education, skills, and experience to ensure findings are appropriate and feasible within statutory, budgetary, and other constraints.

Rigorous evidence-building requires inferences about cause and effect to be well founded (internal validity); clarity about the populations, settings, or circumstances to which results can be generalized (external validity); and the use of measures that accurately capture the intended information (measurement reliability and validity). The NRC's evidence-building activities are conducted by qualified staff with relevant education, skills, and experience for the methods undertaken. The NRC's evidence-building activities use appropriate designs and methods that adhere to widely accepted scientific principles to answer key questions while balancing goals, scale, timeline, feasibility, and available resources. Additionally, the NRC's Information Quality Program ^[13] ensures that all information relied on by the NRC is subject to rigorous quality standards.

2. Relevance and Utility—The NRC ensures that evidence-building activities are relevant and provide useful findings to inform agency activities, actions, and stakeholders.

The NRC performs evidence-building activities to examine questions of importance and serve the information needs of stakeholders. The NRC presents findings that are clear, concise, actionable, and available within a timeline that is appropriate to the questions under consideration. The NRC's evidence-building priorities consider legislative requirements; the NRC's strategic goals, objectives, and strategies; and the interests and views of stakeholders.

3. Transparency—The NRC is committed to conducting evidence-building activities in an open and transparent manner, which keeps stakeholders informed.

The NRC's evidence-building activities are conducted openly and the public must be informed about and have an opportunity to participate in the NRC's regulatory process. As a regulator, the NRC listens to, respects, and analyzes different views from its stakeholders. The NRC ensures open channels of communication are maintained between internal and external stakeholders, including Congress, other government agencies, licensees, nongovernmental organizations, individual members of the public, and international and domestic nuclear communities. The NRC takes reasonable measures to make all information, including information about the NRC's evidence-building activities (including their purpose, objectives, design, findings, and methods), broadly available and accessible. The NRC releases public evidence-building findings in a timely manner and archives the data for secondary use by stakeholders, as appropriate.

4. Collaboration—The NRC is committed to working collaboratively when conducting evidence-building activities and draws on the expertise of subject matter experts to ensure diversity in perspectives.

The NRC fosters a collaborative work environment that encourages diverse views, alternative approaches, critical thinking, creative problem solving, unbiased findings, and honest feedback. The NRC emphasizes trust, respect, and open communication to promote a positive work environment that maximizes the potential of all individuals, which improves evidence building and evaluation activities. A collaborative environment leverages expertise from subject matter experts and enables peer reviews to ensure rigorous evidence-building. The NRC conducts research and collaborates with organizations that develop consensus standards to improve data and methods used in risk analysis. The NRC collaborates with national laboratories, Agreement States, other Federal agencies, universities, and international organizations.

5. Independence and Objectivity—As an independent Federal agency, the NRC is committed to conducting evidence-building activities that are independent and based on objective assessments and analysis of all relevant information.

The NRC was established as an independent agency to regulate civilian uses of radioactive material. The NRC's evidence-building activities are independent and objective to maintain credibility and integrity. The implementation of evidence-building activities, including the selection and assignment of the staff, should be appropriately insulated from factors that may affect objectivity, impartiality, and professional judgment. Evidence-building is inclusive and the NRC seeks input from a broad range of stakeholders in setting priorities, identifying questions, and assessing the implications of findings. The NRC strives for objectivity in the planning and conduct of evidence-building activities.

6. Ethics—The NRC is committed to conducting evidence-building activities that adhere to Government-wide ethics standards to protect the public and maintain public trust.

The NRC's evidence-building activities comply with relevant legal requirements and are conducted in a manner that is free from conflicts of interest, undue influence, the appearance of bias, and safeguards the dignity, rights, safety, and privacy of participants. The NRC complies with Governmentwide ethics standards contained in Federal statutes and regulations, which are intended to ensure that every citizen can have confidence in the integrity of the Federal Government.

Footnotes

1.  Public Law 115-435, 132 Stat 5529 (2019).

2.  Office of Management and Budget, M-19-23, “Phase 1 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Learning Agendas, Personnel, and Planning Guidance,” 2 (July 10, 2019).

3.  5 U.S.C. 313(d)(3).

4.  Office of Management and Budget, M-20-12, “Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices,” Appendix C (March 10, 2020) (M-20-12).

5.  M-20-12, Appendix C.

6.  Id.

7.  Id. at 2.

8.  Id. at 3-5.

9.  Nuclear Regulatory Commission, NUREG-1350, 2019-2020 Information Digest, at 4 (August 2019).

10.  Id. at 7.

11.  The Evidence Act defines “evaluation” as “an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their effectiveness and efficiency” (5 U.S.C. 311(3)). “Evaluation can look beyond the program, policy, or organizational level to include assessment of projects or interventions within a program” (OMB M-20-12).

12.  The NRC's rules of practice and procedure in 10 CFR part 2 govern the Commission's adjudicatory process.

13.  Management Directive 3.17, “Information Quality Program,” ensures that peer review is conducted on all influential scientific information and highly influential scientific assessment that the agency intends to disseminate.