2024-19041. Request for Information (RFI) on National Science Board-National Science Foundation Merit Review Commission Review of NSF's Merit Review Policy and Processes

Document Headings

Document headings vary by document type but may contain the following:

1. the agency or agencies that issued and signed a document
2. the number of the CFR title and the number of each part the document amends, proposes to amend, or is directly related to
3. the agency docket number / agency internal file number
4. the RIN which identifies each regulatory action listed in the Unified Agenda of Federal Regulatory and Deregulatory Actions

See the Document Drafting Handbook for more details.

National Science Foundation

AGENCY:

National Science Foundation

ACTION:

Request for information.

SUMMARY:

The National Science Board-National Science Foundation Commission on Merit Review (MRX) is issuing this Request for Information (RFI) to seek input from interested individuals and parties to inform the MRX's review of NSF's Merit Review criteria, policy and processes.

Information on the MRX is available at https://www.nsf.gov/​nsb/​committees/​mrxcmte.jsp.

DATES:

Interested individuals and parties are invited to submit responses to this Request for Information on or before 5:00 p.m. Eastern time on Wednesday, September 18, 2024.

ADDRESSES:

Online: Respond to this RFI at the following url: https://nsfevaluation.gov1.qualtrics.com/​jfe/​form/​SV_​6xOeZ04jar2xmhU.

Following this link allows you to access an online form where you can provide input on up to six topics described in more detail in the Supplementary Information section below. You are encouraged to respond to only those that are of interest to you. You may, but are not required to, provide input on each topic to submit your response.

Mail: Attn: Portia Flowers, 2415 Eisenhower Avenue, Alexandria, VA, 22314, USA.

FOR FURTHER INFORMATION CONTACT:

Portia Flowers (703/292-7000, [e-mail]).

SUPPLEMENTARY INFORMATION:

Please refer to definitions provided at the end of this notice for terms used in these Information Requests.

Information Requests

1. MRX is interested in identifying opportunities to improve NSF's current Merit Review criteria, policy, and processes. Importantly, this includes documenting and understanding any areas of misunderstanding, gaps, or lack of clarity regarding (a) the three Merit Review Principles which are the foundations of the Merit Review Process, (b) the two statutory Merit Review Criteria which are used to evaluate all proposals to NSF, and (c) the five Merit Review Elements NSF uses to assess each criterion. Are the Principles, Criteria, and Elements clear? Could they be improved upon? The MRX welcomes feedback on any or all of these, and particularly on the Broader Impacts Criterion. Chapter 3 of NSF's Proposal & Award Policies and Procedures Guide (PAPPG) defines terms in this Information Request. See https://new.nsf.gov/​policies/​pappg/​24-1/​ch-3-proposal-processing-review#a-merit-review-principles-and-criteria-af2.

Individuals responding to this request are encouraged to indicate whether their perspectives are informed by experience(s) preparing and/or reviewing proposals to NSF.

2. NSF strives to conduct a fair, competitive, transparent Merit Review process for the selection of projects. To accomplish this, NSF relies on a process that considers both the technical aspects of a proposed project and its potential to contribute more broadly to advancing NSF's mission using the statutory Intellectual Merit and Broader Impacts Merit Review criteria. MRX invites suggestions on the implementation of the Merit Review criteria. We especially invite feedback that would (a) clarify how they can be used in preparing and reviewing proposals, (b) ensure proposals, reviews, and funding decisions demonstrate full consideration of both criteria while maintaining openness to the full spectrum of potential activities under each, and (c) better recognize and support potentially transformative and high-risk/high-reward activities.

Individuals responding to this request are encouraged to indicate whether their perspectives are informed by experience(s) preparing and/or reviewing proposals to NSF.

3. MRX is interested in the experiences and perspectives of those who have considered submitting and/or submitted proposals in the past. We invite you to share your insights and describe any opportunities you believe would improve implementation of the Merit Review criteria, policy, and processes based on your experience as a proposer or investigator. This includes any experiences that may have encouraged or dissuaded you from submitting proposals to NSF. We are especially interested in learning (a) how NSF guidance ( e.g., as provided in the NSF PAPPG, program solicitations, or other funding opportunity announcements), may have played a part in your decision(s) whether to submit proposals, and (b) how NSF might best support investigators interested in submitting a proposal to NSF.

Individuals responding to this request are encouraged to indicate whether they ( print page 68474) submitted or decided not to submit a proposal, and whether these experiences occurred within the past five years.

4. MRX is interested in the experiences and perspectives of those who have reviewed proposals submitted to NSF. We invite you to share your insights and describe any opportunities you believe would improve implementation of the Merit Review criteria, policy, and processes based on your experience reviewing NSF proposals.

Individuals responding to this request are encouraged to indicate whether they served on a panel and/or as ad hoc reviewers, and whether these experiences occurred within the past five years.

5. MRX is interested in exploring how NSF could better support awardees in demonstrating and documenting outcomes of their awards in advancing knowledge (Intellectual Merit) and benefiting society and contributing to the achievement of specific broader or societal outcomes (Broader Impacts). We invite you to share your insights on how NSF might better support awardees in demonstrating and documenting outcomes of their awards without unnecessarily increasing awardees' administrative burden of reporting.

Individuals responding to this request are encouraged to indicate whether their suggestions are based on experiences as investigators, users of public outcomes reports, or another perspective.

6. MRX welcomes any other comments on or suggestions for improving NSF's current Merit Review criteria, policy, and processes. It also welcomes information about aspects of Merit Review criteria, policy and processes that are currently working well.

What will NSF do with this information?

MRX will use the information submitted in response to this RFI to inform its assessment of the efficacy of the current Merit Review criteria, policy, and processes, and to draft recommendations regarding them. The information provided will be analyzed and considered by MRX. Respondents are advised that the government is under no obligation to acknowledge receipt of the information or provide feedback to respondents with respect to any information submitted. No proprietary, classified, confidential, or sensitive information should be included in your response submission. The government reserves the right to use any non-proprietary technical information in any resultant solicitation(s), policies, or procedures. All submitted information may be subject to disclosure under the Freedom of Information Act (FOIA) or other applicable law.

This Notice does not invite research proposals nor is it a funding opportunity.

Background

NSB and NSF, with the assistance of expert third parties, have periodically re-examined and revised the criteria, policy, and processes of Merit Review at NSF. The last time the Board systematically examined the Merit Review criteria was in 2010-2011 when NSB established a Task Force on Merit Review to examine the Intellectual Merit and Broader Impacts merit review criteria and their effectiveness in achieving NSF's goals in support of science and engineering research and education. At that time, Congress was considering, and then passed, the America COMPETES Reauthorization Act directing NSF to apply the Broader Impacts criterion to achieve a specific array of societal goals and charging NSF to develop policies addressing it. The 2011 Task Force report concluded that the Merit Review criteria remained appropriate for evaluating NSF proposals; however, it provided certain revisions and clarifications.

Recent events have underscored the importance of demonstrating that portfolios of funded projects enable NSF to meet its statutory mission “to promote the progress of science; to advance the national health, prosperity and welfare; to secure the national defense; and for other purposes.” In 2022, Congress passed the CHIPS and Science Act, which directed federal research agencies to regularly assess, and update as necessary, policies, and practices to remove or reduce cultural and institutional barriers limiting the recruitment, retention, and success of groups historically underrepresented in STEM research careers, including policies and practices relevant to the unbiased review of Federal research applications. Reexamining the Merit Review policy and process will help ensure that NSF is best placed to meet the requirements set out by Congress.

Definitions for Terms Used in This RFI

Merit Review Policy

Principles

1. All NSF projects should be of the highest quality and have the potential to advance, if not transform, the frontiers of knowledge.

2. NSF projects, in the aggregate, should contribute more broadly to achieving societal goals. These broader impacts may be accomplished through the research itself, through activities that are directly related to specific research projects, or through activities that are supported by, but are complementary to, the project. The project activities may be based on previously established and/or innovative methods and approaches, but in either case must be well justified.

3. Meaningful assessment and evaluation of NSF funded projects should be based on appropriate metrics, keeping in mind the likely correlation between the effect of broader impacts and the resources provided to implement projects. If the size of the activity is limited, evaluation of that activity in isolation is not likely to be meaningful. Thus, assessing the effectiveness of these activities may best be done at a higher, more aggregated, level than the individual project.

Criteria

Both criteria are to be given full consideration during the review and decision-making processes; each criterion is necessary but neither, by itself, is sufficient. Therefore, proposers must fully address both criteria.

• Intellectual Merit (IM): the potential for a proposed project to advance knowledge.
• Broader Impacts (BI): the potential for a proposed project to benefit society and contribute to the achievement of specific, desired societal outcomes.

Elements

1. What is the potential for the proposed activity to:

a. Advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and

b. Benefit society or advance desired societal outcomes (Broader Impacts)?

2. To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts?

3. Is the plan for carrying out the proposed activities well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?

4. How well qualified is the individual, team, or organization to conduct the proposed activities?

5. Are there adequate resources available to the PI (either at the home organization or through collaborations) to carry out the proposed activities?

This description of NSF's merit review policy is from NSF's 2024 Proposal and Award Policies and Procedures Guide (PAPPG), part I, ( print page 68475) chapter 3. See https://new.nsf.gov/​policies/​pappg/​24-1.

Transformative Research

Transformative research is defined as research driven by ideas that have the potential to radically change our understanding of an important existing scientific or engineering concept or leading to the creation of a new paradigm or field of science or engineering. Such research also is characterized by its challenge to current understanding or its pathway to new frontiers. See NSB's statement Enhancing Support of Transformative Research at NSF: https://www.nsf.gov/​pubs/​2007/​nsb0732/​nsb0732.pdf.

Broadening Participation

“Broadening participation in STEM” is the comprehensive phrase NSF uses to refer to the Foundation's goal of increasing the representation and diversity of individuals, organizations, and geographic regions that contribute to STEM education, research, and innovation. To broaden participation in STEM, it is necessary to address issues of equity, inclusion, and access in STEM education, training, and careers. Whereas all NSF funding programs might support broadening participation components, some funding programs primarily focus on supporting broadening participation research and projects. Examples can be found on the NSF Broadening Participation in STEM website. See https://new.nsf.gov/​funding/​initiatives/​broadening-participation, and the NSF PAPPG, Introduction https://new.nsf.gov/​policies/​pappg/​24-1.

Interested Individuals and Parties

The phrase used in this Notice, “interested individuals and parties”, is intended to be interpreted broadly and inclusively by potential respondents; we anticipate interested individuals and parties include, but are not limited to:

• current, past, and prospective NSF proposers, reviewers, and staff
• sponsored research administrators and support professionals
• representatives of organizations and communities working in or supporting the science and engineering research and education enterprise
• members of other communities of practice in the science and engineering research and education fields and
• members of the general public expressing an interest in these topics.

Ann E. Bushmiller,

Senior Counsel to the National Science Board.