2023-08603. Request for Information (RFI) on Recommendations for Improving NRSA Fellowship Review

AGENCY:

National Institutes of Health, HHS.

ACTION:

Request for information.

SUMMARY:

The purpose of this Request for Information (RFI) is to solicit public input on proposed changes to the peer review of Ruth L. Kirschstein National Research Service Award (NRSA) fellowship applications that would restructure the review criteria and modify some sections of the Public Health Service (PHS) Fellowship Supplemental Form within the application. The goal of this effort is to facilitate the mission of NRSA fellowship peer review: to identify the most promising trainees and the excellent, individualized training programs that will help them become the outstanding scientists of the next generation. The proposed changes will allow peer reviewers to better evaluate the applicant's potential and the quality of the scientific training plan without undue influence of the sponsor's or institution's reputation; and ensure that the information provided in the application is aligned with the restructured criteria and targeted to the fellowship candidate's specific training needs.

DATES:

Comments must be received by June 23, 2023 to ensure consideration.

ADDRESSES:

Submissions can be sent electronically to: https://rfi.grants.nih.gov/​?s=​642ed5def0356688b20e6be3. NIH is specifically requesting public comment on the proposed changes to the peer review of NRSA fellowship applications that would restructure the review criteria and modify some sections of the PHS Fellowship Supplemental Form within the application described above and at: https://grants.nih.gov/​policy/​peer/​improving-nrsa-fellowship. Response to this RFI is voluntary.

FOR FURTHER INFORMATION CONTACT:

Questions about this request for information should be directed to Kristin Kramer, 6701 Rockledge Drive, Bethesda, MD 20817, 301–437–0911, [e-mail].

SUPPLEMENTARY INFORMATION:

Current Process

The overall goal of the NIH Ruth L. Kirschstein National Research Service Award (NRSA) program is to help ensure that a diverse pool of highly trained scientists is available in appropriate scientific disciplines to address the Nation's biomedical, behavioral, and clinical research needs. NRSA fellowships support the training of pre-and postdoctoral scientists, dual-degree investigators, and senior researchers. The first stage of NIH peer review serves to provide expert advice to NIH by assessing the likelihood that the fellowship will enhance the candidate's potential for, and commitment to, a productive independent scientific research career in a health-related field. The criteria for the review of NRSA fellowship applications derive from the NRSA regulation at 42 CFR part 66.106 ( https://www.ecfr.gov/​current/​title-42/​chapter-I/​subchapter-E/​part-66/​subpart-A/​section-66.106):

(a) Within the limits of funds available, the Secretary shall make Awards to those applicants:

(1) Who have satisfied the requirements of § 66.105; and

(2) Whose proposed research or training would, in the judgment of the Secretary, best promote the purposes of section 487(a)(1)(A) of the Act, taking into consideration among other pertinent factors:

(i) The scientific, technical, or educational merit of the particular proposal;

(ii) The availability of resources and facilities to carry it out;

(iii) The qualifications and experience of the applicant; and

(iv) The need for personnel in the subject area of the proposed research or training.

The NIH peer review regulation does not address scoring. Scoring of all regulatory factors is determined by NIH policy. Currently, peer reviewers provide an Overall Impact Score (scored 1–9) that reflects their assessment of the likelihood that the fellowship will enhance the candidate's potential for, and commitment to, a productive independent scientific research career in a health-related field. Peer reviewers provide individual criterion scores for five criteria: (1) Applicant; (2) Sponsors and Collaborators; (3) Research Training Plan; (4) Training Potential; and (5) Institutional Environment and Commitment. Additional review criteria are evaluated and factored into the Overall Impact Score but are not given individual scores: Protections for Human Subjects; Inclusion of Women, Minorities, and Individuals Across the Lifespan; Vertebrate Animals; Biohazards; and Resubmission. Beyond these criteria, reviewers are asked to assess the following additional review considerations; these considerations are not considered in the Overall Impact Score: Training in the Responsible Conduct of Research, Applications from Foreign Organizations, Select Agents, Resource Sharing Plans, Budget and Period of Support, and Authentication of Key Biological and/or Chemical Resources.

Proposal Development

NIH gathered input from many sources in forming this proposal. Unsolicited comments over a period of years conveyed persistent concerns that the NRSA fellowship review process disadvantages some highly-qualified, promising applicants. In response, the Center for Scientific Review (CSR) formed a working group to the CSR Advisory Council. To inform that group, CSR published a Review Matters blog at: https://www.csr.nih.gov/​reviewmatters/​2022/​01/​06/​strengthening-fellowship-review/​, inviting comments, which was cross-posted on the Office of Extramural Research blog, Open Mike at: https://nexus.od.nih.gov/​all/​2022/​01/​10/​strengthening-fellowship-review/​. The working group presented an interim report at: https://public.csr.nih.gov/​sites/​default/​files/​2019-10/​Review_​criteria_​wg_​CSRAC_​interim_​report_​7April2020.pdf to the CSR Advisory Council, which adopted the recommendations, at public CSR Advisory Council meetings (March 2022 video https://videocast.nih.gov/​watch=​44677, slides https://public.csr.nih.gov/​sites/​default/​files/​2022-04/​CSRAC_​Fellowship_​WG_​interim_​presentation.pdf; September 2022 video https://videocast.nih.gov/​watch=​45767, slides https://public.csr.nih.gov/​sites/​default/​files/​2022-09/​CSRAC_​WG_​on_​Fellowship_​Review_​Sept_​2022.pdf). Final recommendations from the CSR Advisory Council at: https://public.csr.nih.gov/​sites/​default/​files/​2022-11/​CSRAC_​Fellowship_​review_​WG_​report_​September_​2022_​final.pdf were considered by the CSR Director, as well as major internal NIH extramural-focused committees that included leadership from across NIH institutes and centers. Additional background information can be found at: https://grants.nih.gov/​policy/​peer/​improving-nrsa-fellowship.

Recommendations for Improving NRSA Fellowship Review

Revise the Criteria Used To Evaluate NRSA Fellowship Applications

As is currently the case, the Overall Impact Score (scored 1–9) will reflect the scientific and educational merit of the proposal and an assessment of the likelihood that the fellowship will enhance the applicant's potential for, and commitment to, an independent, productive research career in a health-related field. However, the current 5 scored criteria that inform the Overall Impact Score will be restructured into the following 3 scored criteria. Additional detail on proposed reviewer guidance can be found here: https://grants.nih.gov/​policy/​peer/​improving-nrsa-fellowship/​reviewer-instructions.

Criterion 1: Scientific Potential, Fellowship Goals, and Preparedness of the Applicant (Scored 1–9)

• Evaluate the breadth and depth of scientific understanding the applicant conveys in their statements. To what extent does the candidate articulate the importance of their science and demonstrate an ability to study that problem in a rigorous scientific manner.
• Evaluate the preparedness of the applicant to undertake the proposed training and their capacity to benefit from the fellowship. Evaluate their accomplishments in the context of their stage of training and the scientific opportunities they have had.
• Evaluate the applicant's scientific potential. Consider their trajectory in the context of their opportunities. Also consider other factors that bear on their potential to succeed, such as determination, persistence, and creativity.

Criterion 2: Science and Scientific Resources (Scored 1–9)

• Evaluate the quality of the proposed science. Assess the depth of understanding of the scientific background and the scientific rigor and feasibility of the approach.
• Evaluate the extent to which needed technical, scientific, and clinical resources are specified and are realistically available to the applicant.
• Assess whether the scientific expertise of the mentorship team is appropriate for the proposed science and whether the role of each mentor is clearly defined.
• Evaluate how well the proposed scientific project serves the applicant's training goals.
• Note that peer review of financial support for the proposed research will be eliminated.

Criterion 3: Training Plan and Training Resources (Scored 1–9)

• Evaluate whether the applicant clearly defines their career goals and whether the training plan is linked to them.
• Evaluate whether the applicant has clearly defined areas of needed growth. These could include specific scientific skills and knowledge and other professional needs such as communication, teaching, and mentorship skills.
• Evaluate the training environment for this applicant. Assess whether the necessary institutional training resources are well-specified and available, specifically the practical availability of resources.
• Evaluate whether the trainee articulated a coherent and cohesive plan for interacting with sponsors and mentors.
• Assess whether the sponsor presents a strong pedagogical plan appropriate to the needs and goals of the applicant. Please include an evaluation of the training philosophy of the sponsor, their approach to training, time commitments and their accessibility.
• Evaluate and comment on what impact completion of the training plan will make in meeting the scientific development needs of the applicant and aid them in achieving their career goals.

The Additional Review Criteria ( e.g., Protections for Human Subjects; Inclusion of Women, Minorities and Individuals Across the Lifespan; etc.) would not change.

The Additional Review Considerations ( e.g., Training in the Responsible Conduct of Research, Resource Sharing Plans, Budget, etc.) would not change.

Revising the criteria simplifies the task of reviewers by focusing their attention on just three key assessments: the scientific potential of the applicant, the science and scientific resources, and the training plan and training resources. The criteria are defined to give applicants from heterogeneous backgrounds a fair chance; reviewers are asked to evaluate applicant accomplishments and trajectory in the context of the opportunities they have had. In addition to evaluating applicant accomplishments, reviewers are asked to evaluate personal characteristics that contribute to success in science, factors such as determination, persistence, and creativity. The revised criteria are also expected to reduce bias in review by reducing any consideration of sponsor and institutional reputation and instead focusing review on their specific, realistic, and current contributions to the scientific needs, goals, and training of the specific trainee. NIH believes these changes will better enable peer review to identify those applications with the highest potential for producing productive research scientists, regardless of where the applicant started or the applicant institution.

Revise the Fellowship Supplemental Section of PHS SF424

The NIH proposes to revise the following sections of the PHS 424 Fellowship Supplement ( https://grants.nih.gov/​grants/​how-to-apply-application-guide/​forms-g/​fellowship-forms-g.pdf): (1) Fellowship Applicant; (2) Sponsor(s), Collaborator(s), and Consultant(s); and (3) Letters of Reference. There are no proposed changes to the Research Training Plan section. Additionally, the revision would allow an optional Statement of Special Circumstance.

The changes are intended to restructure the application so that the application content is better aligned with the review criteria, is less duplicative, and is easier for reviewers to assess. The changes emphasize substantive statements that pertain to the individual applicant trainee, require detailed accounts from sponsors explaining their preparation and approach to training, and their availability to the student. The changes would shorten the application by up to 2 1/2 pages. The proposed changes for each section are described below:

1. Revised Applicant Section of the Fellowship Supplement

Applicants would be asked to submit five statements:

1. Statement of professional and fellowship goals.

2. Fellowship qualifications.

3. Self-assessment.

4. Statement of scientific perspective.

5. Activities planned under this award.

Additionally, grades would no longer be required or allowed, however, applicants would be requested to include the titles of relevant courses completed.

2. Revised Sponsor and Co-Sponsor Section of the Fellowship Supplement

Sponsors and Co-sponsors would be asked to submit three statements:

1. Training plan, environment, and research facilities.

2. Number of Fellows/Trainees to be supervised.

3. Applicant's qualifications and potential for a research career.

3. Revised Instructions for Reference Letters

NIH proposes to update the instructions for reference letters with more structure so that the resulting letters better assist reviewers in understanding the applicant's strengths, weaknesses, and potential to pursue a productive career in biomedical science. Writers would be instructed to respond to four questions addressing:

1. Two to four most important characteristics that will contribute to applicant's success.

2. Two to four areas of needed growth.

3. Intellectual contributions made by the applicant during training.

4. Overall assessment of readiness and potential.

4. Allow an Optional Statement of Special Circumstance

NIH recommends allowing fellowship applicants to submit an optional Statement of Special Circumstance to address situations that may have hindered the trainee's progress, such as harassment, the COVID–19 pandemic, or other personal or professional circumstances.

Additional detail on proposed changes to the Fellowship Supplement can be found at: https://grants.nih.gov/​policy/​peer/​improving-nrsa-fellowship/​reviewer-instructions.

Submitting a Response

Comments should be submitted electronically to the following web page at: https://rfi.grants.nih.gov/​?s=​642ed5def0356688b20e6be3.

This RFI is for planning purposes only and should not be construed as a policy, solicitation for applications, or as an obligation on the part of the Government to provide support for any ideas identified in response to it. Please note that the Government will not pay for the preparation of any information submitted or for its use of that information.

Please do not include any proprietary, classified, confidential, or sensitive information in your response. Responses will be compiled and a content analysis will be shared publicly after the close of the comment period. The NIH may use information gathered by this Notice to inform future policy development.

Dated: April 18, 2023.

Tara A. Schwetz,

Acting Principle Deputy Director, National Institutes of Health.