2024-00593. Notice of Intent To Seek Approval To Establish an Information Collection System; Computer Science for All-Evaluation and Systematic Review of Grantee Documents

AGENCY:

National Science Foundation.

ACTION:

Notice and request for comments.

SUMMARY:

Under the Paperwork Reduction Act of 1995, and as part of its continuing effort to reduce paperwork and respondent burden, the National Science Foundation (NSF) is inviting the general public or other Federal agencies to comment on this proposed continuing information collection.

DATES:

Written comments on this notice must be received by March 12, 2024, to be assured consideration. Comments received after that date will be considered to the extent practicable. Send comments to address below.

FOR FURTHER INFORMATION CONTACT:

Suzanne H. Plimpton, Reports Clearance Officer, National Science Foundation, 2415 Eisenhower Avenue, Suite E7400, Alexandria, Virginia 22314; telephone [phone]; or send email to [e-mail]. Individuals who use a telecommunications device for the deaf (TDD) may call the Federal Information Relay Service (FIRS) at 1–800–877–8339, which is accessible 24 hours a day, 7 days a week, 365 days a year (including federal holidays).

Comments: Comments are invited on: (a) whether the proposed collection of information is necessary for the proper performance of the functions of the Foundation, including whether the information will have practical utility; (b) the accuracy of the Foundation's estimate of the burden of the proposed collection of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on those who are to respond, including through the use of automated collection techniques or other forms of information technology.

SUPPLEMENTARY INFORMATION:

Title of Collection: Computer Science for All—Evaluation and Systematic Review of Grantee Documents.

OMB Number: 3145–NEW.

Expiration Date of Approval: Not applicable.

Type of Request: Intent to seek approval to establish an information collection for post-award output and outcome monitoring system.

Abstract: Computer Science for All Researcher Practitioner Partnership (RPP) grantees are required to submit Annual Reports and Project Outcome reports to NSF that summarize the outputs, outcomes, and impact of their funded work. NSF is required by Congress to demonstrate the long-term outcomes for the CSforAll RPP initiative, defined as those that occur at least 5 years since grantees received funding. the first year where these long-term outcomes can be documented for the first CSforAll RPP cohort, funded in 2017, was 2023. This multi-year evaluation is focused on documenting the long-term outcomes for the first three cohorts of the initiative—2017, 2018, and 2019. There are a total of 73 funded grants from the three cohorts. To effectively extract and analyze the needed information to document long-term outcomes for these 73 RPP grants, a systematic review of all grantee reports of outputs, outcomes, and impacts will be conducted, following these steps:

Develop a document review form. The researchers conducting the evaluation of these long-term outcomes for the three cohorts of grantees will develop and use a document review form that will include fields for recording information needed for this evaluation, guided by the required outputs, outcomes, and impacts that NSF must document in a report to Congress. The form will include all information that the researchers were able to glean from the grantees' reports and will highlight where information is missing about each grant's outputs and outcomes during the period of performance, and up to at least 5 years after the grant was funded. The review form will focus on the relevant outputs, outcomes, and impacts related to each of the three strands of the CSforAll RPPs initiative (preK–8, high school, and multi-grade pathways) to document the long-term outcomes for each of those strands.

Review all grantee documents using the review form. The document review form will be completed and compiled by trained researchers who conduct a primary and secondary review of all relevant grantee documents related to the funded RPP, including the grantees' reports to NSF, as well as any related publications and websites, to help ensure thoroughness, consistency, and accuracy. The researchers will document all outputs, outcomes, and impacts they can find in their document reviews. These will be aligned with the list of required outputs, outcomes, and impacts that NSF must report to Congress ( e.g., number and demographics of teachers, number and demographics of students served by the grant).

Produce grantee profile memos for grantee verification. After the researchers complete the document review forms for each funded RPP grant to the best of their ability, the information will be summarized in a memo to be shared with each grantee for their review and to gather any missing information. PIs will be asked to provide any missing information, focused on known outputs, outcomes, and impacts up to at least 5 years after funding was received. After they have had time to review the form and gather the missing information, each PI will be invited to participate in 30–60 minute interview conducted via videoconferencing. The interview will be conducted by a member of the research team, with the purpose of confirming the outputs, outcomes, and impacts in the document review form, and following up with any remaining questions about the impact of the grant on preK–12 computer science education in the education systems that were served by the grant.

Finalize grantee-provided data and identify additional primary data collections. Any additional information provided by grantees will be added to the review document forms to finalize existing grantee data and to determine what additional data are needed to address research questions, the most appropriate method for collecting that information ( e.g., surveys, interviews, focus groups), and from whom ( e.g., district or school administration, teachers). Because this evaluation project involves providing NSF with insights about other relevant outcomes and impacts they may not have anticipated for this evaluation, the information collected from grantees' completion of the document review form and their interviews will be used to identify those additional outcomes and impacts.

Use of the Information

Much of the data needed for this collection will come from a review of the Annual Reports, Final Reports, Evaluation Reports, and Project Outcome Reports that grantees are required to submit to NSF. After a systematic review of all grantee documents for the 73 funded grants, necessary information will be extracted from the documents and reviewed by grantee PIs, following the steps outlined in the abstract.

Estimate of Public Burden

Collection title	Number of respondents	Annual number of responses/respondent	Annual hour burden
Verification of Document Review Form Information by RPP Grantees	73 grantee PIs	2 (1 hour for document review and up to 1 hour for follow-up call)	146

Respondents

The respondents are the Principal Investigator and/or program evaluator of each grant. They will be asked to review their grantee-specific memo, determine whether their data are accurately represented, and provide any additional available information during a 30–60-minute call.

Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall annualized cost to the respondents is estimated to be $8,085.48. The following table shows the annualized estimate of costs to PIs/designee respondents, who are generally university research faculty members. This estimated hourly rate is based on the Bureau of Labor Statistics' Occupational Employment and Wage Statistics from May 2022, for “Education Administrators, Postsecondary.” According to these estimates, the mean hourly wage for a postsecondary education administrator was $55.38.

Respondent type	Number of respondents	Burden hours per respondent	Average hourly rate	Estimated annual cost
Grantees/PIs	73	2	$55.38	$8,085.48
Total	73			8,085.48
Source: https://www.bls.gov/​oes/​current/​oes119033.htm.

Estimated Number of Responses per Report

Data collection involves all 73 grantees for the funded CSforAll RPP grants in the 2017, 2018, and 2019 cohorts.

Dated: January 9, 2024.

Suzanne H. Plimpton,

Reports Clearance Officer, National Science Foundation.