2024-05698. Agency Information Collection Activities: Comment Request; Computer Science for All-Evaluation and Systematic Review of Grantee Documents

AGENCY:

National Science Foundation.

ACTION:

Submission for OMB review; comment request.

SUMMARY:

The National Science Foundation (NSF) has submitted the following information collection requirement to OMB for review and clearance under the Paperwork Reduction Act of 1995. This is the second notice for public comment; the first was published in the Federal Register , and no comments were received. NSF is forwarding the proposed submission to the Office of Management and Budget (OMB) for clearance simultaneously with the publication of this second notice.

DATES:

Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to www.reginfo.gov/​public/​do/​PRAmain. Find this particular information collection by selecting “Currently under 30-day Review—Open for Public Comments” or by using the search function.

FOR FURTHER INFORMATION CONTACT:

Suzanne H. Plimpton, Reports Clearance Officer, National Science Foundation, 2415 Eisenhower Avenue, Alexandria, Virginia 22314; telephone [phone]; or send email to [e-mail]. Individuals who use a telecommunications device for the deaf (TDD) may call the Federal Information Relay Service (FIRS) at 1–800–877–8339, which is accessible 24 hours a day, 7 days a week, 365 days a year (including Federal holidays).

SUPPLEMENTARY INFORMATION:

Comments: Comments regarding (a) whether the proposed collection of information is necessary for the proper performance of the functions of the NSF, including whether the information shall have practical utility; (b) the accuracy of the NSF's estimate of the burden of the proposed collection of information; (c) ways to enhance the quality, use, and clarity of the information on respondents; and (d) ways to minimize the burden of the collection of information on those who are to respond, including through the use of appropriate automated, electronic, mechanical, or other technological collection techniques or other forms of information technology should be addressed to the points of contact in the FOR FURTHER INFORMATION CONTACT section.

Copies of the submission may be obtained by calling 703–292–7556. NSF may not conduct or sponsor a collection of information unless the collection of information displays a currently valid OMB control number, and the agency informs potential persons who are to respond to the collection of information that such persons are not required to respond to the collection of information unless it displays a currently valid OMB control number.

Title of Collection: Computer Science for All—Evaluation and Systematic Review of Grantee Documents.

OMB Number: 3145–NEW.

Type of Request: Intent to seek approval to establish an information collection for post-award output and outcome monitoring system.

Abstract

Computer Science for All Researcher Practitioner Partnership (RPP) grantees are required to submit Annual Reports and Project Outcome reports to NSF that summarize the outputs, outcomes, and impact of their funded work. NSF is required by Congress to demonstrate the long-term outcomes for the CSforAll RPP initiative, defined as those that occur at least 5 years since grantees received funding. the first year where these long-term outcomes can be documented for the first CSforAll RPP cohort, funded in 2017, was 2023. This multi-year evaluation is focused on documenting the long-term outcomes for the first three cohorts of the initiative—2017, 2018, and 2019. There are a total of 73 funded grants from the three cohorts. To effectively extract and analyze the needed information to document long-term outcomes for these 73 RPP grants, a systematic review of all grantee reports of outputs, outcomes, and impacts will be conducted, following these steps:

Develop a document review form. The researchers conducting the evaluation of these long-term outcomes for the three cohorts of grantees will develop and use a document review form that will include fields for recording information needed for this evaluation, guided by the required outputs, outcomes, and impacts that NSF must document in a report to Congress. The form will include all information that the researchers were able to glean from the grantees' reports and will highlight where information is missing about each grant's outputs and outcomes during the period of performance, and up to at least 5 years after the grant was funded. The review form will focus on the relevant outputs, outcomes, and impacts related to each of the three strands of the CSforAll RPPs initiative (preK–8, high school, and multi-grade pathways) to document the long-term outcomes for each of those strands.

Review all grantee documents using the review form. The document review form will be completed and compiled by trained researchers who conduct a primary and secondary review of all relevant grantee documents related to the funded RPP, including the grantees' reports to NSF, as well as any related publications and websites, to help ensure thoroughness, consistency, and accuracy. The researchers will document all outputs, outcomes, and impacts they can find in their document reviews. These will be aligned with the list of required outputs, outcomes, and impacts that NSF must report to Congress ( e.g., number and demographics of teachers, number and demographics of students served by the grant).

Produce grantee profile memos for grantee verification. After the researchers complete the document review forms for each funded RPP grant to the best of their ability, the information will be summarized in a memo to be shared with each grantee for their review and to gather any missing information. PIs will be asked to provide any missing information, focused on known outputs, outcomes, and impacts up to at least 5 years after funding was received. After they have had time to review the form and gather the missing information, each PI will be invited to participate in 30–60 minute interview conducted via videoconferencing. The interview will be conducted by a member of the research team, with the purpose of confirming the outputs, outcomes, and impacts in the document review form, and following up with any remaining questions about the impact of the grant on preK—12 computer science education in the education systems that were served by the grant.

Finalize grantee-provided data and identify additional primary data collections. Any additional information provided by grantees will be added to the review document forms to finalize existing grantee data and to determine what additional data are needed to address research questions, the most appropriate method for collecting that information ( e.g., surveys, interviews, focus groups), and from whom ( e.g., district or school administration, teachers). Because this evaluation project involves providing NSF with insights about other relevant outcomes and impacts they may not have anticipated for this evaluation, the information collected from grantees' completion of the document review form and their interviews will be used to identify those additional outcomes and impacts.

Use of the Information

Much of the data needed for this collection will come from a review of the Annual Reports, Final Reports, Evaluation Reports, and Project Outcome Reports that grantees are required to submit to NSF. After a systematic review of all grantee documents for the 73 funded grants, necessary information will be extracted from the documents and reviewed by grantee PIs, following the steps outlined in the abstract.

Estimate of Public Burden

Collection title	Number of respondents	Annual number of responses/respondent	Annual hour burden
Verification of Document Review Form Information by RPP Grantees	72 grantee PIs	2 (1 hour for document review and up to 1 hour for follow-up call)	144

Respondents

The respondents are the Principal Investigator and/or program evaluator of each grant. They will be asked to review their grantee-specific memo, determine whether their data are accurately represented, and provide any additional available information during a 30–60-minute call.

Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall annualized cost to the respondents is estimated to be $8,085.48. The following table shows the annualized estimate of costs to PIs/designee respondents, who are generally university research faculty members. This estimated hourly rate is based on the Bureau of Labor Statistics' Occupational Employment and Wage Statistics from May 2022, for “Education Administrators, Postsecondary.” According to these estimates, the mean hourly wage for a postsecondary education administrator was $55.38.

Respondent type	Number of respondents	Burden hours per respondent	Average hourly rate	Estimated annual cost
Grantees/PIs	72	2	$55.38	$8,085.48
Total	72			8,085.48
Source: https://www.bls.gov/​oes/​current/​oes119033.htm.

Estimated Number of Responses per Report

Data collection involves all 72 grantees for the funded CSforAll RPP grants in the 2017, 2018, and 2019 cohorts.

Dated: March 13, 2024.

Suzanne H. Plimpton,

Reports Clearance Officer, National Science Foundation.