2014-03534. Notice of Intent To Seek Approval To Establish an Information Collection System

AGENCY:

National Science Foundation.

ACTION:

Notice and request for comments.

SUMMARY:

Under the Paperwork Reduction Act of 1995, Public Law 104-13 (44 U.S.C. 3501 et seq.), and as part of its continuing effort to reduce paperwork and respondent burden, the National Science Foundation (NSF) is inviting the general public or other Federal agencies to comment on this proposed continuing information collection.

Comments: Comments are invited on: (a) Whether the proposed collection of information is necessary for the proper performance of the functions of the Foundation, including whether the information will have practical utility; (b) the accuracy of the Foundation's estimate of the burden of the proposed collection of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on those who are to respond, including through the use of automated collection techniques or other forms of information technology.

DATES:

Written comments on this notice must be received by April 21, 2014, to be assured consideration. Comments received after that date will be considered to the extent practicable. Send comments to address below.

FOR FURTHER INFORMATION CONTACT:

Ms. Suzanne H. Plimpton, Reports Clearance Officer, National Science Foundation, 4201 Wilson Boulevard, Suite 1265, Arlington, Virginia 22230; telephone [phone]; or send email to [e-mail]. Individuals who use a telecommunications device for the deaf (TDD) may call the Federal Information Relay Service (FIRS) at 1-800-877-8339, which is accessible 24 hours a day, 7 days a week, 365 days a year (including federal holidays).

SUPPLEMENTARY INFORMATION:

 

Title of Collection: Engineering Program Monitoring Data Collections.

OMB Number: 3145-NEW.

Expiration Date of Approval: Not applicable.

Type of Request: Intent to seek approval to establish an information collection for post-award output and outcome monitoring system.

Abstract:

Proposed Project:

NSF provides nearly 20 percent of federal funding for basic research to academic institutions.^[1] Within NSF, the Directorate for Engineering (ENG) has primary responsibility for promoting the progress of engineering in the United States in order to enable the Nation's capacity to perform. Its investments in engineering research and education aim to build and strengthen a national capacity for innovation that can lead over time to the creation of new shared wealth and a better quality of life. Most NSF programs in engineering are funded through the Directorate for Engineering, which also sponsors the NSF's Industrial Innovation and Partnerships (IIP) Division. To these ends, ENG provides support for research and implementation activities that may meet national needs. While scientists seek to discover what is not yet known, engineers apply fundamental science to design and develop new devices and engineered systems to solve societal problems. ENG also focuses on broadening participation in engineering research and careers.

The Directorate for Engineering (ENG) requests of the Office of Management and Budget (OMB) a clearance that will allow NSF-ENG to improve the rigor of our surveys for evaluations and program monitoring, as well as to initiate new data collections to monitor the immediate, intermediate and long-term outcomes of our investments by periodically surveying the grantees and their students involved in the research. The clearance will allow any program in the Directorate for Engineering at NSF to rigorously develop, test, and implement survey instruments and methodologies.

Some NSF-ENG programs regularly conduct a variety of data collection activities that include routine program monitoring, program evaluations, and education-related data collections from federally funded institutions of higher education. The primary objective of this clearance is to allow other programs in NSF-ENG to collect outcome and output data from grantees, their partners and students, which will enable the evaluation of the impact of its investments in engineering research over time. With that purpose, this clearance will allow us to use a bank of approved question items as needed as long as the resources consumed to do not exceed this request. The second related objective is to improve our questionnaires and/or data collection procedures through pilot tests and other survey methods used in these activities for different programs. Under this clearance a variety of surveys could be pre-tested, modified and used. The exact combination of questions from the question bank is currently unknown for each program, but it will be based on their respective logic models and program goals. Following standard OMB requirements, NSF will submit to OMB an individual request for each survey project it undertakes under this clearance. NSF will request OMB approval in advance and provide OMB with a copy of the questionnaire (if one is used) and materials describing the project.

In doing so, this request seeks approval for multiple data collections that have similar elements and purposes and will provide essential information for program monitoring purposes through multiple possible methods of collection. Data collected by ENG program outcome monitoring systems will be used for program planning, management, evaluation, and audit purposes. Summaries of output and outcome monitoring data are used to respond to queries from Congress, the public, NSF's external merit reviewers who serve as advisors, including Committees of Visitors (COVs), and NSF's Office of the Inspector General. These data are needed for effective administration, program and project monitoring, evaluation, strategic reviews and for measuring attainment of NSF's program and strategic goals, as identified by the President's Accountable Government Initiative, the Government Performance and Results Act (GPRA) Modernization Act of 2010, and NSF's Strategic Plan.

Outcome and output monitoring data represented in this collection is complementary to the data collected in the RPPR both with respect to type of questions and indicators (content) and timeliness of the collection. All questions asked are questions that are NOT included in the final or annual report and the intention is to ask them even beyond the period of performance on voluntary basis in order to capture impacts of the research that occur beyond the life of the award. Questionnaire items fall into the category of general items that could be used across programs as well as items of interest to a particular division. We are seeking to collect additional information from the grantees about the outcomes of their research that go above and beyond the standard reporting requirements used by the NSF and could span a period of up to 10 years after the award.

The six (6) divisions or offices in NSF-ENG which oversee multiple programs are included in this request. They are designed to assist in management of specific programs, divisions, or multi-agency initiatives and to serve as data resources for current and future program evaluations.

Program/office	Type of program
Emerging Frontiers in Research and Innovation (EFRI)	Fundamental Research.
Engineering Education and Centers (EEC)	Large research center's research (Implementation & Development) & Research and Education.
Industrial Innovation and Partnerships (IIP)	Translational Research.
Chemical, Bioengineering, Environmental, and Transport Systems (CBET)	Fundamental Research.
Civil, Mechanical, and Manufacturing Innovation (CMMI)	Fundamental Research.
Electrical, Communications, and Cyber Systems (ECCS)	Fundamental Research.

ENG-funded projects could include research opportunities and mentoring for educators, scholars, and university students, as well as outreach programs that help stir the imagination of K-12 students, often with a focus on groups underrepresented in science and engineering. The surveys to be tested and implemented would be designed to assist in management of specific division programs, divisions, or multi-agency initiatives and to serve as data resources for current and future program evaluations.

This data collection effort will enable program officers to longitudinally monitor outputs and outcomes given the unique goals and purpose of their programs. This is very important to enable appropriate and accurate evidence-based management of the programs and to determine whether or not the specific goals of the programs are being met.

Grantees will be invited to submit this information on a periodic basis to support performance review and the management of ENG grants by ENG officers. Once the survey tool for a specific program is tested, ENG grantees will be invited to submit these indicators to NSF via data collection methods that include but are not limited to online surveys, interviews, focus groups, phone interviews, etc. These indicators are both quantitative and descriptive and may include, for example, the characteristics of project personnel and students; sources of complementary cash and in-kind support to the ENG project; characteristics of industrial and/or other sector participation; research activities; education activities; knowledge transfer activities; patents, licenses; publications; descriptions of significant advances and other outcomes of the ENG-funded effort.

Use of the Information: The data collected will be used for NSF internal reports, historical data, program level studies and evaluations, and for securing future funding for the ENG program maintenance and growth. These data could be used for program evaluation purposes if deemed necessary for a particular program. Evaluation designs could make use of metadata associated with the award, and other characteristics to identify a comparison group to evaluate the impact of the program funding and other interesting research questions. Different designs could be possible based on the research questions varying from program to program but the fact that NSF-ENG has already collected data on the outcomes of interest will result in substantial savings on the evaluation per se.

Estimate of Burden:

Collection title	Number of respondents	Annual number of responses/ respondent	Annual hour burden
Emerging Frontiers in Research and Innovation (EFRI)	85	0.25	21.25
Civil, Mechanical, and Manufacturing Innovation (CMMI)	1300	0.25	325
Chemical, Bioengineering, Environmental, and Transport Systems (CBET)	1750	0.25	437.5
Electrical, Communications, and Cyber Systems (ECCS)	1000	0.25	250
Engineering Education and Centers (EEC)	100	0.25	100
Industrial Innovation and Partnerships (IIP)	1000	4	4000
Total	5,235		5,133.75

Below is an example that shows how the hour burden was estimated for the monitoring system.

The estimated average number of annual respondents is 5,235, with an estimated annual response burden of 5,133.75 hours. For post-award monitoring systems, most divisions expect to collect data at 1, 2, 5, and 10 years post-award, in order to have the best chance of capturing the more immediate outcomes expected by 1-2 years post-award, intermediate outcomes at 5 years post-award, and long-term outcomes/impacts at 10 years post-award. These four (4) data collections spread over the span of 10 years; this averages to 0.25 data collections/year. For the IIP division, many awards are made in translational research, such that we might expect a shorter and more condensed timeline of outcomes and impacts. Thus, some programs may wish to collect data quarterly for the first two years of the award, and then once annually at 5 and 10 years post-award. The annual number of responses for the first 2 years post-award is included in this table.

For life-of-award monitoring, the data collection burden to awardees will be limited to no more than 2 hours of the respondents' time in each instance.

Respondents: The respondents are either PIs or program coordinators. One PI or program coordinator per award completes the questionnaire.

Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall annualized cost to the respondents is estimated to be $214,635. The following table shows the annualized estimate of costs to PI/program coordinator respondents, who are generally university professors. This estimated hourly rate is based on a report from the American Association of University Professors, “Annual Report on the Economic Status of the Profession, 2011-12,” Academe, March-April 2012, Survey Report Table 4. According to this report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private-independent, religiously affiliated) was $86,319. When divided by the number of standard annual work hours (2,080), this calculates to approximately $41 per hour.

Respondent type	Number of respondents	Burden hours per respondent	Average hourly rate	Estimated annual cost
PIs/Program Coordinators (EFRI, CBET, CMMI, ECCS, EEC)	4,235	0.25	$41	$173,635
PIs/Program Coordinators (IIP Division)	1,000	1	41	41,000
Total	5,235			214,635

Estimated Number of Responses per Report: Data collection for the collections involves all awardees in the programs involved. The table below shows the total universe and sample size for each of the collections.

Respondent Universe and Sample Size of ENG Program Monitoring Clearance Collections

Collection title	Universe of respondents	Sample size
Emerging Frontiers in Research and Innovation (EFRI)	85	85
Civil, Mechanical, and Manufacturing Innovation (CMMI)	1300	1300
Chemical, Bioengineering, Environmental, and Transport Systems (CBET)	1750	1750
Electrical, Communications, and Cyber Systems (ECCS)	1000	1000
Engineering Education and Centers (EEC)	100	100
Industrial Innovation and Partnerships (IIP)	1000	1000

Dated: February 12, 2014.

Suzanne H. Plimpton,

Reports Clearance Officer, National Science Foundation.

Footnotes

1.  National Science Foundation. (2012). NSF at a glance. Retrieved from http://www.nsf.gov/​about/​glance.jsp.