03-14716. Office of Elementary and Secondary Education-Evaluating State Education Technology Programs Grant Competition-Notice Inviting Applications for New Awards for Fiscal Year (FY) 2003

Purpose of Program: The purpose of this program is to increase the capacity of States to design, conduct, and procure high-quality evaluations of educational technology. To do so, this competition supports grants to States to: (1) Build their capacity to conduct scientifically based evaluations of educational technology interventions, by planning and conducting an experimental or quasi-experimental evaluation of a State-selected educational technology initiative; and (2) widely disseminate pertinent information, based on what is learned about the evaluation methods, practices, analyses, and instruments used, that will help other States enhance their ability to conduct similar empirical evaluations.

States receiving awards will:

(1) Develop a plan to conduct a scientifically based evaluation of an educational intervention that uses technology applications as a tool to increase student achievement in one or more core academic subjects.

(2) Conduct the evaluation in a manner that tests the impact of the intervention as well as the efficacy of the empirical methods, practices, and instruments used to assess the impact of the intervention on student achievement.

(3) Disseminate information about the evaluation plan, its implementation, and the results to other States and to school districts so they may learn from and replicate the approach.

For FY 2003, the competition for new awards focuses on projects designed to meet the priority we describe in the Priority section of this application notice.

Eligible Applicants: The Secretary seeks to fund applications that are submitted by a State educational agency (SEA), or an SEA on behalf of a consortium (partnership).

Eligible partnerships are comprised of an SEA (that must serve as the fiscal agent) and at least one entity from among the following entities:

(1) Institutions of higher education.

(2) Other public institutions.

(3) Research organizations.

(4) Not-for-profit organizations.

(5) For-profit organizations.

(6) Other State educational agencies (SEAs).

(7) Local educational agencies (LEAs).

(8) Regional educational entities.

Applications Available: June 11, 2003.

Deadline for Transmittal of Applications: July 28, 2003.

Estimated Available Funds: $4,200,000.

Estimated Range of Awards: $300,000 to $650,000 (per annum).

Estimated Average Size of Awards: $475,000 (per annum).

Maximum Award: We will reject any application that proposes a budget that exceeds $650,000 for any of the three 12-month budget periods.

Estimated Number of Awards: 6-9.

Note:

The Department is not bound by any estimates in this notice.

Project Period: Up to 36 months.

Applicable Statute and Regulations: (a) Section 2421(c) of part D of the Elementary and Secondary Education Act (ESEA), as reauthorized by the No Child Left Behind Act of 2001 (NCLB); (b) The Education Department General Administrative Regulations (EDGAR) in 34 CFR parts 74, 75, 77, 80, 81, 82, 85, 86, 97, 98, and 99.

SUPPLEMENTARY INFORMATION:

Background

Section 2421(c) of part D (Enhancing Education Through Technology Act) of the ESEA authorizes the Secretary to award competitive grants for technical assistance to States to carry out activities to achieve the purposes of that part. Section 2402 (a)(7) of part D provides that one of the purposes of the Act is “To support the rigorous evaluation of programs funded under this part, particularly regarding the impact of such programs on student academic achievement, and ensure that timely information on the results of such evaluations is widely accessible through electronic means.”

The intent of this competition is to help States learn to conduct and procure high-quality evaluations by making funds available to a limited number of States to plan, conduct, and report such evaluations. States receiving awards will be expected not only to benefit directly from the experience, but actively share their work with other States. Therefore, grant awards from this competition are for States, with assistance from their grant partners or contractors, to: Plan and conduct rigorous, scientifically based evaluations of technology-enhanced educational interventions; test and refine the methods, practices, analyses, and instruments used; document project activities and outcomes; and to inform the evaluation efforts of other States by making available to those States documented information about the evaluation plan, its implementation, and results.

The Department expects that the projects it funds under this grant announcement will yield the following outcomes:

(1) Increased capacity within recipient States to routinely design and conduct scientifically based evaluations, particularly in carrying out directed State technology grant competitions under title II, part D of the ESEA.

(2) A body of knowledge that can inform other States about effective methods, practices, instruments, and conditions for conducting scientifically based evaluations, including:

a. Replicable methods, practices, analyses, and instruments States and districts can use to conduct rigorous scientifically based evaluations of educational interventions that use technology as a tool to enhance teaching and learning.

b. Methodological frameworks States and districts can use to identify technology-enhanced educational interventions that measurably increase student academic achievement, improve instruction, and enhance curriculum, including interventions that integrate new and emerging technologies into the curriculum.

In addition, the Department expects the evaluation findings that result from this competition will yield empirical evidence about the conditions and practices under which educational technology is effective in helping students meet challenging academic content standards and in increasing student academic achievement.

The Department expects to have substantial involvement with applicants that obtain grants under this competition. Therefore, the Department will enter into a cooperative agreement with each grantee. Under the agreement, the Department will work with the grantee to refine the evaluation and dissemination plans proposed in the application and will review plans and other deliverables before further work proceeds. (The terms “cooperative agreement” and “grant” are used interchangeably in this solicitation.)

In applying for awards under this announcement, applicants must propose a plan to evaluate the impact of technology-based educational interventions on student achievement in the core academic subjects. Of particular interest to the Department are elementary and secondary school technology interventions that are consistent with the mission and goals of the No Child Left Behind Act of 2001, that is, interventions to help students meet challenging State academic content standards and student academic achievement standards and to close the achievement gap between low-income and minority students and their peers. To identify and select an intervention for evaluation applicants might consider, for example:

(a) Innovative distance learning strategies that deliver, particularly to high-need student populations, specialized or rigorous academic courses and curricula through the use of advanced technologies, including video conferencing and virtual instruction strategies.

(b) Professional development programs to enable teachers to integrate advanced technologies, including emerging technologies, into curricula and instruction in order to prepare students to meet challenging State academic content and academic achievement standards.

(c) Programs that use technology to connect schools and teachers with parents and students in order to promote meaningful parental involvement; foster increased communication (about curricula, assignments, and assessments) between students, parents, and teachers; and assist parents to understand the technology being applied in their child's education, so that parents are able to reinforce at home the instruction that their child receives at school.

(d) Classroom-based courses and curricula that include integration of technology and are designed to help students meet challenging State academic content and student academic achievement standards.

(e) Programs that use technology to help teachers meet the high standards of teacher quality defined in ESEA.

(f) Programs that use technology to meet the educational needs of students in rural areas.

Examples of evaluation activities that may be funded include, but are not limited to:

(a) Studies that compare the achievement of students who take high school virtual courses with the achievement of students who take the same courses in regular classrooms in order to determine whether the virtual courses result in the same or different levels of student achievement as courses taught in the regular classroom;

(b) Studies to determine the relative effectiveness of various locally adopted, technology-enhanced instructional programs in increasing student achievement. These studies might compare the academic achievement of students in classrooms or schools where one instructional program or approach is being used with the academic achievement of students in classrooms or schools where another technology-enhanced instructional program or approach is being used; or

(c) Studies to determine the impact on student achievement of technology-enhanced instruction in specific academic content areas versus academic content area instruction without the use of technology.

Applicants, in developing their proposals, should detail:

1. The evaluation approach they propose to take, including the methods, practices, and analyses for: (a) Selecting the intervention, population, and problem for study; (b) choosing the sample; and (c) collecting and analyzing data.

2. Provisions for assessing the efficacy of the evaluation approach and making refinements as warranted.

3. Plans for documenting project activities and disseminating to other States and school districts the knowledge gained over the course of the grant about how and under what conditions similar evaluations can be replicated. Proposed strategies should include, but not be limited to, the use of the Internet and other emerging technologies and venues. At a minimum, dissemination strategies should include the establishment and maintenance of a project Website that will host continually updated information about: (a) The technology intervention being evaluated; (b) the evaluation methods, practices, and analyses carried out, including the data collection plans and instruments used; (c) any modifications that occurred during the implementation of the project along with the rationale for those modifications; (d) the processes and conditions necessary for other States and school districts to replicate the approach; and (e) the evaluation results and other project findings.

Priority

This competition focuses on projects designed to meet the following priority. Under 34 CFR 75.105(c)(2)(i), applications will receive up to 20 additional points depending on how well they meet the priority. These points are in addition to any points the application earns under the selection criteria.

Competitive Preference Priority

The project is designed to determine whether the program implemented produces meaningful effects on student achievement or teacher performance through a rigorous evaluation. Evaluations using an experimental design are the strongest for determining program effectiveness. Thus, the project preferably uses an experimental design. An evaluation using an experimental design is one where subjects (students, teachers, classrooms, or schools) are randomly assigned to receive the program being evaluated or to be in a control group that does not receive the program. Evaluations using an experimental design will receive up to 20 points.

If random assignment is not feasible, the project may employ a quasi-experimental design with carefully matched comparison conditions. This alternative design attempts to approximate a randomly assigned control group by matching program participants (students, teachers, classrooms or schools) with non-participants possessing similar pre-program characteristics. Evaluations of this type will receive up to 10 points.

Proposed evaluations that use neither experimental designs with random assignment or quasi-experimental designs using a matched comparison group will receive 0 points under the competitive preference priority.

Data from reliable and valid measures of the intervention that the program intends to implement and of the outcomes that the program intends to effect should be collected before and after participation in the program or the comparison condition.

Points awarded under this priority will be determined by the quality of the proposed evaluation. In determining the quality of the evaluation, we will consider the extent to which the applicant presents a feasible, credible plan that includes: The type of design to be used (random assignment or matched comparison); outcomes to be measured; a discussion of how students, teachers, classrooms, or schools will be assigned to the program or matched for comparison with other students, teachers, classrooms, or schools; and a proposed evaluator, preferably independent, with the necessary background and technical expertise to carry out the proposed evaluation.

Selection Criteria

We will use the following selection criteria to evaluate applications for new grants under this program.

The maximum score for all of these criteria is 100 points.

The maximum score for each criterion is indicated in parentheses.

(a) Significance (20 points). In determining the significance of the proposed project, the Secretary will consider the following factors:

(1) The extent to which the application supports the Department's strategic interests embodied in the No Child Left Behind Act of 2001: greater accountability for student achievement, increased flexibility and local control, more parental choice, and a focus on what works.

(2) The extent to which the application offers a reasonable and sound plan that likely will produce outcomes, products, or publications that will inform the field about evaluation practice to determine the effectiveness of new and advanced technology tools and applications in education, and that are easily exportable to different settings, including urban, rural, and suburban communities.

(3) The extent to which the application proposes to disaggregate evaluation results so that the impact of the intervention on the academic achievement of sub-groups of students, such as students who are ethnic or language minorities, rurally isolated, or from families with incomes below the poverty line, can be determined.

(b) Quality of the project design (35 points). In determining the quality of the design of the proposed project, the Secretary will consider the following factors:

(1) The extent to which the methodology proposed is thorough, feasible, and employs a methodologically sound experimental or quasi-experimental design to determine the effectiveness of the educational technology intervention under study.

(2) The extent to which the methodology proposed includes the use of valid, reliable, and objective performance measures that are clearly related to the intended outcomes of the intervention being evaluated.

(3) The extent to which the proposed project is designed to build evaluation capacity and practice that will extend beyond the period of Federal financial assistance.

(4) The extent to which the design for implementing the proposed evaluation, documenting evaluation activities, and disseminating knowledge, will result in information to guide replication of project activities or strategies, including information about the effectiveness of the approach or strategies employed by the project.

(c) Quality of project personnel (15 points). In determining the quality of project personnel, the Secretary will consider:

(1) The extent to which the applicant encourages applications for employment from persons without regard to race, color, national origin, gender, age, or disability.

(2) In addition, the Secretary considers the following factors:

(i) The qualifications, including relevant training and experience, of the project director or principal investigator.

(ii) The qualifications, including relevant training and experience, of project consultants or subcontractors.

(d) Adequacy of resources (15 points). In determining the adequacy of resources for the proposed project, the Secretary will consider the following factors:

(1) The extent to which the partnership members contribute to the activities assisted under the grant by providing substantial support in the form of non-Federal funds and/or in-kind contributions, including staff and facilities.

(2) The adequacy of support, including facilities, equipment, supplies, and other resources, from the applicant organization or the lead applicant organization.

(3) The relevance and demonstrated commitment of each partner in the proposed project to the implementation and success of the project.

(4) The potential for the incorporation of project purposes, activities, or benefits into the ongoing program of the agency or organization at the end of Federal funding.

(e) Quality of the management plan (15 points). In determining the quality of the management plan for the proposed project, the Secretary will consider the following factors:

(1) The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing project tasks.

(2) The adequacy of procedures for ensuring feedback and continuous improvement in the operation of the proposed project.

Waiver of Proposed Rulemaking

Under the Administrative Procedure Act (5 U.S.C. 553), the Department generally offers interested parties the opportunity to comment on proposed priorities and other program requirements. Ordinarily, this practice would have applied to the competitive priority, selection criteria, and requirements in this notice. Section 437(d)(1) of the General Education Provisions Act (GEPA), however, exempts rules that apply to the first competition under a new program from this requirement. The competition covered by this notice is a new activity under the National Technology Activities authorized by the ESEA, as reauthorized by the No Child Left Behind Act of 2001. The Secretary, in accordance with section 437(d)(1) of GEPA, in order to ensure timely grant awards, has decided to forego public comment with respect to the competitive priority, selection criteria and program requirements. The competitive priority, selection criteria, and requirements of this grant notice will apply only to the FY 2003 grant competition.

Application Procedures

The Government Paperwork Elimination Act (GPEA) of 1998, (Pub. L. 105-277) and the Federal Financial Assistance Management Improvement Act of 1999, (Pub. L. 106-107) encourage us to undertake initiatives to improve our grant processes. Enhancing the ability of individuals and entities to conduct business with us electronically is a major part of our response to these Acts. Therefore, we are taking steps to adopt the Internet as our chief means of conducting transactions in order to improve services to our customers and to simplify and expedite our business processes.

We are requiring that applications to the FY 2003 Evaluating State Education Technology Programs Grant Competition be submitted electronically using e-Application available through the Education Department's e-GRANTS system. The e-GRANTS system is accessible through its portal page at: http://e-grants.ed.gov.

Applicants who are unable to submit an application through the e-GRANTS system may apply for a waiver to the electronic submission requirement. To apply for a waiver, applicants must explain the reason(s) that prevent them from using the Internet to submit their applications. The reasons(s) must be outlined in a letter addressed to: Enid Simmons, U.S. Department of Education, 400 Maryland Avenue, SW., room 3E215, Washington, DC 20202. We must receive your letter no later than two weeks before the closing date.

Any application that receives a waiver to the electronic submission requirement will be given the same consideration in the review process as an electronic application.

Pilot Project for Electronic Submission of Applications

In FY 2003, the U.S. Department of Education is continuing to expand its pilot project for electronic submission of applications to include additional formula grant programs and additional discretionary grant competitions. The Evaluating State Education Technology Programs Grant Competition is one of the programs included in the pilot project.

The pilot project involves the use of the Electronic Grant Application System (e-Application) portion of the Grant Administration and Payment System (GAPS). Users of e-Application will be entering data on-line while completing their applications. You may not e-mail a soft copy of a grant application to us. The data you enter on-line will be saved into a database. We shall continue to evaluate the success of the electronic submission of applications and solicit suggestions for improvement.

If you participate in e-Application, please note the following:

• When you enter the e-Application system, you will find information about its hours of operation.
• You must submit all documents electronically, including the Application for Federal Education Assistance (ED 424), Budget Information—Non-Construction Programs (ED 524), and all necessary assurances and certifications.
• After you electronically submit your application, you will receive an automatic acknowledgement, which will include a PR/Award number (an identifying number unique to your application).
• Within three working days after submitting your electronic application, fax a signed copy of the Application for Federal Education Assistance (ED 424) to the Application Control Center after following these steps:

1. Print ED 424 from the e-Application system.

2. The institution's Authorizing Representative must sign this form.

3. Place the PR/Award number in the upper right hand corner of the hard copy signature page of the ED 424.

4. Fax the signed ED 424 to the Application Control Center at [phone].

• We may request that you give us original signatures on all other forms at a later date.
• Closing Date Extension in Case of System Unavailability: If you are prevented from submitting your application on the closing date because the e-Application system is unavailable, we will grant you an extension of one business day in order to transmit your application electronically, by mail, or by hand delivery. For us to grant this extension—

1. You must be a registered user of e-Application, and have initiated an e-Application for this competition; and

2. (a) The e-Application system must be unavailable for 60 minutes or more between the hours of 8:30 and 3:30 p.m., Washington, DC time, on the deadline date; or

(b) The e-Application system must be unavailable for any period of time during the last hour of operation (that is, for any period of time between 3:30 and 4:30 p.m., Washington, DC time) on the deadline date.

The Department must acknowledge and confirm these periods of unavailability before granting you an extension. To request this extension you must contact either (1) the person listed elsewhere in this notice under FOR FURTHER INFORMATION CONTACT or (2) the e-GRANTS help desk at 1-888-336-8930.

You may access the electronic grant application for the Evaluating State Education Technology Programs at: http://e-grants.ed.gov.

We have included additional information about the e-Application pilot project (see Parity Guidelines Between Paper and Electronic Applications) in the application package.

FOR FURTHER INFORMATION CONTACT:

Enid Simmons, U.S. Department of Education, 400 Maryland Avenue, SW., room 3E215, Washington, DC 20202. Telephone: [phone] or via Internet: [e-mail].

If you use a telecommunications device for the deaf (TODD), you may call the Federal Information Relay Service (FIRS) at 1-800-877-8339.

Individuals with disabilities may obtain this document in an alternative format (e.g., Braille, large print, audiotape, or computer diskette) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT.

Individuals with disabilities may obtain a copy of the application package in an alternative format by contacting that person. However, the Department is not able to reproduce in an alternative format the standard forms included in the application package.

Electronic Access to This Document

You may view this document, as well as all other Department of Education documents published in the Federal Register, in text or Adobe Portable Document Format (PDF) on the Internet at the following site: www.ed.gov/​legislation/​FedRegister.

To use PDF you must have Adobe Acrobat Reader, which is available free at this site. If you have questions about using PDF, call the U.S. Government Printing Office (GPO), toll free, at 1-888-293-6498; or in the Washington, DC, area at [phone].

Note:

The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available on GPO Access at: http://www.access.gpo.gov/​nara/​index.html.

Program Authority: 20 U.S.C. 6771(c).

Dated: June 5, 2003.

Eugene W. Hickock,

Under Secretary of Education.