2012-30199. Proposed Priorities, Requirements, Definitions, and Selection Criteria-Investing in Innovation Fund

Catalog of Federal Domestic Assistance (CFDA) Numbers: 84.411A, 84.411B, and 84.411C

AGENCY:

Office of Innovation and Improvement, Department of Education.

ACTION:

Proposed priorities, requirements, definitions, and selection criteria.

SUMMARY:

The Assistant Deputy Secretary for Innovation and Improvement proposes priorities, requirements, definitions, and selection criteria under the Investing in Innovation Fund (i3). The Assistant Deputy Secretary may use these priorities, requirements, definitions, and selection criteria for competitions in fiscal year (FY) 2013 and later years.

The U.S. Department of Education (Department) has conducted three competitions under the i3 program and awarded 92 i3 grants since the program was established under the American Recovery and Reinvestment Act of 2009 (ARRA). These proposed priorities, requirements, definitions, and selection criteria maintain the overall purpose and structure of the i3 program, which is discussed later in this document, and incorporate changes based on specific lessons learned from the first three competitions.

DATES:

We must receive your comments on or before January 14, 2013.

ADDRESSES:

Submit your comments through the Federal eRulemaking Portal or via postal mail, commercial delivery, or hand delivery. We will not accept comments by fax or by email. To ensure that we do not receive duplicate copies, please submit your comments only once. In addition, please include the Docket ID at the top of your comments.

• Federal eRulemaking Portal: Go to www.regulations.gov to submit your comments electronically. Information on using Regulations.gov, including instructions for accessing agency documents, submitting comments, and viewing the docket, is available on the site under “How to Use This Site.”
• Postal Mail, Commercial Delivery, or Hand Delivery: If you mail or deliver your comments about these proposed regulations, address them to Carol Lyons, U.S. Department of Education, 400 Maryland Avenue SW., room 4W203, LBJ, Washington, DC 20202-5930.

Privacy Note:

The Department's policy is to make all comments received from members of the public available for public viewing in their entirety on the Federal eRulemaking Portal at www.regulations.gov. Therefore, commenters should be careful to include in their comments only information that they wish to make publicly available.

FOR FURTHER INFORMATION CONTACT:

Carol Lyons. Telephone: [phone]. Or by email: [e-mail]. If you use a telecommunications device for the deaf (TDD) or text telephone (TTY), call the Federal Relay Service (FRS), toll free, at 1-800-877-8339.

SUPPLEMENTARY INFORMATION:

Invitation to Comment: We invite you to submit comments regarding this notice. To ensure that your comments have maximum effect in developing the notice of final priorities, requirements, definitions, and selection criteria, we urge you to identify clearly the specific proposed priority, requirement, definition, or selection criterion that each comment addresses. We make additional, specific requests for comment in the sections setting out the proposed priorities, requirements, definitions, and selection criteria elsewhere in this notice.

We invite you to assist us in complying with the specific requirements of Executive Orders 12866 and 13563 and their overall requirement of reducing regulatory burden that might result from these proposed priorities, requirements, definitions, and selection criteria. Please let us know of any further ways we could reduce potential costs or increase potential benefits while preserving the effective and efficient administration of the program.

During and after the comment period, you may inspect all public comments about this notice by accessing Regulations.gov. You may also inspect the comments in person in room 4W335, LBJ, 400 Maryland Avenue SW., Washington, DC, between the hours of 8:30 a.m. and 4:00 p.m., Washington, DC time, Monday through Friday of each week except Federal holidays. Please contact the person listed under FOR FURTHER INFORMATION CONTACT.

Assistance to Individuals with Disabilities in Reviewing the Rulemaking Record: On request we will provide an appropriate accommodation or auxiliary aid to an individual with a disability who needs assistance to review the comments or other documents in the public rulemaking record for this notice. If you want to schedule an appointment for this type of accommodation or auxiliary aid, please contact the person listed under FOR FURTHER INFORMATION CONTACT.

Purpose of Program: The i3 program addresses two related challenges. First, there are too few practices in education supported by rigorous evidence of effectiveness, despite national attention paid to finding practices that are effective at improving education outcomes in the decade since the establishment of the Department's Institute of Education Sciences (IES). Second, there are limited incentives to expand effective practices substantially and to use those practices to serve more students across schools, districts, and States. Student achievement suffers as a result.

The central innovation of the i3 program, and how it addresses these two challenges, is its multi-tier structure that links the amount of funding that an applicant may receive to the quality of the evidence supporting the efficacy of the proposed project. Applicants proposing practices supported by limited evidence can receive small grants that support the development and initial evaluation of promising practices and help to identify new solutions to pressing challenges; applicants proposing practices supported by evidence from rigorous evaluations, such as large randomized controlled trials, can receive sizable grants to support expansion across the Nation. This structure provides incentives for applicants to build evidence of effectiveness of their proposed projects and to address the barriers to serving more students across schools, districts, and States so that applicants can compete for more sizeable grants.

As importantly, all i3 projects are required to generate additional evidence of effectiveness. All i3 grantees must use part of their budgets to conduct independent evaluations (as defined in this notice) of their projects. This ensures that projects funded under the i3 program contribute significantly to improving the information available to practitioners and policymakers about which practices work, for which types of students, and in which contexts.

Program Authority: American Recovery and Reinvestment Act of 2009 (ARRA), Division A, Section 14007, Pub. L. 111-5.

Background

The Statutory Context

The ARRA established the i3 program to provide competitive grants to local educational agencies (LEAs) and nonprofit organizations with a record of improving student achievement in order to expand the implementation of, and investment in, innovative practices that are demonstrated to improve student achievement (as defined in this notice) or student growth (as defined in this notice), close achievement gaps, decrease dropout rates, increase high school graduation rates (as defined in this notice), or increase college enrollment and completion rates. The ARRA provided funding for the i3 program's first competition carried out during FY 2010; the FY 2011 and FY 2012 competitions were funded under the Department's annual appropriations. The Administration's reauthorization proposal for the Elementary and Secondary Education Act of 1965, as amended (20 U.S.C. 6301 et seq.) (ESEA) would authorize the i3 program under that act.

Overview of the Investing in Innovation Fund (i3)

As the Department's primary evidence-based grantmaking program, the i3 program is designed to generate and validate solutions to persistent educational challenges and support the expansion of effective solutions across the country to serve substantially larger numbers of students.

There are a number of features that make the i3 program different from many other Federal grant programs in education.

First, the i3 program builds a portfolio of different practices in critical priority areas. As the Proposed Priorities section of this document makes clear, the i3 program supports projects in a broad range of areas, from increasing teacher and principal effectiveness to turning around low-performing schools. We anticipate that after a number of i3 competitions, practices will emerge that can address challenges in each of these areas that are effective in improving student outcomes across the Nation.

Second, the i3 program links funding to the quality and extent of existing evidence showing the likelihood of a proposed practice improving student outcomes. Different tiers of grants, with increasing funding available at each tier, are linked to different levels of evidence.

Third, the i3 program supports the expansion (scaling) of effective programs by providing sufficient funding to build organizational capacity and to overcome barriers to reaching additional students. The different tiers of i3 grants comprise a funding continuum for effective programs that spans initial, localized development to implementation on a national scale, in the hope that more effective practices will displace less effective ones and lead to increases in student achievement and improvements in other student outcomes.

Fourth, the i3 program both requires and provides funding for an independent evaluation of each project to build understanding of “what works” in critical priority areas. An independent evaluation addresses issues such as for which populations or student subgroups particular practices are most effective and whether practices maintain their effectiveness as they expand to serve more students in more diverse contexts. An independent evaluation also provides an opportunity for grantees to generate the evidence needed to compete for funds at the next level of i3 funding (e.g., from a Development grant to a Validation grant; see description of the three types of grants that follows) if their projects are successful.

As in prior i3 competitions, in FY 2013 we intend to award three types of grants under this program: “Development” grants, “Validation” grants, and “Scale-up” grants. These grants differ in terms of the level of prior evidence of effectiveness required for consideration of funding, the level of scale the funded project should reach, and consequently the amount of funding available to support the project. We provide an overview to clarify the expectations for each grant type:

1. Development grants provide funding to support the development or testing of practices that are supported by evidence of promise (as defined in this notice) or strong theory (as defined in this notice) and whose efficacy should be systematically studied. We intend Development grants to support new or substantially more effective practices for addressing widely shared challenges. Development projects should be novel and significant nationally, not projects that simply implement existing practices in additional locations or support needs that are primarily local in nature.

All Development grantees must evaluate the effectiveness of the project at the level of scale proposed in the application. Development grant evaluations should assess whether the i3-supported practice is better than other approaches at increasing student achievement (as defined in this notice) or student growth (as defined in this notice), closing achievement gaps, decreasing dropout rates, increasing high school graduation rates (as defined in this notice), or increasing college enrollment and completion rates.

2. Validation grants provide funding to support expansion of projects supported by moderate evidence of effectiveness (as defined in this notice) to the national or regional level (as defined in this notice). Validation projects must further assess the effectiveness of the i3-supported practice through a rigorous evaluation, with particular focus on the populations for and the contexts in which the practice is most effective.

The outcomes of the first three i3 competitions have demonstrated that Validation grantees vary widely in their organizational maturity and capacity to expand significantly, far more than have Scale-up grantees. Given this history, we expect and consider it appropriate that each applicant would propose to use the Validation funding to build its capacity to deliver the i3-supported practice, particularly early in the funding period, to successfully reach the level of scale proposed in its application. The applicant would need to address any specific barriers to the growth or scaling of the organization or practice (including barriers related to cost-effectiveness) in order to deliver the i3-supported practice at the proposed level of scale and provide strategies to address these barriers as part of its proposed scaling plan.

All Validation grantees must evaluate the effectiveness of the practice that the supported project implements and expands. We expect that these evaluations will be conducted in a variety of contexts and for a variety of students, will identify the core elements of the practice, and will codify the practices to support adoption or replication by the applicant and other entities.

3. Scale-up grants provide funding to support expansion of projects supported by strong evidence of effectiveness (as defined in this notice) to the national level (as defined in this notice). In addition to improving outcomes for an increasing number of high-need students, we expect that Scale-up projects will generate information about the students and contexts for which a practice is most effective. We expect that Scale-up projects will increase understanding of strategies that allow organizations or practices to expand quickly and efficiently while maintaining their effectiveness.

A Scale-up grant may support the expansion of practices that have demonstrated through prior experience and rigorous evaluation that they are effective at improving student achievement. An entity applying for a Scale-up grant should use the grant funding, at least in part, to address specific barriers to the growth or scaling up of an organization or practice (including barriers related to cost-effectiveness) in order to deliver the i3-supported practice at the proposed level of scale so that the entity is well-positioned to continue expansion following the expiration of Federal funding.

Similar to Validation grants, all Scale-up grantees must evaluate the effectiveness of the i3-supported practice that the project implements and expands; this is particularly important in instances in which the proposed project includes changing the i3-supported practice in order to more efficiently reach the proposed level of scale (for example, by developing technology-enabled training tools). We expect that these evaluations would be conducted in a variety of contexts and for a variety of students in order to determine the context(s) and population(s) for which the i3-supported practice is most effective. Regardless, the evaluation of a Scale-up grant must identify core elements of and codify the i3-supported practice that the project implements to support adoption or replication by other entities.

Proposed Priorities

This notice contains 10 proposed priorities. In addition, in any i3 competition we may include priorities from the notice of final supplemental priorities and definitions for discretionary grant programs, published in the Federal Register on December 15, 2010 (75 FR 78486), and corrected on May 12, 2011 (76 FR 27637) (Supplemental Priorities). We are not proposing in this notice priorities in such areas as early learning or standards and assessments, which are already included in the Supplemental Priorities, because the language in the Supplemental Priorities adequately addresses those areas for the purposes of the i3 program.

Proposed Priorities

Background

The original set of four absolute priorities that the Department used for the FY 2010 i3 competition focused on the four assurances (or education reform areas) the Department used in implementing multiple programs funded under ARRA. We continue to consider these reform areas important and, thus, either include them in these proposed priorities or may include them in future competitions through the Supplemental Priorities.

The original i3 priorities were written broadly and generated a wide range of projects in the first three competitions. Now we are interested in supporting a more focused set of projects within areas of acute need and in more directly addressing particular challenges. Thus, we propose to modify our approach to the structure of the priorities so that each priority area includes the particular needs that the Secretary may address when establishing the priorities for a particular i3 competition. Our intent is to establish the flexibility to select from a variety of possible project focus areas within a given priority rather than using broad priorities as we have in the past; however, we expect to use only a subset of the priorities and the project focus areas within them in any particular future notice inviting applications. The Department will consider several factors when selecting the priorities to use in a given year, including the Administration's policy priorities, the need for new solutions in a particular priority area, other available funding for a particular priority area, and the results and lessons learned from prior i3 competitions. Further, the Department will consider the level of evidence or research available across the different priorities when determining which of the priorities would be most appropriate for the different types of grants under the i3 program. In a given year, the notice inviting applications will provide a concise list of the priorities that will be used for that year's i3 competition.

We propose that the Secretary may use any of the priorities established in the notice of final priorities, requirements, definitions, and selection criteria when establishing the priorities for each particular type of grant (Development, Validation, and Scale-up) in an i3 competition in FY 2013 and in subsequent years.

Proposed Priority 1—Improving the Effectiveness of Teachers or Principals

Background: Research indicates that teachers and principals are the most critical in-school factors in improving student achievement.^[1] Proposed priority 1, therefore, focuses on improving the effectiveness of teachers and principals. Specifically, the proposed priority focuses on all dimensions of the teacher and principal career path and seeks to identify effective methods for recruiting, preparing, supporting, evaluating, and retaining effective principals and teachers, particularly at schools that serve high-needs students.

The proposed priority highlights the need for schools and districts to consider how to recruit effective teachers and principals, create distinct career pathways based on the strengths of its teachers and principals and the needs of its schools, and develop evaluation systems that provide information that can be used to provide timely and useful feedback for teachers and principals. Schools and districts can use these evaluation data to identify and provide necessary resources and tailored professional development in order to support the teachers and principals currently in the schools and to improve the processes for recruiting new talent. Providing teachers with tailored development and supports is important for improving teacher effectiveness and retaining teachers to ensure all schools have highly effective teachers and principals. Thus, the priority includes developing professional development supports and tools for teachers, including creating and implementing models that help teachers utilize time and resources more efficiently while maintaining or improving outcomes.

Finally, to ensure that all schools, especially those serving high-need students, benefit from projects funded under this priority, the priority also supports efforts to equitably distribute effective teachers and principals among schools.

Proposed Priority 1—Improving the Effectiveness of Teachers or Principals

Under this proposed priority, we would provide funding to projects that address one or more of the following priority areas:

(a) Developing new methods and sources for recruiting:

(1) Highly effective teachers (as defined in this notice);

(2) Highly effective principals (as defined in this notice); or

(3) Highly effective teachers and principals (as defined in this notice).

(b) Developing models for teacher preparation that deepen pedagogical knowledge and skills, such as knowledge of instructional practices or knowledge and skills in classroom management, or that deepen pedagogical content knowledge, that have been demonstrated to improve student achievement.

(c) Developing models of induction and support for improving the knowledge and skills of novice teachers to increase teacher retention, improve teaching effectiveness, and accelerate student performance.

(d) Creating career pathways with differentiated opportunities and roles for teachers or principals, which may include differentiated compensation.

(e) Designing and implementing teacher or principal evaluation systems that provide clear, timely, and useful feedback, including feedback that identifies areas for improvement and that guides professional development for teachers and principals.

(f) Developing supports for ongoing development and improvement of teachers, principals, or instructional leaders, such as local and virtual communities, tools, training, and other mechanisms.

(g) Increasing the equitable distribution of effective teachers or principals across schools.

(h) Extending the reach of highly effective teachers to more students such as through developing and implementing school models that improve conditions for teaching and learning; or offering new opportunities for teachers to collaborate to accelerate student performance.

(i) Other projects addressing pressing needs related to improving teacher or principal effectiveness.

Proposed Priority 2—Improving Low-Performing Schools

Background: Approximately 10 percent of all high schools produce nearly half of the Nation's dropouts.^[2] Proposed priority 2 addresses the pressing need to ensure all students receive a quality K-12 education by providing funding for activities that are designed to accelerate the performance of severely low-performing schools and the schools that feed students into them. Given the range of schools that this proposed priority aims to address, we are designing this priority to identify and support multiple approaches that can successfully turn around low-performing schools and improve outcomes for students in them.

Providing a combination of reform strategies, including effective teachers, strong school leadership, embedded professional development, greater use of data to inform instruction, increased learning time, and collaboration among teachers, can improve instruction and student outcomes in low-performing schools. Additionally, whole-school and “wraparound” reform strategies also can be used to improve the school environment and address other non-academic factors that affect student achievement. Thus, this proposed priority supports projects that would implement these strategies in low-performing schools.

Community engagement also is crucial to successfully turning around low-performing schools, so the proposed priority provides for enhancing the capacity of external partners to support these schools. Finally, to support States and districts specifically in their ongoing school reform efforts, the proposed priority supports projects designed to expand State and district capacity to turn around low-performing schools.

Proposed Priority 2—Improving Low-Performing Schools

Under this proposed priority, we would provide funding to projects that address one or more of the following priority areas:

(a) Designing whole-school models that incorporate such strategies as providing strong school leadership; strengthening the instructional program; embedding professional development that provides teachers with frequent feedback to increase the rigor and effectiveness of their instructional practice; redesigning the school day, week, or year; using data to inform instruction and improvement; establishing a school environment that promotes a culture of high expectations and addresses non-academic factors that affect student achievement; and providing ongoing mechanisms for parent and family engagement.

(b) Changing selected elements of the school's organizational design, such as by differentiating staff roles, changing student groupings, or enhancing instructional time.

(c) Recruiting, developing, or retaining highly effective staff, specifically teachers, principals, or instructional leaders, to work in low-performing schools.

(d) Implementing “wraparound” and social supports for students that address non-academic factors that impede student learning.

(e) Developing and enhancing the capacity of external partners to support efforts to turn around low-performing schools or districts.

(f) Expanding district- or State-level capacity to turn around low-performing schools by developing systems and processes to improve State and district support and oversight.

(g) Other projects addressing pressing needs related to improving low-performing schools.

Other Proposed Requirements Related to Proposed Priority 2

To meet this priority, a project must serve schools among (1) The lowest-performing schools in the State on academic performance measures; (2) schools in the State with the largest within-school performance gaps between student subgroups described in section 1111(b)(2) of the ESEA; or (3) secondary schools in the State with the lowest graduation rate over a number of years or the largest within-school gaps in graduation rates between student subgroups described in section 1111(b)(2) of the ESEA.

Proposed Priority 3—Improving Science, Technology, Engineering, and Mathematics (STEM) Education

Background: Ensuring that all students can access and excel in STEM fields is essential to our Nation's innovation economy and future prosperity. An increasing number of careers require an understanding of STEM concepts and the application of the skills and techniques of science, technology, engineering and mathematics; this proposed priority addresses this growing need.

The President's Council of Advisors on Science and Technology (PCAST) ^[3] has produced reports on K-12 and undergraduate STEM education that provided recommendations on increasing achievement and postsecondary enrollment in STEM fields. The recommendations include cultivating and recruiting STEM teachers, creating STEM-related experiences to inspire and engage students, and encouraging partnerships among stakeholders in order to diversify pathways to STEM careers. Proposed priority 3 supports projects that would address these recommendations by revising STEM courses, making STEM learning more engaging to a wider range of students, increasing the number of effective STEM teachers, and expanding STEM education and career opportunities for groups traditionally underrepresented in the STEM fields, including minorities, individuals with disabilities, and women and girls.

Proposed Priority 3—Improving Science, Technology, Engineering, and Mathematics (STEM) Education

Under this proposed priority, we would provide funding to projects that address one or more of the following priority areas:

(a) Providing students with increased access to rigorous and engaging coursework in STEM.

(b) Redesigning STEM course content and instructional practices to engage students and increase student academic success.

(c) Developing new methods and resources for recruiting individuals with content expertise in STEM subject areas into teaching.

(d) Increasing the opportunities for high-quality preparation of, or professional development for, teachers or other educators in STEM subjects, through activities that include building content and pedagogical content knowledge.

(e) Expanding opportunities for high-quality out-of-school and extended-day activities that provide students with opportunities for deliberate practice that increase STEM learning, engagement, and expertise.

(f) Increasing the number of individuals from groups traditionally underrepresented in STEM, including minorities, individuals with disabilities, and women and girls, who are provided with access to rigorous and engaging coursework in STEM and are prepared for postsecondary study in STEM.

(g) Increasing the number of individuals from groups traditionally underrepresented in STEM, including minorities, individuals with disabilities, and women, who are teachers or educators of STEM subjects and have increased opportunities for high-quality preparation or professional development.

(h) Other projects addressing pressing needs for improving STEM education.

Proposed Priority 4—Improving Academic Outcomes for Students With Disabilities

Background: One of the primary goals of the ESEA is to improve the quality of education for all students, including students with disabilities, and ensuring the provision of an appropriate education to students with disabilities is the primary objective of the Individuals with Disabilities Education Act. Proposed priority 4 would support activities focused on improving the instruction for and assessment of students with disabilities from early learning through postsecondary education. Thus, the proposed priority would support projects that coordinate technical assistance across programs serving infants, toddlers, or preschoolers with disabilities to ensure the operation of coherent systems supporting these children and their families. And, at the postsecondary level, the priority would support projects that collect data on academic and other outcomes for students with disabilities to better understand their transition into postsecondary education and how their secondary school education prepares them for higher education.

Consistent with our approach under proposed priority 1 and recognizing the critical importance of evaluating teacher effectiveness, this proposed priority also would support projects to design and implement teacher evaluation systems that measure the performance of special education teachers and related service providers.

Finally, because we know that students with differing abilities can learn and excel at high levels, provided they receive appropriate academic and non-academic supports, this priority would support projects designed to improve academic outcomes for students with disabilities in inclusive settings.

Proposed Priority 4—Improving Academic Outcomes for Students With Disabilities

Under this proposed priority, we would provide funding to projects that address one or more of the following priority areas:

(a) Coordinating technical assistance across programs that address the needs of infants, toddlers, or preschoolers with disabilities, in order to ensure the operation of coherent systems of support for those children and their families.

(b) Designing and implementing teacher evaluation systems that define and measure effectiveness of special education teachers and related service providers.

(c) Improving academic outcomes for students with disabilities in inclusive settings.

(d) Improving postsecondary data collection and tracking of academic and related outcomes for students with disabilities to understand their transition into postsecondary education and how their secondary school education prepared them for higher education.

(e) Other projects addressing pressing needs related to improving academic outcomes for students with disabilities.

Proposed Priority 5—Improving Academic Outcomes for English Learners (ELs)

Background: School districts across the country have experienced a substantial increase in the enrollment of students who cannot speak, read, or write English well enough to participate meaningfully in educational programs without appropriate support services. Proposed priority 5 would support activities that are designed to address the language-related limitations that can impede student learning.

A student's ability to master core academic subjects depends on the student's ability to understand academic language, including discipline-specific vocabulary. Therefore, proposed priority 5 aims to increase opportunities for ELs to develop their academic and literacy skills and for ELs to build their skills in using and understanding English language oral discourse, varying and complex text types, and discipline-specific vocabulary that are typical of core academic courses.

Consistent with our approach under Proposed Priorities 1 and 4 and recognizing the critical importance of evaluating teacher effectiveness, this proposed priority also would support projects to design and implement teacher evaluation systems that measure the performance of teachers of ELs.

The proposed priority also aims to improve the high school graduation rates and college-readiness of ELs by supporting projects that would align the curriculum used in the language development and content courses in which they enroll with college- and career-ready standards as well as projects that would provide robust and targeted professional development to teachers, administrators, and other school personnel serving EL students.

Proposed Priority 5—Improving Academic Outcomes for English Learners (ELs)

Under this proposed priority, we would provide funding to projects that address one or more of the following priority areas:

(a) Increasing the number and proportion of ELs successfully completing courses in core academic subjects by developing, implementing, and evaluating new instructional approaches and tools that are sensitive to the language demands necessary to access challenging content, including technology-based tools.

(b) Aligning and implementing the curriculum and instruction used in grades 6-12 for language development and content courses to provide the academic vocabuarly and discourse skills necessary for preparing ELs to be college- and career-ready.

(c) Preparing young ELs to be on track to be college- and career-ready when they graduate from high school by developing comprehensive, developmentally appropriate, early learning programs (birth-grade 3) that are aligned with the State's high-quality early learning standards, designed to improve readiness for kindergarten, and support development of literacy and academic skills in English or in English and another language.

(d) Developing and implementing school-wide professional development for teachers, administrators, and other personnel in schools in which a significant percentage of students are ELs.

(e) Designing and implementing teacher evaluation systems that define and measure effectiveness of teachers of ELs.

(f) Other projects addressing pressing needs related to improving academic outcomes for ELs.

Proposed Priority 6—Improving Parent and Family Engagement

Background: Parents and families are instrumental in helping children improve their academic performance. Proposed priority 6 addresses the need for building parents' and families' awareness of their role in improving their children's educational outcomes and enhancing their ability to support student learning and school improvement through training. Additionally, the proposed priority addresses the corresponding need to provide professional development to school staff so that they have the skills needed to support and cultivate environments that are welcoming to parents and families and to build relationships that increase their capacity to support their children's educational needs.

Finally, to ensure that parents and families have the information they need to be full partners in their children's education, this proposed priority would support the development of tools and initiatives that provide them with ongoing access to data about their children's progress and performance.

Proposed Priority 6—Improving Parent and Family Engagement

Under this proposed priority, we would provide funding to projects that address one or more of the following priority areas:

(a) Developing and implementing initiatives that provide training for parents and families to learn skills and strategies that will support their students in improving academic outcomes.

(b) Implementing initiatives that are designed to enhance the skills and competencies of school and other administrative staff in building relationships and collaborating with families, particularly those who have been underengaged with the school(s) in the past, in order to support student achievement and school improvement.

(c) Implementing initiatives that cultivate sustainable partnerships and increase connections between parents and school staff in order to support student achievement and school improvement.

(d) Developing tools or practices that provide students and parents with improved, ongoing access to data and other information about the students' progress and performance.

(e) Other projects addressing pressing needs related to improving student outcomes by improving parent and family engagement.

Proposed Priority 7—Improving Cost-Effectiveness and Productivity

Background: It is essential for schools and LEAs to closely examine their spending practices and reallocate resources toward more efficient and more cost-effective strategies. Accordingly, through proposed priority 7, the Department continues to emphasize the importance of cost-effectiveness and productivity. Improvements in operational, organizational, and instruction processes and structures will allow organizations to achieve the best possible results in the most efficient manner.

With proposed priority 7, we continue and strengthen this focus by including specific requirements that applicants must address. These additional details clarify important elements to ensure that an applicant's proposed plan to improve productivity would provide sufficient detail about how the applicant aims to modify its processes and structures and how the applicant would evaluate whether the proposed project was cost-effective when implemented. A detailed budget, an examination of different types of costs, and a plan to monitor and evaluate the cost savings are essential to any reasoned attempt at improving productivity.

Proposed Priority 7—Improving Cost-Effectiveness and Productivity

Under this proposed priority, we would provide funding to projects that address one of the following areas:

(a) Substantially improving student outcomes without commensurately increasing per-student costs.

(b) Maintaining student outcomes while substantially decreasing per-student costs.

(c) Substantially improving student outcomes while substantially decreasing per-student costs.

Other Proposed Requirements Related to Proposed Priority 7

An application proposing to address this priority must provide—

(1) A clear and coherent budget that identifies expected student outcomes before and after the practice, the cost per student for the practice, and a clear calculation of the cost per student served;

(2) A compelling discussion of the expected cost-effectiveness of the practice compared with alternative practices;

(3) A clear delineation of one-time costs versus ongoing costs and a plan for sustaining the project, particularly ongoing costs, after the expiration of i3 funding;

(4) Identification of specific activities designed to increase substantially the cost-effectiveness of the practice, such as re-designing costly components of the practice (while maintaining efficacy) or testing multiple versions of the practice in order to identify the most cost-effective approach; and

(5) A project evaluation that addresses the cost-effectiveness of the proposed practice.

Proposed Priority 8—Effective Use of Technology

Background: Technology can improve student academic outcomes, often rapidly and in unprecedented ways. While there have been significant advances in the use of technology, the core operations of most schools and LEAs remain untouched. The Department's National Education Technology Plan 2010 ^[4] highlighted the potential of “connected teaching” to extend the reach of the most effective teachers by using online tools, and it also highlighted the need for high-quality learning resources that can reach learners wherever and whenever they are needed. Thus, proposed priority 8 supports strategies that address these needs.

Technological solutions also can be used effectively to assess the learning progress of individual students and to provide appropriate feedback to students and teachers. Proposed priority 8 would therefore support projects using instructional platforms that provide customized instruction for different learners, including integrated assessments and continuous feedback.

Proposed Priority 8—Effective Use of Technology

Under this proposed priority, we would provide funding to projects that use technology to address one or more of the following priority areas:

(a) Providing real-time access to learning experiences that are adaptive and self-improving in order to optimize the delivery of instruction to learners with a variety of learning needs.

(b) Providing students and teachers with “anytime, anywhere” access to academic content and learning experiences that they otherwise would not have access to, such as rigorous coursework that is not offered in a particular school, or effective professional development activities or learning communities enabled by technology.

(c) Developing new methods and resources for teacher preparation or professional development that increase a teacher's ability to utilize technology in the classroom to improve student outcomes.

(d) Assessing student proficiencies in complex skills, such as critical thinking and collaboration across academic disciplines.

(e) Developing and implementing technology-enabled strategies for teaching and learning, such as models and simulations, collaborative virtual environments, or “serious games,” especially for teaching concepts and content (e.g., systems thinking) that are difficult to teach using traditional approaches.

(f) Integrating technology with the implementation of rigorous college- and career-ready standards.

(g) Other projects that increase the use of technology for effective teaching and learning.

Proposed Priority 9—Formalizing and Codifying Effective Practices

Background: A primary goal of the i3 program is to identify and support the expansion of effective practices. The education field's knowledge management systems and dissemination of effective practices, particularly in instances where an effective practice could displace a less effective or ineffective practice, is underdeveloped. Proposed priority 9 aims to address these challenges and improve student outcomes by supporting strategies that identify key elements of effective practices and capturing lessons learned about the implementation of the practices. An applicant meeting this priority must commit to sharing knowledge about the practice broadly and supporting the implementation of the practice in other settings and locations in order to assess whether the practice can be successfully replicated.

Proposed Priority 9—Formalizing and Codifying Effective Practices

Under this proposed priority, we would provide funding to projects that formalize and codify effective practices. An application proposing to address this priority must, as part of its application:

(a) Identify the practice or practices that the application proposes to formalize (i.e., establish and define key elements of the practice) and codify (i.e., develop a guide or tools to support the dissemination of information on key elements of the practice) and explain why there is a need for formalization and codification.

(b) Evaluate different forms of the practice to identify the critical components of the practice that are crucial to its success and sustainability, including the adaptability of critical components to different teaching and learning environments.

(c) Provide a coherent and comprehensive plan for developing materials, training, toolkits, or other supports that other entities would need in order to implement the practice effectively and with fidelity.

(d) Commit to assessing the replicability and adaptability of the practice by supporting the implementation of the practice in a variety of locations during the project period using the materials, training, toolkits, or other supports that were developed for the i3-supported practice.

Proposed Priority 10—Serving Rural Communities

Background: Educational challenges and the corresponding solutions frequently are different in rural areas from those in urban or suburban areas. Proposed priority 10 recognizes this and would support projects that serve students from rural areas. In so doing, proposed priority 10 would help ensures that rural areas have access to and benefit from innovative education reforms that specifically address their needs.

Proposed Priority 10—Serving Rural Communities

Under this proposed priority, we would provide funding to projects that address one of the absolute priorities established for a particular i3 competition and under which the majority of students to be served are enrolled in rural local educational agencies (as defined in this notice).

Specific Requests for Comment

In addition to our general interest in receiving comment on the priorities proposed in this notice, we are particularly interested in comments related to proposed priority 7, Improving Cost-Effectiveness and Productivity, and proposed priority 5, Improving Academic Outcomes for ELs. We seek comments on whether the language of proposed priority 7 should establish a specific numeric target or threshold of cost-effectiveness or productivity improvement and, if we were to establish such a target, suggestions for what that target or threshold should be and how we should require that applicants or grantees measure progress toward and attainment of it. With regards to (c) of proposed priority 5, we seek comments on whether the Department should allow applicants to meet the priority by proposing processes, products, strategies, or practices that address instruction in English or in English and a language other than English.

We also recognize that the goals of supporting practices that are both innovative and evidence-based has the potential to limit the universe of applicants. Therefore, we are interested in receiving comments on whether we should establish a priority for applicants that have never received or partnered with an entity that has received a grant under the i3 program.

Types of Priorities

When inviting applications for a competition using one or more priorities, we designate the type of each priority as absolute, competitive preference, or invitational through a notice in the Federal Register. The effect of each type of priority follows:

Absolute priority: Under an absolute priority, we consider only applications that meet the priority (34 CFR 75.105(c)(3)). In the i3 competition, each application must choose to address one of the absolute priorities and projects are grouped by that absolute priority for the purposes of peer review and funding determinations.

Competitive preference priority: Under a competitive preference priority, we give competitive preference to an application by (1) awarding additional points, depending on the extent to which the application meets the priority (34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets the priority over an application of comparable merit that does not meet the priority (34 CFR 75.105(c)(2)(ii)).

Invitational priority: Under an invitational priority, we are particularly interested in applications that meet the priority. However, we do not give an application that meets the priority a preference over other applications (34 CFR 75.105(c)(1)).

Proposed Requirements

Background

We propose to revise some of the nonstatutory i3 program requirements that the Department has previously established based on our experiences with the three i3 competitions the Department has held to date. For example, many existing, widespread practices in the field currently lack the evidence base to compete for Scale-up or Validation grants because of limited prior investments in rigorous, high-quality evaluations and limited internal capacity to conduct these evaluations. One of the primary goals of the i3 program is to increase knowledge of what works in education for i3 grantees and non-grantees alike. As such, we propose to strengthen the project evaluation requirement so that i3 grantees will produce high-quality evaluations that estimate the impact of the i3-supported practice (as implemented at the proposed level of scale) on a relevant outcome (as defined in this notice).

Evaluations might consider whether the i3-supported practice is more effective than other approaches or its effect on improving student achievement (as defined in this notice) or student growth (as defined in this notice), closing achievement gaps, decreasing dropout rates, increasing high school graduation rates (as defined in this notice), or increasing college enrollment and completion rates.

Proposed Requirements

The Assistant Deputy Secretary proposes the following requirements for this program. We may apply one or more of these requirements in any year in which this program is in effect.

1. Innovations that Improve Achievement for High-Need Students: All grantees must implement practices that are designed to improve student achievement (as defined in this notice) or student growth (as defined in this notice), close achievement gaps, decrease dropout rates, increase high school graduation rates (as defined in this notice), or increase college enrollment and completion rates for high-need students (as defined in this notice).

2. Innovations that Serve Kindergarten-through-Grade-12 (K-12) Students: All grantees must implement practices that serve students who are in grades K-12 at some point during the funding period. To meet this requirement, projects that serve early learners (i.e., infants, toddlers, or preschoolers) must provide services or supports that extend into kindergarten or later years, and projects that serve postsecondary students must provide services or supports during the secondary grades or earlier.

3. Eligible Applicants: Entities eligible to apply for i3 grants include either of the following:

(a) An LEA.

(b) A partnership between a nonprofit organization and—

(1) One or more LEAs; or

(2) A consortium of schools.

Statutory Eligibility Requirements: Except as specifically set forth in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization that follows, to be eligible for an award, an eligible applicant must—

(a)(1) Have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA (economically disadvantaged students, students from major racial and ethnic groups, students with limited English proficiency, students with disabilities); or

(2) Have demonstrated success in significantly increasing student academic achievement for all groups of students described in that section;

(b) Have made significant improvements in other areas, such as high school graduation rates (as defined in this notice) or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data;

(c) Demonstrate that it has established one or more partnerships with the private sector, which may include philanthropic organizations, and that organizations in the private sector will provide matching funds in order to help bring results to scale; and

(d) In the case of an eligible applicant that includes a nonprofit organization, provide in the application the names of the LEAs with which the nonprofit organization will partner, or the names of the schools in the consortium with which it will partner. If an eligible applicant that includes a nonprofit organization intends to partner with additional LEAs or schools that are not named in the application, it must describe in the application the demographic and other characteristics of these LEAs and schools and the process it will use to select them.

Note about LEA Eligibility:

For purposes of this program, an LEA is an LEA located within one of the 50 States, the District of Columbia, or the Commonwealth of Puerto Rico.

Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization:

The authorizing statute specifies that an eligible applicant that includes a nonprofit organization meets the requirements in paragraphs (a) and (b) of the eligibility requirements for this program if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. For an eligible applicant that includes a nonprofit organization, the nonprofit organization must demonstrate that it has a record of significantly improving student achievement, attainment, or retention through its record of work with an LEA or schools. Therefore, an eligible applicant that includes a nonprofit organization does not necessarily need to include as a partner for its i3 grant an LEA or a consortium of schools that meets the requirements in paragraphs (a) and (b) of the eligibility requirements in this notice.

In addition, the authorizing statute specifies that an eligible applicant that includes a nonprofit organization meets the requirements of paragraph (c) of the eligibility requirements in this notice if the eligible applicant demonstrates that it will meet the requirement for private-sector matching.

4. Cost-Sharing or Matching Funds: To be eligible for an award, an applicant must demonstrate that one or more private sector organizations, which may include philanthropic organizations, will provide matching funds in order to help bring project results to scale. An eligible applicant must obtain matching funds or in-kind donations equal to an amount that the Secretary will specify in the notice inviting applications for the specific i3 competition. The Secretary will announce in the notice inviting applications when and how selected eligible applicants must submit evidence of the private-sector matching funds.

The Secretary may consider decreasing the matching requirement in the most exceptional circumstances. The Secretary will provide instructions for how to request a reduction of the matching requirement in the notice inviting applications.

5. Evidence Standards: To be eligible for an award, an application for a Development grant must be supported by one of the following:

(a) Evidence of promise (as defined in this notice);

(b) Strong theory (as defined in this notice); or

(c) Evidence of promise (as defined in this notice) or strong theory (as defined in this notice).

The Secretary will announce in the notice inviting applications which options will be used as the evidence standard for a Development grant in a given competition. Note that under (c), applicants must identify whether the application is supported by evidence of promise (as defined in this notice) or strong theory (as defined in this notice).

To be eligible for an award, an application for a Validation grant must be supported by moderate evidence of effectiveness (as defined in this notice);

To be eligible for an award, an application for a Scale-up grant must be supported by strong evidence of effectiveness (as defined in this notice).

6. Funding Categories: An applicant will be considered for an award only for the type of i3 grant (Development, Validation, or Scale-up grant) for which it applies. An applicant may not submit an application for the same proposed project under more than one type of grant.

7. Limit on Grant Awards: (a) No grantee may receive more than two new grant awards of any type under the i3 program in a single year; (b) In any two-year period, no grantee may receive more than one new Scale-up or Validation grant; and (c) No grantee may receive in a single year new i3 grant awards that total an amount greater than the sum of the maximum amount of funds for a Scale-up grant and the maximum amount of funds for a Development grant for that year. For example, in a year when the maximum award value for a Scale-up grant is $25 million and the maximum award value for a Development grant is $5 million, no grantee may receive in a single year new grants totaling more than $30 million.

8. Subgrants: In the case of an eligible applicant that is a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools, the partner serving as the applicant and, if funded, as the grantee, may make subgrants to one or more entities in the partnership.

9. Evaluation: The grantee must conduct an independent evaluation (as defined in this notice) of its project. This evaluation must estimate the impact of the i3-supported practice (as implemented at the proposed level of scale) on a relevant outcome (as defined in this notice). The grantee must make broadly available digitally and free of charge, through formal (e.g., peer-reviewed journals) or informal (e.g., newsletters) mechanisms, the results of any evaluations it conducts of its funded activities. For Scale-up and Validation grants, the grantee must also ensure that the data from its evaluation are made available to third-party researchers consistent with applicable privacy requirements.

In addition, the grantee and its independent evaluator must agree to cooperate with any technical assistance provided by the Department or its contractor and comply with the requirements of any evaluation of the program conducted by the Department. This includes providing to the Department, within 100 days of a grant award, an updated comprehensive evaluation plan in a format and using such tools as the Department may require. Grantees must update this evaluation plan at least annually to reflect any changes to the evaluation. All these updates must be consistent with the scope and objectives of the approved application.

10. Communities of Practice: Grantees must participate in, organize, or facilitate, as appropriate, communities of practice for the i3 program. A community of practice is a group of grantees that agrees to interact regularly to solve a persistent problem or improve practice in an area that is important to them.

11. Management Plan: Within 100 days of a grant award, the grantee must provide an updated comprehensive management plan for the approved project in a format and using such tools as the Department may require. This management plan must include detailed information about implementation of the first year of the grant, including key milestones, staffing details, and other information that the Department may require. It must also include a complete list of performance metrics, including baseline measures and annual targets. The grantee must update this management plan at least annually to reflect implementation of subsequent years of the project.

Proposed Definitions

Background: To ensure that terms used in the i3 program have clear and commonly understood meanings and are aligned with other Department programs, we propose the following definitions. The majority of these definitions are the same as, or substantially similar to, those we have established and used in prior i3 competitions. However, we are proposing some changes to those definitions related to evidence of effectiveness. In that regard, we are particularly interested in comments on the level of rigor required under the proposed definitions for “strong evidence of effectiveness,” “moderate evidence of effectiveness,” “evidence of promise,” and “strong theory.” We have attempted to clarify the definitions so that applicants can better understand what is required to meet each level of evidence. We have also narrowed the allowable evaluation methodologies at the strong and moderate evidence of effectiveness levels so that the allowable evaluation methodologies are those that are most likely to support causal conclusions. We welcome comments about whether the updated definitions are too restrictive or not restrictive enough and whether there are particular parts of the definitions that remain unclear or undefined.

Proposed Definitions

The Assistant Deputy Secretary proposes the following definitions for this program. We may apply one or more of these definitions in any year in which this program is in effect.

Consortium of schools means two or more public elementary or secondary schools acting collaboratively for the purpose of applying for and implementing an i3 grant jointly with an eligible nonprofit organization.

Evidence of promise means there is empirical evidence to support the theoretical linkage between at least one critical component and at least one relevant outcome presented in the logic model (as defined in this notice) for the proposed process, product, strategy, or practice. Specifically, evidence of promise means the following conditions are met:

(a) There is at least one study that is either a—

(1) Correlational study with statistical controls for selection bias;

(2) Quasi-experimental study (as defined in this notice) that meets the What Works Clearinghouse Evidence Standards with reservations; ^[5] or

(3) Randomized controlled trial (as defined in this notice) that meets the What Works Clearinghouse Evidence Standards with or without reservations; ^[6] and

(b) Such a study found a statistically significant or substantively important (defined as a difference of 0.25 standard deviations or larger), favorable association between at least one critical component and one relevant outcome presented in the logic model for the proposed process, product, strategy, or practice.

High-need student means a student at risk of educational failure or otherwise in need of special assistance and support, such as students who are living in poverty, who attend high-minority schools (as defined in this notice), who are far below grade level, who have left school before receiving a regular high school diploma, who are at risk of not graduating with a diploma on time, who are homeless, who are in foster care, who have been incarcerated, who have disabilities, or who are English learners.

High-minority school is defined by a school's LEA in a manner consistent with the corresponding State's Teacher Equity Plan, as required by section 1111(b)(8)(C) of the ESEA. The applicant must provide, in its i3 application, the definition(s) used.

High school graduation rate means a four-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1) and may also include an extended-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1)(v) if the State in which the proposed project is implemented has been approved by the Secretary to use such a rate under Title I of the ESEA.

Highly effective principal means a principal whose students, overall and for each subgroup as described in section 1111(b)(3)(C)(xiii) of the ESEA (economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with disabilities, students with limited English proficiency, and students of each gender), achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth. Eligible applicants may include multiple measures, provided that principal effectiveness is evaluated, in significant part, based on student growth. Supplemental measures may include, for example, high school graduation rates; college enrollment rates; evidence of providing supportive teaching and learning conditions, support for ensuring effective instruction across subject areas for a well-rounded education, strong instructional leadership, and positive family and community engagement; or evidence of attracting, developing, and retaining high numbers of effective teachers.

Highly effective teacher means a teacher whose students achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth. Eligible applicants may include multiple measures, provided that teacher effectiveness is evaluated, in significant part, based on student academic growth. Supplemental measures may include, for example, multiple observation-based assessments of teacher performance or evidence of leadership roles (which may include mentoring or leading professional learning communities) that increase the effectiveness of other teachers in the school or LEA.

Independent evaluation means that the evaluation is designed and carried out independent of, but in coordination with, any employees of the entities who develop a process, product, strategy, or practice and are implementing it.

Innovation means a process, product, strategy, or practice that improves (or is expected to improve) significantly upon the outcomes reached with status quo options and that can ultimately reach widespread effective usage.

Large sample means a sample of 350 or more students (or other single analysis units) who were randomly assigned to a treatment or control group, or 50 or more groups (such as classrooms or schools) that contain 10 or more students (or other single analysis units) and that were randomly assigned to a treatment or control group.

Logic model (also referred to as theory of action) means a well-specified conceptual framework that identifies key components of the proposed process, product, strategy, or practice (i.e., the active “ingredients” that are hypothesized to be critical to achieving the relevant outcomes) and describes the relationships among the key components and outcomes, theoretically and operationally.

Moderate evidence of effectiveness means one of the following conditions is met:

(a) There is at least one study of the effectiveness of the process, product, strategy, or practice being proposed that meets the What Works Clearinghouse Evidence Standards without reservations; ^[7] found a statistically significant favorable impact on a relevant outcome (as defined in this notice) (with no statistically significant unfavorable impacts on that outcome for relevant populations in the study or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse); and includes a sample that overlaps with the populations or settings proposed to receive the process, product, strategy, or practice.

(b) There is at least one study of the effectiveness of the process, product, strategy, or practice being proposed that meets the What Works Clearinghouse Evidence Standards with reservations,^[8] found a statistically significant favorable impact on a relevant outcome (as defined in this notice) (with no statistically significant unfavorable impacts on that outcome for relevant populations in the study or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse), includes a sample that overlaps with the populations or settings proposed to receive the process, product, strategy, or practice, and includes a large sample (as defined in this notice) and a multi-site sample (as defined in this notice) (Note: multiple studies can cumulatively meet the large and multi-site sample requirements as long as each study meets the other requirements in this paragraph).

Multi-site sample means more than one site, where site can be defined as an LEA, locality, or State.

National level describes the level of scope or effectiveness of a process, product, strategy, or practice that is able to be effective in a wide variety of communities, including rural and urban areas, as well as with different groups (e.g., economically disadvantaged, racial and ethnic groups, migrant populations, individuals with disabilities, English learners, and individuals of each gender).

Quasi-experimental design study means a study using a design that attempts to approximate an experimental design by identifying a comparison group that is similar to the treatment group in important respects. These studies, depending on design and implementation, can meet What Works Clearinghouse Evidence Standards with reservations ^[9] (they cannot meet What Works Clearinghouse Evidence Standards without reservations).

Randomized controlled trial means a study that employs random assignment of, for example, students, teachers, classrooms, schools, or districts to receive the intervention being evaluated (the treatment group) or not to receive the intervention (the control group). The estimated effectiveness of the intervention is the difference between the average outcome for the treatment group and for the control group. These studies, depending on design and implementation, can meet What Works Clearinghouse Evidence Standards without reservations.^[10]

Regional level describes the level of scope or effectiveness of a process, product, strategy, or practice that is able to serve a variety of communities within a State or multiple States, including rural and urban areas, as well as with different groups (e.g., economically disadvantaged, racial and ethnic groups, migrant populations, individuals with disabilities, English learners, and individuals of each gender). For an LEA-based project to be considered a regional-level project, a process, product, strategy, or practice must serve students in more than one LEA, unless the process, product, strategy, or practice is implemented in a State in which the State educational agency is the sole educational agency for all schools.

Relevant outcome means the student outcome or outcomes (or the ultimate outcome if not related to students) that the proposed project is designed to improve, consistent with the specific goals of the project and the i3 program.

Rural local educational agency means a local educational agency (LEA) that is eligible under the Small Rural School Achievement (SRSA) program or the Rural and Low-Income School (RLIS) program authorized under Title VI, Part B of the ESEA. Eligible applicants may determine whether a particular LEA is eligible for these programs by referring to information on the Department's Web site at www2.ed.gov/​nclb/​freedom/​local/​reap.html.

Strong evidence of effectiveness means that one of the following conditions is met:

(a) There is at least one study of the effectiveness of the process, product, strategy, or practice being proposed that meets the What Works Clearinghouse Evidence Standards without reservations; ^[11] found a statistically significant favorable impact on a relevant outcome (as defined in this notice) (with no statistically significant unfavorable impacts on that outcome for relevant populations in the study or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse); includes a sample that overlaps with the populations and settings proposed to receive the process, product, strategy, or practice; and includes a large sample (as defined in this notice) and a multi-site sample (as defined in this notice) (Note: multiple studies can cumulatively meet the large and multi-site sample requirements as long as each study meets the other requirements in this paragraph).

(b) There are at least two studies of the effectiveness of the process, product, strategy, or practice being proposed, each of which meets the What Works Clearinghouse Evidence Standards with reservations,^[12] found a statistically significant favorable impact on a relevant outcome (as defined in this notice) (with no statistically significant unfavorable impacts on that outcome for relevant populations in the studies or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse), includes a sample that overlaps with the populations and settings proposed to receive the process, product, strategy, or practice, and includes a large sample (as defined in this notice) and a multi-site sample (as defined in this notice).

Strong theory means a rationale for the proposed process, product, strategy, or practice that includes a logic model (as defined in this notice).

Student achievement means—

(a) For grades and subjects in which assessments are required under ESEA section 1111(b)(3): (1) A student's score on such assessments and may include (2) other measures of student learning, such as those described in paragraph (b), provided they are rigorous and comparable across schools within an LEA.

(b) For grades and subjects in which assessments are not required under ESEA section 1111(b)(3): Alternative measures of student learning and performance such as student results on pre-tests, end-of-course tests, and objective performance-based assessments; student learning objectives; student performance on English language proficiency assessments; and other measures of student achievement that are rigorous and comparable across schools within an LEA.

Student growth means the change in student achievement (as defined in this notice) for an individual student between two or more points in time. An applicant may also include other measures that are rigorous and comparable across classrooms.

Proposed Selection Criteria

Background

The proposed selection criteria are designed to ensure that applications selected for funding have the potential to generate substantial improvements in student achievement and other key outcomes and include well-articulated plans for the implementation and evaluation of the proposed project. Peer reviewers will use these criteria to determine how well an applicant's proposed project aligns with our expectations for the Development, Validation, or Scale-up grant the applicant seeks. As such, although we are proposing these criteria as a single list, the criteria selected and the number of points that each may be worth would vary by the type of i3 grant (Development, Validation, or Scale-up grant).

The proposed selection criteria are similar to those used in prior i3 competitions; the revisions reflect our experiences with their use. In particular, the selection criteria used in prior competitions did not articulate as clearly as intended our expectations for scaling up projects and what peer reviewers should assess to determine whether a project could feasibly achieve its proposed scale. In the proposed selection criteria, we include several factors that address whether there is unmet demand for the services that a grantee would provide and whether an applicant has identified and will address barriers that prevent the applicant from reaching that scale at the time of its application.

Proposed Selection Criteria

The Secretary proposes the following selection criteria for evaluating an application under this program. We may apply one or more of these criteria in any year in which this program is in effect. We propose that the Secretary may use:

• One or more of the selection criteria established in the notice of final priorities, requirements, definitions, and selection criteria;
• Any of the selection criteria in 34 CFR 75.210; criteria based on the statutory requirements for the i3 program in accordance with 34 CFR 75.209; or
• Any combination of these when establishing selection criteria for each particular type of grant (Development, Validation, and Scale-up) in any i3 competition. We propose that the Secretary may further define each criterion by selecting specific factors for it. The Secretary may select these factors from any selection criterion in the list above. In the notice inviting applications, the application package, or both we will announce the specific selection criteria that apply to a competition and the maximum possible points assigned to each criterion.

(a) Significance

In determining the significance of the proposed project, the Secretary proposes to consider one or more of the following factors:

(1) The extent to which the proposed project addresses a national need.

(2) The extent to which the proposed project addresses a challenge for which there is a national need for solutions that are better than the solutions currently available.

(3) The extent to which the proposed project would implement a novel approach as compared with what has been previously attempted nationally.

(4) The extent of the expected impact of the project on relevant outcomes (as defined in this notice), including the estimated impact of the project on student outcomes (particularly those related to student achievement (as defined in this notice)) and the breadth of the project's impact, compared with alternative practices or methods of addressing similar needs.

(5) The extent to which the proposed project demonstrates that it is likely to have a meaningful impact on relevant outcomes (as defined in this notice), particularly those related to student achievement (as defined in this notice), if it were implemented and evaluated in a variety of settings.

(6) The extent to which the proposed project will substantially improve on the outcomes achieved by other practices, such as through better student outcomes, lower cost, or accelerated results.

(7) The importance and magnitude of the proposed project's expected impact on a relevant outcome (as defined in this notice), particularly one related to student achievement (as defined in this notice).

(8) The likelihood that the project will have the estimated impact, including the extent to which the applicant demonstrates that unmet demand for the proposed project or the proposed services will enable the applicant to reach the proposed level of scale.

(9) The feasibility of national expansion if favorable outcomes are achieved.

(b) Quality of the Project Design

In determining the quality of the project design, the Secretary proposes to consider one or more of the following factors:

(1) The extent to which the proposed project addresses the national need and priorities the applicant is seeking to meet.

(2) The extent to which the proposed project addresses the absolute priority the applicant is seeking to meet.

(3) The clarity and coherence of the project goals, including the extent to which the proposed project articulates an explicit plan or actions to achieve its goals (e.g., a fully developed logic model of the proposed project).

(4) The extent to which the proposed project has a clear set of goals and an explicit plan or actions to achieve the goals, including identification of any elements of the project logic model that require further testing or development.

(5) The extent to which the proposed project will produce a fully codified practice, including a fully articulated logic model of the project by the end of the project period.

(6) The clarity, completeness, and coherence of the project goals and whether the application includes a description of project activities that constitute a complete plan for achieving those goals, including the identification of potential risks to project success and strategies to mitigate those risks.

(7) The extent to which the applicant addresses potential risks to project success and strategies to mitigate those risks.

(8) The extent to which the applicant will use grant funds to address a particular barrier or barriers that prevented the applicant, in the past, from reaching the level of scale proposed in the application.

(9) The extent to which the project would build the capacity of the applicant to scale up and sustain the project or would create an organization capable of expanding if successful outcomes are achieved.

(10) The sufficiency of the resources to support effective project implementation, including the project's plan for ensuring funding after the period of the Federal grant.

(11) The sufficiency of the resources to support effective project implementation.

(c) Quality of the Management Plan

In determining the quality of the management plan, the Secretary proposes to consider one or more of the following factors:

(1) The extent to which the management plan articulates key responsibilities and well-defined objectives, including the timelines and milestones for completion of major project activities, the metrics that will be used to assess progress on an ongoing basis, and annual performance targets the applicant will use to monitor whether the project is achieving its goals.

(2) The clarity and coherence of the applicant's multi-year financial and operating model and accompanying plan to operate the project at a national level (as defined in this notice) during the project period.

(3) The clarity and coherence of the applicant's multi-year financial and operating model and accompanying plan to operate the project at a national or regional level (as defined in this notice) during the project period.

(4) The extent to which the applicant demonstrates that it will have the resources to operate the project at the proposed level of scale during the project period and beyond the length of the grant, including the demonstrated commitment of any partners and evidence of broad support from stakeholders critical to the project's long-term success (e.g., State educational agencies, teachers' unions).

(5) The extent of the demonstrated commitment of any key partners or evidence of broad support from stakeholders whose participation is critical to the project's long-term success.

(d) Personnel

When evaluating the personnel of the proposed project, the Secretary proposes to consider one or more of the following factors:

(1) The adequacy of the project's staffing plan, particularly for the first year of the project, including the identification of the project director and, in the case of projects with unfilled key personnel positions at the beginning of the project, that the staffing plan identifies how critical work will proceed.

(2) The qualifications and experience of the project director and other key project personnel and the extent to which they have the expertise to accomplish the proposed tasks.

(3) The extent to which the project director has experience managing large, complex, and rapidly growing projects.

(4) The extent to which the project director has experience managing large, complex projects.

(5) The extent to which the project director has experience managing projects of similar size and scope as the proposed project.

(e) Quality of the Project Evaluation

In determining the quality of the project evaluation, the Secretary proposes to consider one or more of the following factors:

(1) The clarity and importance of the key questions to be addressed by the project evaluation, and the appropriateness of the methods for how each question will be addressed.

(2) The extent to which the methods of evaluation will, if well implemented, produce evidence about the project's effectiveness that would meet the What Works Clearinghouse Evidence Standards without reservations.^[13]

(3) The extent to which the methods of evaluation will, if well implemented, produce evidence about the project's effectiveness that would meet the What Works Clearinghouse Evidence Standards with or without reservations.^[14]

(4) The extent to which the methods of evaluation will provide valid and reliable performance data on relevant outcomes, particularly student achievement outcomes.

(5) The extent to which the evaluation will study the project at the proposed level of scale, including, where appropriate, generating information about potential differential effectiveness of the project in diverse settings and for diverse student population groups.

(6) The extent to which the evaluation will study the project at the proposed level of scale, including in diverse settings.

(7) The extent to which the evaluation plan includes a clear and credible analysis plan, including a proposed sample size and minimum detectable effect size that aligns with the expected project impact, and an analytic approach for addressing the research questions.

(8) The extent to which the evaluation plan includes a clear, well-documented, and rigorous method for measuring implementation of the critical features of the project, as well as the intended outcomes.

(9) The extent to which the evaluation plan clearly articulates the key components and outcomes of the project, as well as a measurable threshold for acceptable implementation.

(10) The extent to which the evaluation plan will provide sufficient information on the project's effect as compared to alternative practices addressing similar need.

(11) The extent to which the proposed project plan includes sufficient resources to carry out the project evaluation effectively.

Specific Requests for Comment

We are particularly interested in comments about whether there are important aspects of identifying promising projects or assessing the likelihood of project success that the proposed selection criteria and factors do not address. In addition, we are interested in feedback about whether there is ambiguity in the language of specific criteria or factors that will make it difficult for applicants to respond to the criteria and peer reviewers to evaluate the applications with respect to the selection criteria.

Final Priorities, Requirements, Definitions, and Selection Criteria

We will announce the final priorities, requirements, definitions, and selection criteria in a notice in the Federal Register. We will determine the final priorities, requirements, definitions, and selection criteria after considering responses to this notice and other information available to the Department. This notice does not preclude us from proposing additional priorities, requirements, definitions, or selection criteria, subject to meeting applicable rulemaking requirements.

Note:

This notice does not solicit applications. In any year in which we choose to use one or more of these priorities, requirements, definitions, and selection criteria, we invite applications through a notice in the Federal Register.

Executive Orders 12866 and 13563

Regulatory Impact Analysis

Under Executive Order 12866, the Secretary must determine whether a regulatory action is “significant” and, therefore, subject to the requirements of the Executive order and subject to review by the Office of Management and Budget (OMB). Section 3(f) of Executive Order 12866 defines a “significant regulatory action” as an action likely to result in a rule that may—

(1) Have an annual effect on the economy of $100 million or more, or adversely affect a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or Tribal governments or communities in a material way (also referred to as an “economically significant” rule);

(2) Create serious inconsistency or otherwise interfere with an action taken or planned by another agency;

(3) Materially alter the budgetary impacts of entitlement grants, user fees, or local programs or the rights and obligations of recipients thereof; or

(4) Raise novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles stated in the Executive order.

This proposed regulatory action would have an annual effect on the economy of more than $100 million because Department anticipates more than that amount will be appropriated for i3 and awarded as grants. Therefore, this proposed action is “economically significant” and subject to review by OMB under section 3(f)(1) of Executive Order 12866. Notwithstanding this determination, we have assessed the potential costs and benefits, both quantitative and qualitative, of this proposed regulatory action and have determined that the benefits would justify the costs.

The Department has also reviewed these proposed requirements under Executive Order 13563, which supplements and explicitly reaffirms the principles, structures, and definitions governing regulatory review established in Executive Order 12866. To the extent permitted by law, Executive Order 13563 requires that an agency—

(1) Propose or adopt regulations only upon a reasoned determination that their benefits justify their costs (recognizing that some benefits and costs are difficult to quantify);

(2) Tailor its regulations to impose the least burden on society, consistent with obtaining regulatory objectives, taking into account—among other things, and to the extent practicable—the costs of cumulative regulations;

(3) In choosing among alternative regulatory approaches, select those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity);

(4) To the extent feasible, specify performance objectives, rather than specifying the behavior or manner of compliance a regulated entity must adopt; and

(5) Identify and assess available alternatives to direct regulation, including providing economic incentives—such as user fees or marketable permits—to encourage the desired behavior, or provide information that enables the public to make choices.

Executive Order 13563 also requires an agency “to use the best available techniques to quantify anticipated present and future benefits and costs as accurately as possible.” The Office of Information and Regulatory Affairs of OMB has emphasized that these techniques may include “identifying changing future compliance costs that might result from technological innovation or anticipated behavioral changes.”

We are issuing these proposed priorities, requirements, definitions, and selection criteria only on a reasoned determination that their benefits justify their costs. In choosing among alternative regulatory approaches, we selected those approaches that would maximize net benefits. Based on the analysis that follows, the Department believes these proposed regulations are consistent with the principles in Executive Order 13563.

We have also determined that this regulatory action would not unduly interfere with State, local, and Tribal governments in the exercise of their governmental functions.

Discussion of Costs and Benefits

The Secretary believes that the proposed priorities, requirements, definitions, and selection criteria would not impose significant costs on eligible LEAs, nonprofit organizations, or other entities that would receive assistance through the i3 program. The Secretary also believes that the benefits of implementing the proposals contained in this notice outweigh any associated costs.

The Secretary believes that the proposed priorities, requirements, definitions, and selection criteria would result in selection of high-quality applications to implement activities that are most likely to have a significant national impact on educational reform and improvement. The proposed priorities, requirements, definitions, and selection criteria in this notice clarify the scope of activities the Secretary expects to support with program funds and the expected burden of work involved in preparing an application and implementing a project under the program. The pool of possible applicants is very large, and there is great interest in the program. During the first 3 years of implementation the Department received over 3,000 applications. Potential applicants, both LEAs and nonprofit organizations, need to consider carefully the effort that will be required to prepare a strong application, their capacity to implement a project successfully, and their chances of submitting a successful application.

Program participation is voluntary. The Secretary believes that the costs imposed on applicants by the proposed priorities, requirements, definitions, and selection criteria would be limited to paperwork burden related to preparing an application and that the benefits of implementing these proposals would outweigh any costs incurred by applicants. The costs of carrying out activities would be paid for with program funds and with matching funds provided by private-sector partners. Thus, the costs of implementation would not be a burden for any eligible applicants, including small entities. However, under the proposed selection criteria the Secretary would assess the extent to which an applicant would be able to sustain a project once Federal funding through the i3 program is no longer available. Thus, eligible applicants should propose activities that they will be able to sustain without funding from the program and, thus, in essence, should include in their project plans the specific steps they will take for sustained implementation of the proposed project. The continued proposal for the three types of grants under i3—Development, Validation, or Scale-up grants—would allow potential applicants to determine which type of grant they are best suited to apply for, based on their own priorities, resources, and capacity to implement grant activities.

Regulatory Flexibility Act Certification

The Secretary certifies that this proposed regulatory action will not have a significant economic impact on a substantial number of small entities. The small entities that this proposed regulatory action will affect are small LEAs or nonprofit organizations applying for and receiving funds under this program. The Secretary believes that the costs imposed on applicants by the proposed priorities, requirements, definitions, and selection criteria would be limited to paperwork burden related to preparing an application and that the benefits of implementing these proposals would outweigh any costs incurred by applicants.

Participation in this program is voluntary. For this reason, the proposed priorities, requirements, definitions, and selection criteria would impose no burden on small entities in general. Eligible applicants would determine whether to apply for funds, and have the opportunity to weigh the requirements for preparing applications, and any associated costs, against the likelihood of receiving funding and the requirements for implementing projects under the program. Eligible applicants most likely would apply only if they determine that the likely benefits exceed the costs of preparing an application. The likely benefits include the potential receipt of a grant as well as other benefits that may accrue to an entity through its development of an application, such as the use of that application to spur educational reforms and improvements without additional Federal funding.

The U.S. Small Business Administration Size Standards defines as “small entities” for-profit or nonprofit institutions with total annual revenue below $7,000,000 or, if they are institutions controlled by small governmental jurisdictions (that are comprised of cities, counties, towns, townships, villages, school districts, or special districts), with a population of less than 50,000. The Urban Institute's National Center for Charitable Statistics reported that of 196,663 nonprofit organizations that had an educational mission and reported revenue to the IRS by March of 2012, 168,784 (or about 86 percent) had revenues of less than $5 million. In addition, there are approximately 16,000 LEAs in the country that meet the definition of small entity. However, the Secretary believes that only a small number of these entities would be interested in applying for funds under this program, thus reducing the likelihood that the proposals contained in this notice would have a significant economic impact on small entities. As discussed earlier, the number of applications received during the last 3 competitions from any type of applicant is approximately 3,000.

In addition, the Secretary believes that the proposed priorities, requirements, definitions, and selection criteria discussed in this notice do not impose any additional burden on small entities applying for a grant than they would face in the absence of the proposed action. That is, the length of the applications those entities would submit in the absence of the regulatory action and the time needed to prepare an application would likely be the same.

Further, the proposed action may help small entities determine whether they have the interest, need, or capacity to implement activities under the program and, thus, prevent small entities that do not have such an interest, need, and capacity from absorbing the burden of applying, or assist those entities in determining whether they should seek a capable partner to pursue the application process.

This proposed regulatory action would not have a significant economic impact on small entities once they receive a grant because they would be able to meet the costs of compliance using the funds provided under this program and with any matching funds provided by private-sector partners.

The Secretary invites comments from small nonprofit organizations and small LEAs as to whether they believe this proposed regulatory action would have a significant economic impact on them and, if so, requests evidence to support that belief.

Accounting Statement

As required by OMB Circular A-4 (available at www.whitehouse.gov/​sites/​default/​files/​omb/​assets/​omb/​circulars/​a004/​a-4.pdf), in the following table we have prepared an accounting statement showing the classification of the expenditures associated with the provisions of this regulatory action. This table provides our best estimate of the changes in annual monetized transfers as a result of this regulatory action. Expenditures are classified as transfers from the Federal Government to LEAs and nonprofit organizations.

Accounting Statement Classification of Estimated Expenditures

[In millions]

Category	Transfers
Annualized Monetized Transfers	$140.9 million.
From Whom To Whom?	From the Federal Government to LEAs and nonprofit organizations.

Paperwork Reduction Act of 1995

The requirements and selection criteria proposed in this notice will require the collection of information that is subject to review by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 (44 U.S.C. 3501-3520). The burden associated with the i3 program was approved by OMB under OMB Control Number 1855-0021, which expires on October 31, 2013. These proposed priorities, requirements, definitions, and selection criteria would allow the Department to improve the design of the i3 program to better achieve its purposes and goals. However, the revisions do not change the number of applications an organization may submit or the burden that an applicant would otherwise incur in the development and submission of a grant application under the i3 program. Therefore, the Department expects that this proposed regulatory action will not affect the total burden of hours.

Intergovernmental Review: This program is subject to Executive Order 12372 and the regulations in 34 CFR part 79. One of the objectives of the Executive order is to foster an intergovernmental partnership and a strengthened federalism. The Executive order relies on processes developed by State and local governments for coordination and review of proposed Federal financial assistance.

This document provides early notification of our specific plans and actions for this program.

Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., braille, large print, audiotape, or compact disc) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT.

Electronic Access to This Document: The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available via the Federal Digital System at: www.gpo.gov/​fdsys. At this site you can view this document, as well as all other documents of this Department published in the Federal Register, in text or Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at the site.

You may also access documents of the Department published in the Federal Register by using the article search feature at: www.federalregister.gov. Specifically, through the advanced feature at this site, you can limit your search to documents published by the Department.

Dated: December 11, 2012.

James H. Shelton, III,

Assistant Deputy Secretary for Innovation and Improvement.

Footnotes

1.  Wright, S.P., Horn, S.P., Sanders, W.L. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education 11:57-67; Rivkin, S.G., Hanushek, E.A., Kain, J.F. (2005). Teachers, schools, and academic achievement. Economerica, 73(2):417-458.

Leithwood, K., Louis, K.S., Anderson, S., and Wahlstrom, K. (2004). Review of research: How leadership influences student learning. University of Minnesota, Center for Applied Research and Educational Improvement. Found at www.cehd.umn.edu/​carei/​Leadership/​ReviewofResearch.pdf.

2.  Balfanz, R., Bridgeland, J.M., Horning Fox, J., Moore, L.A. (2010). Building a Grad nation: Progress and Challenge in Ending the High School Dropout Epidemic 2010-2011 Annual Update. See www.americaspromise.org/​Our-Work/​Grad-Nation/​Building-a-Grad-Nation.aspx.

3.  See www.whitehouse.gov/​administration/​eop/​ostp/​pcast/​docsreports.

4.  See www.ed.gov/​edblogs/​technology/​netp-2010/​.

5.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

6.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

7.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

8.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

9.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

10.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.

11.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

12.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

13.  See What Works Clearinghouse Procedures and Standards Handbook. (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.

14.  See What Works Clearinghouse Procedures and Standards Handbook (Version 2.1, September 2011), which can currently be found at the following link: http://ies.ed.gov/​ncee/​wwc/​DocumentSum.aspx?​sid=​19.