E9-25600. Race to the Top Fund

Catalog of Federal Domestic Assistance (CFDA) Number: 84.395C.

AGENCY:

Department of Education.

ACTION:

Notice of public meetings and request for input to gather technical expertise pertaining to a possible Race to the Top program, and provide technical assistance for the development and implementation of high-quality assessments based on common standards.

SUMMARY:

By March 2010, the Secretary of Education (Secretary) intends to announce a competition for a program that would support one or more consortia of States that are working toward jointly developing and implementing common, high-quality assessments aligned with a consortium's common set of K-12 standards that are internationally benchmarked and that build toward college and career readiness by the time of high school completion. To inform the design of this program and the development of a notice inviting applications that establishes the requirements for this competition, and to provide technical assistance to States, the Secretary is seeking input from States, technical experts, and members of the public through public meetings and written submissions. Following the public meetings and review of the written submissions, the Department intends to publish a notice inviting applications for such a competition.

DATES:

Public meetings will be held on the dates and at the locations specified later in this notice. Written submissions must be received by the Department on or before 5:00 p.m., Eastern time, on Wednesday, December 2, 2009.

ADDRESSES:

For those submitting written input, we encourage submissions by e-mail using the following address: [e-mail]. You must include the term “Race to the Top Assessment Program” in the subject line of your e-mail. If you prefer to send your input by mail, address it to Office of Elementary and Secondary Education, Attention: Race to the Top Assessment Program—Public Input Meetings, U.S. Department of Education, 400 Maryland Avenue, SW., room 3E108, Washington, DC 20202.

FOR FURTHER INFORMATION CONTACT:

U.S. Department of Education, 400 Maryland Avenue, SW., room 3E108, Washington, DC 20202. Telephone: [phone] or by e-mail: [e-mail].

If you use a telecommunications device for the deaf (TDD), call the Federal Relay Service (FRS), toll free, at 1-800-877-8339.

SUPPLEMENTARY INFORMATION:

Background: The Race to the Top Fund, authorized under the American Recovery and Reinvestment Act of 2009 (ARRA), Public Law 111-5, provides $4.35 billion for competitive grants to States to encourage and reward States that are creating the conditions for education innovation and reform; implementing ambitious plans in the four education reform areas described in the ARRA; and achieving significant improvement in student outcomes, including making substantial gains in student achievement, closing achievement gaps, improving high school graduation rates, and ensuring student preparation for success in college and careers.

The Department is considering implementing two separate programs under the Race to the Top Fund. The first, a general program, will be announced later this Fall through a notice inviting applications and notice of final priorities, requirements, definitions, and selection criteria. Under this general program, the Department will award approximately $4 billion to State applicants that have demonstrated that they have created certain conditions for reform and for increased student achievement and propose to develop and implement comprehensive reform strategies that are integrated across the four ARRA education reform areas.

Through this notice, we are seeking input on a second proposed program (Assessment Program), which would provide for approximately $350 million in grants to consortia of States for the development of common, high-quality assessments aligned with an applicant consortium's common set of K-12 standards that are internationally benchmarked and that build toward college and career readiness by the time of high school completion. In addition, at least 50 percent of the award to States under this proposed competition must be used to provide subgrants to local educational agencies (LEAs), including public charter schools identified as LEAs under State law, based upon LEAs' relative shares of funding under Part A of Title I of the Elementary and Secondary Education Act of 1965, as amended (ESEA).

At a later date and depending upon the input from the public meetings and written submissions described in this notice, the Secretary intends to issue a notice inviting applications for a competition for this second program that will set forth the requirements and criteria for the submission of applications. If the Secretary determines that it is not feasible to conduct this second program, the $350 million designated for this program will revert to fund additional grants under the general Race to the Top program.

Because requirements for an assessment program are highly technical, the Department wishes to solicit input from assessment experts, directors of large-scale assessment programs, States, other key stakeholders, and members of the public to inform the design and development of this program, including the notice inviting applications and to provide technical assistance to States. Therefore, the Department will hold a series of public meetings at which invited experts and members of the public will have the opportunity to provide input, as well as the opportunity to submit written input. Should we decide to implement this Assessment Program by holding a competition, we do not intend to conduct notice and comment rulemaking. Section 437(d)(1) of the General Education Provisions Act, 20 U.S.C. 1232(d)(1), allows the Department to waive rulemaking for the first grant competition under a new or substantially revised program authority. This would be the first competition for an Assessment Program under the Race to the Top Fund.

In addition to informing the design and development of the potential competition and the notice inviting applications, the Department anticipates that these meetings will also enable both the Department and States to learn more about the design, development, and implementation of high-quality assessments and will support State consortia in developing the highest-quality proposals with the greatest likelihood of impact. We anticipate that States, in particular, will acquire critical knowledge about best practices in assessments, and then be able to employ that knowledge in developing their applications and in designing high-quality assessments.

Details of Public Meetings

Structure of Public Meetings

The Department anticipates that each meeting will have two components as follows:

(1) Input from invited panels of experts and stakeholders:

○ Each meeting will have an invited set of panelists who will have a set amount of time to respond individually to the questions in this notice.

○ The Department representatives will then ask questions of individual panelists and facilitate cross-panelist discussion.

(2) Open opportunity to share input:

○ Each meeting will have 60 to 90 minutes dedicated to opportunities for interested members of the public, who have registered to speak, to respond to the questions in this notice.

○ Each individual scheduled to speak will have 5 minutes to provide oral input.

○ Written submissions will also be accepted as described in the “Submission of Written Input” section.

Each meeting will likely focus on a particular topic as indicated in the next section. The Department will share any updates, including posting additional questions, online at http://www.ed.gov/​programs/​racetothetop-assessment.index.html.

Topic Areas, Dates, Times, Locations, and Registration Information

The public meetings will occur on the following dates at the times and locations indicated below.

• Topic Area: General Assessment:

○ Thursday, November 12; in Boston, MA; at the Embassy Suites Boston at Logan Airport, 207 Porter Street, Boston, MA; from 10 a.m. to 5 p.m.

○ Tuesday, November 17; in Atlanta, GA; at the Atlanta Airport Marriott, 4711 Best Road, Atlanta, GA; from 10 a.m. to 5 p.m.

○ Tuesday, December 1; in Denver, CO; at the Grand Hyatt Denver, 1750 Welton Street, Denver, CO; from 10 a.m. to 5 p.m.

• Topic Area: High School Assessments:

○ Friday, November 13; in Boston, MA; at the Embassy Suites Boston at Logan Airport, 207 Porter Street, Boston, MA; from 1:30 p.m. to 5 p.m.

• Topic Area: Assessment of Students with Disabilities:

○ Wednesday, November 18; in Atlanta, GA; at the Atlanta Airport Marriott, 4711 Best Road, Atlanta, GA; from 9 a.m. to 12:30 p.m.

• Topic Area: Assessment of English Language Learners:

○ Wednesday, December 2; in Denver, CO; at the Grand Hyatt Denver, 1750 Welton Street, Denver, CO; from 9 a.m. to 12:30 p.m.

• Topic Area: Technology and Innovation in Assessment:

○ Friday, November 13; in Boston, MA; at the Embassy Suites Boston at Logan Airport, 207 Porter Street, Boston, MA; from 9 p.m. to 12:30 p.m.

Attendance: If you are interested in attending an event, you must register by sending an e-mail to [e-mail]. You must include in the subject line of your email the city in which you wish to attend, and the date(s) on which you wish to attend. Registrations will be processed on a first-come, first-served basis with space reserved for State participants.

Providing input: If you are interested in speaking during the open input portion of the meeting, you must register by sending an e-mail to [e-mail]. You must include in the subject line of your email the word “Speaker”, the city in which you wish to speak, and the topic area to which you wish to respond. Registrations will be processed on a first-come, first-served basis. People who are unable to attend a meeting in person or who do not register early enough to speak during the meeting are encouraged to submit written input.

Assistance to Individuals With Disabilities at the Public Meetings

The meeting sites will be accessible to individuals with disabilities and sign language interpreters will be available. If you need an auxiliary aid or service other than a sign language interpreter to participate in the meeting (e.g., interpreting service such as oral, cued speech, or tactile interpreter; assisted listening device; or materials in alternate format), notify the contact person listed under FOR FURTHER INFORMATION CONTACT at least two weeks before the scheduled meeting date. Although we will attempt to meet a request we receive after this date, we may not be able to make available the requested auxiliary aid or service because of insufficient time to arrange it.

Submission of Written Input

All interested parties, including those who cannot attend a meeting or from whom we do not have time to hear at a meeting, may submit written input in response to this notice.

Written input will be accepted at the meeting site or via e-mail and mail at the addresses listed in the ADDRESSES section of this notice. Written input must be submitted by the date listed in the DATES section.

When submitting input at the meetings, we request that you submit three written copies and an electronic file (CD or diskette) of your statement at the meeting. Please include your name and contact information on the written and electronic files.

Both at the meetings and in your written submission, we encourage you to be as specific as possible. To ensure that your input is fully considered, we urge you to identify clearly the specific question, purpose, and characteristic that each of your suggestions addresses and to arrange your submission in the order of the questions listed later in this notice. Please also include a description of your involvement, if any, in statewide assessment practices.

Sharing Input Publicly

The Department is committed to gathering and sharing publicly the input from the meetings and written submissions. Each meeting will be video-taped and/or transcribed, and the video and/or transcript will be available for viewing at http://www.ed.gov/​programs/​racetothetop-assessment.index.html. All written input received will be available for viewing via this Web site, as well.

Assessment Program Design and Questions

The Assessment Program is intended to support consortia of States working toward jointly developing and implementing a next generation of common summative assessments that are aligned with a common set of kindergarten-through-grade-12 internationally benchmarked, college and career ready standards that model and support effective teaching and student learning. Such summative assessments would allow students, including students with disabilities and English language learners, to demonstrate at each grade level tested their mastery of knowledge and skills and the extent to which each student is on track to college and career readiness by the time of high school graduation.

In designing the requirements for this program, the Secretary is particularly interested in innovative and effective approaches to assessment that will assist States in creating powerful and useful systems of assessment that meet these requirements.

In the following paragraphs, we have provided a framework that outlines the characteristics we believe should be required or encouraged in assessment systems supported by a grant under this proposed program. We then list the specific questions on which we seek input, taking into account this framework. In addition, at least 50 percent of the award to States under any Race to the Top competition must be used to provide subgrants to local educational agencies (LEAs), including public charter schools identified as LEAs under State law, based upon LEAs' relative shares of funding under Part A of Title I of the Elementary and Secondary Education Act of 1965, as amended (ESEA). This notice also highlights potential uses and questions for the LEA portion of the funding.

It is important to note that this proposed program, the public meetings, and the framework below would focus on the design and quality of assessment systems and not accountability policies, such as those described in section 1116 of the ESEA. Given the pending reauthorization of the ESEA, we intend that the Assessment Program would support the development of the best possible assessments that could be not only appropriately used by States under the current ESEA assessment and accountability requirements, but could also serve additional purposes as outlined later in this notice.

Framework

Design of Assessment Systems—General Requirements

The Department is particularly interested in supporting the development of summative assessments that measure—

• Individual student achievement as measured against standards that build toward college and career readiness by the time of high school completion;
• Individual student growth (that is, the change in student achievement data for an individual student between two or more points in time); and
• The extent to which each individual student is on track, at each grade level tested, toward college or career readiness by the time of high school completion.

At a minimum, we would expect that the common assessments would measure each of these elements in the subject areas of reading/language arts and mathematics, and would provide information for each student annually in grades 3 through 8, and provide information at the high school level about each student's college and/or career readiness. The assessments need not be limited to a single end-of-year assessment but could include multiple summative components administered at different points during the school year. Moreover, the assessments might be viewed as replacing rather than adding to the assessments currently in use in States participating in the consortia.

Information gathered from the assessments should be useable in informing—

• Teaching, learning, and program improvement;
• Determinations of school effectiveness;
• Determinations of principal and teacher effectiveness to inform evaluation and the provision of support to teachers and principals; and
• Determinations of individual student college and career readiness, such as determinations made for high school exit decisions, college course placement in credit-bearing classes, or college entrance.

Design of Assessment Systems—Required Characteristics

With respect to the design of the assessment system, the Department would likely require that the assessments, at a minimum, meet the following characteristics:

(1) Reflect and support good instructional practice by eliciting complex responses and demonstrations of knowledge and skills consistent with the goal of being college and career ready by the time of high school completion;

(2) Be accessible to the broadest possible range of students, with appropriate accommodations for students with disabilities and English language learners;

(3) Contain varied and unpredictable item types and content sampling, so as not to create incentives for inappropriate test preparation and curriculum narrowing;

(4) Produce results that can be aggregated at the classroom, school, LEA, and State levels;

(5) Produce reports that are relevant, actionable, timely, accurate, and displayed in ways that are clear and understandable for target audiences, including teachers, students and their families, schools, LEAs, communities, States, institutions of higher education, policymakers, researchers, and others;

(6) Make effective and appropriate use of technology;

(7) Be valid, reliable, and fair;

(8) Be appropriately secure for the intended purposes;

(9) Have the fastest possible turnaround time on scoring, without forcing the use of lower-quality assessment items; and

(10) Be able to be maintained, administered, and scored at a cost that is sustainable over time.

Design of Assessment Systems—Desired Characteristics

In addition, the Department is particularly interested in assessment systems in which—

(1) Teachers are involved in scoring of constructed responses and performance tasks in order to measure effectively students' mastery of higher-order content and skills and to build teacher expertise and understanding of performance expectations;

(2) The assessment approach can be easily adapted to include summative assessments in other content areas (e.g., science, social studies) in the future;

(3) The technology “platform” created for summative assessments supports assessment and item development, administration, scoring, and reporting that increases the quality and cost-effectiveness of assessments; and

(4) The technology infrastructure created for summative assessments can be easily adapted to support practitioners and professionals in the development, administration, and/or scoring of high-quality interim assessments.

Design of Assessment Systems—LEA-Level Activities

With funds that are directed to LEAs under this program, the Department is interested in supporting LEA-level activities that are designed by the State consortium to support development and implementation of its assessment system. With respect to LEA-level funds, the Department would likely require that the funds be used to support the following types of activities conducted by LEAs that choose to participate:

• Pilot testing of the new assessments with different populations, including English-language learners and students with disabilities;
• Designing systems to support and enable effective and consistent teacher scoring, providing professional development support for these activities, and implementing them statewide;
• Statewide transition to the consortium's K-12 common, college and career ready, internationally benchmarked standards, with new high-quality assessments (consistent with the State plans described in the notice of proposed priorities, requirements, definitions, and selection criteria for the Race to the Top Fund general program (74 FR 37804, July 29, 2009). Such LEA activities might include: developing a rollout plan for implementation of the standards and assessments together with all of their supporting components; developing or acquiring, disseminating, and implementing high-quality instructional materials and assessments; developing or acquiring and delivering high-quality professional development to support the transition to new standards and assessments; and engaging in other strategies that translate the standards and information from assessments into classroom practice for all students; and
• Development of formative or interim assessments that align with State summative assessments as part of a comprehensive assessment system.

Questions for Input

The specific questions on which the Department seeks input are listed below. All input, including expert presentations and discussions, public input, and written submissions, should be primarily focused on responding to these questions in the context of the framework outlined above, and may also provide input on the framework itself. We encourage you to make your input as specific as possible, to provide evidence to support your proposals, and to present the information in a context and format that will be helpful to States implementing high-quality assessments. Questions focus on the topics of general assessment, high school assessment, assessment of English language learners, assessment of students with disabilities, technology and innovation in assessment, specific technical assessment questions, and project management.

To ensure that your input is fully considered in the development of the notice inviting applications, we urge you to identify clearly the specific question, purpose, or characteristic that you are addressing, and to arrange your input in the order of the questions as they are listed in the next section.

General Assessment Questions

(1) Propose an assessment system (that is, a series of one or more assessments) that you would recommend and that meets the general requirements and required characteristics described in this notice. Describe how this assessment system would address the tensions or tradeoffs in meeting all of the general requirements and required characteristics. Describe the strengths and limitations of your recommended system, including the extent to which it is able to validly meet each of the requirements described in this notice. Where possible, provide specific illustrative examples.

(2) For each assessment proposed in response to question (1), describe the—

• Optimal design, including—

○ Type (e.g., norm-referenced, criterion-referenced, adaptive, other);

○ Frequency, length, and timing of assessment administrations (including a consideration of the value of student, teacher, and administrative time);

○ Format, item-type specifications (including the pros and cons of using different types of items for different purposes), and mode of administration;

○ Whether and how the above answers might differ for different grade levels and content areas;

• Administration, scoring, and interpretation of any open-ended item types, including methods for ensuring consistency in teacher scoring;
• Approach to releasing assessment items during each assessment cycle in order to ensure public access to the assessment questions; and
• Technology and other resources needed to develop, administer, and score the assessments, and/or report results.

(3) ARRA requires that States award at least 50 percent of their Race to the Top funds to LEAs. The section of this notice entitled Design of Assessment Systems—LEA-Level Activities, describes how LEAs might be required to use these funds. What activities at the LEA level would best advance the transition to and implementation of the consortium's common, college and career ready standards and assessments?

(4) If a goal is that teachers are involved in the scoring of constructed responses and performance tasks in order to measure effectively students' mastery of higher-order content and skills and to build teacher expertise and understanding of performance expectations, how can such assessments be administered and scored in the most time-efficient and cost-effective ways?

(5) Given the assessment design you proposed in response to question (1), what is your recommended approach to competency-based student testing versus grade-level-based student testing? Why? How would your design ensure high expectations for all students?

(6) Given the assessment design you proposed in response to question (1), how would you recommend that the assessments be designed, timed, and scored to provide the most useful information on teacher and principal effectiveness?

Specific Technical Assessment Questions

(1) What is the best technical approach for ensuring the vertical alignment of the entire assessment system across grades (e.g., grades 3 through 8 and high school)?

(2) What would be the best technical approach for ensuring external validity of such an assessment system, particularly as it relates to postsecondary readiness and high-quality internationally benchmarked content standards?

(3) What is the proportion of assessment questions that you recommend releasing each testing cycle in order to ensure public access to the assessment while minimizing linking risk? ^[1] What are the implications of this proportion for the costs of developing new assessment questions and for the costs and design of linking studies across time?

High School Assessment

Provide recommendations on the optimal approach to measuring each student's college and career readiness by the time of high school completion. In particular, consider—

(1) How would you demonstrate that high school students are on track to college and career readiness, and at what points throughout high school would you recommend measuring this?

Discuss your recommendations on the use of end-of-course assessments versus comprehensive assessments of college and career readiness.

Note:

If you recommend end-of-course assessments, please share your input on how to reconcile the fact that college and career ready standards might not include all of the topics typically covered in today's high school courses.

Assessment of English Language Learners

(1) Provide recommendations for the development and administration of assessments for each content area that are valid and reliable for English language learners. How would you recommend that the assessments take into account the variations in English language proficiency of students in a manner that enables them to demonstrate their knowledge and skills in core academic areas? Innovative assessment designs and uses of technology have the potential to be inclusive of more students. How would you propose we take this into account?

(2) In the context of reflecting student achievement, what are the relative merits of developing and administering content assessments in native languages? What are the technical, logistical, and financial requirements?

Assessment of Students With Disabilities

(1) Taking into account the diversity of students with disabilities who take the assessments, provide recommendations for the development and administration of assessments for each content area that are valid and reliable, and that enable students to demonstrate their knowledge and skills in core academic areas. Innovative assessment designs and uses of technology have the potential to be inclusive of more students. How would you propose we take this into account?

Technology & Innovation in Assessment

(1) Propose how you would recommend that different innovative technologies be deployed to create better assessments, and why. Please include illustrative examples in areas such as novel item types, constructed response scoring solutions, uses of mobile computing devices, and so on.

(2) We envision the need for a technology platform for assessment development, administration, scoring, and reporting that increases the quality and cost-effectiveness of the assessments. Describe your recommendations for the functionality such a platform could and should offer.

(3) How would you create this technology platform for summative assessments such that it could be easily adapted to support practitioners and professionals in the development, administration, and/or scoring of high-quality interim assessments?

(4) For the technology “platform” vision you have proposed, provide estimates of the associated development and ongoing maintenance costs, including your calculations and assumptions behind them.

Project Management

(1) Provide estimates of the development, maintenance, and administration costs of the assessment system you propose, and your calculations and assumptions behind them.

(2) Describe the range of development and implementation timelines for your proposed assessment system, from the most aggressive to more conservative, and describe the actions that would be required to achieve each option.

(3) How would you recommend organizing a consortium to achieve success in developing and implementing the proposed assessment system? What role(s) do you recommend for third parties (e.g., conveners, project managers, assessment developers/partners, intermediaries)? What would you recommend that a consortium demonstrate to show that it has the capacity to implement the proposed plan?

Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., braille, large print, audiotape, or computer diskette) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT.

Electronic Access to This Document: You can view this document, as well as all other documents of this Department published in the Federal Register, in text or Adobe Portable Document Format (PDF) on the Internet at the following site: http://www.ed.gov/​news/​fedregister.

To use PDF you must have Adobe Acrobat Reader, which is available free at this site. If you have questions about using PDF, call the U.S. Government Printing Office (GPO), toll free, at 1-888-293-6498; or in the Washington, DC, area at [phone].

Note:

The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available on GPO Access at: http://www.gpoaccess.gov/​nara/​index.html.

Dated: October 20, 2009.

Arne Duncan,

Secretary of Education.

Footnotes

1.  Michael J. Kolen and Robert L. Brennan, Test Equating, Scaling, and Linking: Methods and Practices (2nd ed), 2004, New York: Springer-Verlag. See especially: Chapter 6, “Item Response Theory Methods,” Section 9, “Using IRT Calibrated Item Pools”; and Chapter 8, “Practical Issues in Equating,” Section 1, “Equating and the Test Development Process” and Section 6, “Conditions Conducive to Satisfactory Equating.”

See also Hedges, L. V., and Vevea, J. L. (1997). A study of equating in NAEP. http://www.air.org/​publications/​documents/​hedges_​rpt.pdf.