[Federal Register Volume 63, Number 207 (Tuesday, October 27, 1998)]
[Rules and Regulations]
[Pages 57570-57575]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 98-28729]
[[Page 57569]]
_______________________________________________________________________
Part VII
Department of Education
_______________________________________________________________________
34 CFR Part 702
Standards for Conduct and Evaluation of Activities Carried Out by the
Office of Educational Research and Improvement (OERI)--Evaluation of
the Performance of Recipients of Grants, Cooperative Agreements, and
Contracts; Final Rule
Federal Register / Vol. 63, No. 207 / Tuesday, October 27, 1998 /
Rules and Regulations
[[Page 57570]]
DEPARTMENT OF EDUCATION
34 CFR Part 702
RIN 1850-AA54
Standards for Conduct and Evaluation of Activities Carried Out by
the Office of Educational Research and Improvement (OERI)--Evaluation
of the Performance of Recipients of Grants, Cooperative Agreements, and
Contracts
AGENCY: Office of Educational Research and Improvement, Department of
Education.
ACTION: Final regulations.
-----------------------------------------------------------------------
SUMMARY: The Assistant Secretary establishes regulations pursuant to
OERI's authorizing legislation, the Educational Research, Development,
Dissemination, and Improvement Act of 1994. The major purpose of these
standards is to ensure that the research, development, and
dissemination activities carried out by the recipients of grants from
and contracts and cooperative agreements with OERI meet the highest
standards of professional excellence.
EFFECTIVE DATE: These regulations take effect November 27, 1998.
FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt, U.S. Department of
Education, 555 New Jersey Avenue, NW, Room 508C, Washington, D.C.
Telephone: (202) 219-2126. Internet: (Sharon--Bobbitt@ed.gov).
Individuals who use a telecommunications device for the deaf (TDD) may
call the Federal Information Relay Service (FIRS) at 1-800-877-8339
between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.
Individuals with disabilities may obtain this document in an
alternate format (e.g., Braille, large print, audiotape, or computer
diskette) on request to the contact person listed in the preceding
paragraph.
SUPPLEMENTARY INFORMATION:
Background
On March 31, 1994, President Clinton signed Pub. L. 103-227, which
includes Title IX, the Educational Research, Development,
Dissemination, and Improvement Act of 1994 (the Act). The Act
restructured OERI and provided it with a broad mandate to conduct an
array of research, development, dissemination, and improvement
activities aimed at strengthening the education of all students.
Statutory Requirements
The Act directed the Assistant Secretary to develop, in
consultation with the National Educational Research Policy and
Priorities Board (the Board), such standards as may be necessary to
govern the conduct and evaluation of all research, development, and
dissemination activities carried out by OERI to ensure that these
activities meet the highest standards of professional excellence. The
Board is responsible for reviewing and approving the standards. The
legislation requires that the standards be developed in three phases.
In the first phase, standards were created and promulgated to
establish the peer review process and evaluation criteria to be used
for the review of applications for grants and cooperative agreements
and proposals for contracts. The final regulations setting out these
standards were published on September 14, 1995 (60 FR 47808). In the
second phase, standards were created and promulgated to establish the
criteria to be used in reviewing potentially exemplary and promising
educational programs. The final regulations setting out these standards
were published on November 17, 1997 (62 FR 61427).
In the third phase, which is the subject of these final
regulations, the Act requires that OERI develop standards for
evaluating and assessing the performance of all recipients of grants
from and cooperative agreements and contracts with OERI. This
evaluation must take place both during and at the conclusion of the
performance of the grant, cooperative agreement, or contract, and must
include the use of a system of peer review for the final assessment.
In developing the standards, the Assistant Secretary was required
to review the procedures utilized by the National Institutes of Health
(NIH), the National Science Foundation (NSF), and other Federal
departments or agencies engaged in research and development and to
solicit recommendations from research organizations and members of the
general public. OERI has reviewed the procedures used to evaluate the
performance of recipients of grants, contracts, or cooperative
agreements by several offices within NIH and NSF, the Office of Energy
Research in the Department of Energy, the Food and Drug Administration,
the National Institute of Standards and Technology, the National
Aeronautics and Space Administration, and the University Research
Initiative of the Department of Defense. Recommendations concerning
these standards have been obtained from the American Educational
Research Association, the Council for Educational Development and
Research, and the Organization of Research Centers.
Standards
The standards have been developed by the Assistant Secretary in
consultation with the Board. These standards cover all grants,
cooperative agreements, and contracts administered by OERI, ranging
from the smallest purchase orders and commissioned papers to the
largest research projects and research centers. The standards:
Require at least one interim assessment as well as a final
assessment of the performance of recipients of grants, cooperative
agreements, and contracts.
Establish procedures for selecting peer review panels to
conduct the assessments.
Establish procedures and criteria that the peer review
panels use in conducting the assessments.
Establish specific additional criteria that peer review
panels use in conducting the assessments for National Research and
Development Centers, Regional Educational Laboratories, Field-Initiated
Studies, and ERIC Clearinghouses.
In an effort to fulfill the law's intention of ensuring high-
quality research, development, and evaluation, OERI has developed
standards in which interim and final assessments may be supplemented by
a self-assessment by the recipient of a grant, cooperative, agreement,
or contract. The Board and the Assistant Secretary believe that the
collection and review of evidence on one's own performance is itself a
useful tool for improvement.
The Government Performance and Results Act requires the
establishment of performance indicators for Department activities.
Information collected pursuant to those indicators will be considered,
as appropriate, in the evaluation of individual recipients.
On February 24, 1998, the Assistant Secretary published a notice of
proposed rulemaking (NPRM) for these standards in the Federal Register
(63 FR 9393). These final regulations contain four major changes from
the NPRM. These changes are fully explained in the ``Analysis of
Comments and Changes'' elsewhere in this preamble. The major changes
pertain to clarification of the purpose of the regulation, how OERI
determines the number of interim assessments necessary, the role of
Department of Education staff in the assessments, and the use of
interim assessments as a source of information for the final
assessment.
Analysis of Comments and Changes
In response to the Secretary's invitation in the NPRM, four parties
[[Page 57571]]
submitted comments on the proposed regulations. In addition to the
public comment, comments from the Board's Subcommittee on Standards are
addressed as required by the legislation. The full Board approved the
final regulations at a meeting on September 18, 1998. An analysis of
the comments and of the changes in the regulations since publication of
the NPRM follows.
Major issues are grouped according to subject with appropriate
sections of the regulations referenced in parentheses. Technical and
other minor changes--and suggested changes the Secretary is not legally
authorized to make under the applicable statutory authority--are not
addressed.
Purpose (Sec. 702.1)
Comments: Three commenters suggested that the purpose of the
standards be clarified. One commenter suggested that the standards
themselves cannot ensure the highest standards of professional
excellence. Another commenter asked specifically whether the purpose
for conducting assessments was to make decisions about future funding
or to provide a system for monitoring and enhancing current and future
projects.
Discussion: The Secretary agrees that the purpose of the standards
should be clarified to go beyond their stated statutory purpose, which
is to ``ensure the highest standards of professional excellence,'' to
include the objectives of continuously improving the quality of funded
activities and of considering the results as one of the factors in
determining continuation funding for multi-year awards.
Changes: Section 702.1 has been modified to include a provision
that the purpose of the standards is to provide feedback to help
improve the quality of funded activities and to provide information for
consideration as continuation funding decisions are made.
Additional Activities that May be Evaluated (Sec. 702.3)
Comment: One commenter thought that the statement that these
standards could be applied to other activities funded by the Department
was too broad and should be deleted.
Discussion: The Secretary believes that this statement is
necessarily broad to allow all Department programs to use these
standards, when appropriate, to assess the performance of any of their
funded activities without developing their own unique regulations. This
statement is also consistent with the earlier standards which
established the peer review process and evaluation criteria to be used
for the review of applications for grants and cooperative agreements
and proposals for contracts.
Changes: None.
Number of Interim Assessments (Sec. 702.4)
Comments: Two commenters suggested changes to this provision. One
commenter suggested since there may be more than one interim
assessment, that it be clear in Sec. 702.4(d)(1). The OERI Board
suggested that the requirements for a single interim assessment for
total awards of $5,000,000 or less be modified to reflect total awards
of $3,000,000 or less.
Discussion: In response to the comments, the Secretary now believes
that considerations such as difficulty in achieving project objectives
rather than the dollar levels of awards should determine whether a
particular project merits more than one interim assessment. Elimination
of the dollar threshold clarifies the original intent of this section
which is to require that all awards receive one interim assessment.
More than one interim assessment will be performed only when a
recipient is having difficulty achieving project objectives as
determined by the initial interim assessment or through the monitoring
efforts of Department of Education staff. The Assistant Secretary will
make the determination of the number of interim assessments on a case-
by-case basis.
Changes: Section 702.4(b) has been modified to delete the dollar
threshold and to reflect that all awards will receive at least one
interim assessment. A new paragraph 702.4(c) has been added to clarify
that the Assistant Secretary will require more than one interim
assessment when a recipient has been identified, either in the initial
interim review or through monitoring efforts of Department of Education
staff, as having difficulty in achieving project objectives. Former
paragraph 702.4(c) has been redesignated as Sec. 702.4(d). Section
702.4(d)(1) has been modified to define an interim assessment as ``any
assessment'' conducted during a recipient's period of performance.
Definitions (Sec. 702.5)
Comment: One commenter suggested that the terms referred to in this
section include the specific definitions and not references to the OERI
statute and to the Education Department General Administrative
Regulations.
Discussion: The Secretary believes that providing the citations for
specific terms rather than the definitions themselves keeps regulations
short and concise while still cross referencing easily accessible
resources for the definitions.
Changes: None.
Characteristics of Peer Reviewers (Sec. 702.10)
Comment: One commenter suggested that paragraph 702.10(a) ``(4)
knowledge of a broad range of education policies and practices;'' be
deleted from the list of knowledge and expertise required of peer
reviewers, because it is redundant with the other criteria and is very
vague.
Discussion: The Secretary believes that this criterion provides for
a balance between specific program knowledge and a broader perspective
of education policies and practices and is therefore not redundant with
the other, more focused, characteristics required of peer reviewers.
Changes: None.
Role of Department Staff (Sec. 702.10)
Comments: Two commenters expressed concern over the appropriate
role of the OERI staff in the review process. One commenter urged the
Department to use all outside reviewers. The other commenter
acknowledged the knowledge and skills of the OERI staff but suggested
that staff not serve as peer reviewers within the primary division of
an agency in which they work and that each peer review panel be limited
to one Department staff person. This commenter suggested that the staff
focus on the important role of mentoring and designing competitions.
Discussion: The Secretary agrees that the primary role of the OERI
staff should be management of competitions including assessing the
results of peer reviews and monitoring awards. The Secretary believes
that the purpose of the peer review process should be to acquire the
perspective of outside experts independent of OERI. The Secretary also
believes that there may be exceptional circumstances where expertise
resides in OERI or in the Department, or where outside reviewers are
not required such as in the review of small purchase orders. The
exceptions should be determined by the Assistant Secretary.
Changes: Section 702.10(d) has been reworded to preclude OERI and
other Department staff from serving as peer reviewers except in
exceptional circumstances as determined by the Assistant Secretary.
Conflict of Interest (Sec. 702.11)
Comment: One commenter was concerned that while the conflict of
interest requirements were ``legally correct'' they failed to address
the problem occasioned by reviewers who
[[Page 57572]]
may have ideological or methodological view points that differ from
those of the recipient to be evaluated, or who are affiliated with
competing institutional organizations.
Discussion: The commenter appears to be concerned that the proposed
conflict of interest provision does not address the potential problem
of bias on the part of a panel against a particular grantee on
ideological or other grounds. The Secretary first believes that it is
essential to retain the present language, which parallels the provision
in the standards at 34 CFR 701.11(c), because it highlights the
important issue of improper financial gain or the appearance of
improper gain. However, the Secretary agrees that adding a requirement
to the effect that panels selected by the Assistant Secretary reflect a
broad range of perspectives could strengthen the regulation.
Changes: A new paragraph ``(c)'' has been added to Sec. 702.13
requiring the Assistant Secretary, to the greatest extent feasible, to
select peer reviewers for each evaluation who represent a broad range
of perspectives.
Sources of Information (Secs. 702.22 and 702.23)
Comment: One commenter suggested that the use of Government
Performance and Results Act (GPRA) information should be encouraged
rather than required for both interim and final assessments. The
commenter is concerned that information currently being collected under
GPRA to evaluate the effectiveness of a program or a system-level
activity will not provide information relevant to the assessment of
individual awards under that program or system-level activity and
therefore should not be required.
Discussion: The Secretary agrees that information obtained by GPRA-
related reports on the effectiveness of a program or system level
activity, e.g., how effectively a program is meeting the overall
objectives defined for it in its authorizing legislation, may not
necessarily include information related to an individual award being
reviewed under this regulation. However, the Secretary believes that
information on the effectiveness of the particular program under which
a recipient receives funding will help to provide a context for the
review of an individual award and must be considered by the panel.
Moreover, these regulations make it clear that the GPRA information is
only one of a number of sources used in conducting the review.
Changes: None.
Comment: One commenter suggested that the findings and information
from interim assessments would be an important source of information
for the final assessments and should be included under Sec. 702.23(a).
Discussion: The Secretary agrees that the results of interim
assessments should be a source of information for final assessments.
Change: Section 702.23(a) has been modified to add a new paragraph
(Sec. 702.23(a)(5)) to require that the results of interim assessments
be considered as a source of information for final assessments.
Evaluation Criteria (Sec. 702.24)
Comments: Two commenters suggested changes to this section. One
commenter suggested that there be a single menu of criteria for the
standards, because the proposed menu is too long. The second commenter
suggested that since Field Initiated Studies are not likely to provide
services, the word ``services'' be deleted from the criterion in
Sec. 702.24(e)(4)(ii): ``* * * addresses issues of national
significance through its products or services, or both.''
Discussion: The Secretary believes the current menu approach
provides a comprehensive strategy for assessing the performance of all
activities, ranging from the smallest purchase order to the largest
research investments. The categories in the regulation reflect the
specific authorities in the OERI statute. In addition, the menu
provides for other criteria for future research investments that do not
fit within the statutory authorities yet also must be assessed. A
single menu would, of necessity, be too generic to apply to the wide
range of activities covered by these standards. The Secretary agrees
that assessing ``services'' is not appropriate for Field Initiated
Studies projects.
Change: Section 702.24(e)(4)(ii) has been modified to delete the
word, ``services.''
Paperwork Reduction Act of 1995
Under the Paperwork Reduction Act of 1995, no persons are required
to respond to a collection of information unless it displays a valid
OMB control number. The valid OMB control number assigned to the
collection of information in these final regulations is displayed at
the end of the affected sections of the regulations.
Assessment of Educational Impact
In the NPRM the Secretary requested comments on whether the
proposed regulations would require transmission of information that is
being gathered by or is available from any other agency or authority of
the United States.
Based on the response to the NPRM and on its own review, the
Department has determined that the regulations in this document do not
require transmission of information that is being gathered by or is
available from any other agency or authority of the United States.
Electronic Access to This Document
Anyone may view this document, as well as all other Department of
Education documents published in the Federal Register, in text or
portable document format (pdf) on the World Wide Web at either of the
following sites:
http://ocfo.ed.gov/fedreg.htm
http://www.ed.gov/news.html
To use the pdf you must have the Adobe Acrobat Reader Program with
Search, which is available free at either of the previous sites. If you
have questions about using the pdf, call the U.S. Government Printing
Office at (202) 512-1530 or, toll free, at 1-888-293-6498.
Anyone may also view these documents in text copy only on an
electronic bulletin board of the Department. Telephone: (202) 219-1511
or, toll free, 1-800-222-4922. The documents are located under Option
G--Files/Announcements, Bulletins and Press Releases.
Note: The official version of this document is the document
published in the Federal Register.
List of Subjects in 34 CFR Part 702
Education, Educational research, Reporting and recordkeeping
requirements.
(Catalog of Federal Domestic Assistance Number does not apply.)
Dated: October 22, 1998.
C. Kent McGuire,
Assistant Secretary for Educational Research and Improvement.
The Secretary amends Chapter VII of Title 34 of the Code of Federal
Regulations by adding a new Part 702 to read as follows:
[[Page 57573]]
PART 702--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES
CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT
(OERI)--EVALUATION OF THE PERFORMANCE OF RECIPIENTS OF GRANTS,
COOPERATIVE AGREEMENTS, AND CONTRACTS
Subpart A--General
Sec.
702.1 What is the purpose of these standards?
702.2 What activities must be evaluated by these standards?
702.3 What additional activities may be evaluated by these
standards?
702.4 When is performance assessed under these standards?
702.5 What definitions apply?
Subpart B--Selection of Peer Review Panels
702.10 What are the characteristics of peer reviewers?
702.11 What constitutes a conflict of interest for grants and
cooperative agreements?
702.12 What constitutes a conflict of interest for contracts?
702.13 How are peer reviewers selected for panels?
Subpart C--The Evaluation Process
702.21 How does a peer review panel evaluate the performance of a
recipient?
702.22 What information does a peer review panel consider for an
interim assessment?
702.23 What information does a peer review panel consider for a
final assessment?
702.24 What evaluation criteria must be used for performance
assessments?
Authority: 20 U.S.C. 6011(i), unless otherwise noted.
Subpart A--General
Sec. 702.1 What is the purpose of these standards?
(a) The standards in this part implement section 912(i) of the
Educational Research, Development, Dissemination, and Improvement Act
of 1994 (the Act).
(b) These standards establish criteria and a peer review process to
provide recipients of OERI grants, cooperative agreements and contract
awards with assessments of their projects.
(1) The purpose of the assessments is to provide feedback to
recipients to improve the quality of funded activities and to provide
information to OERI as it determines if a recipient of a multi-year
award merits continuation funding.
(2) The criteria and peer review process are intended to address
the statutory requirement that the research, development, and
dissemination activities carried out by the recipients of grants from
and contracts and cooperative agreements with the Office of Educational
Research and Improvement (OERI) meet the highest standards of
professional excellence.
(Authority: 20 U.S.C. 6011(i)(2)(F))
Sec. 702.2 What activities must be evaluated by these standards?
These standards apply to activities carried out by OERI using funds
appropriated under section 912(m) of the Act including activities
carried out by the following entities or programs:
(a) The National Education Research Institutes.
(b) The Office of Reform Assistance and Dissemination.
(c) The Educational Resources Information Center.
(d) The Regional Educational Laboratories.
(e) The Teacher Research Dissemination Demonstration Program.
(f) The Goals 2000 Community Partnerships Program.
(g) The National Educational Research Policy and Priorities Board.
(Authority: 20 U.S.C. 6011(i)(1))
Sec. 702.3 What additional activities may be evaluated by these
standards?
The Secretary may apply these standards to other activities funded
by the Department, as appropriate.
(Authority: 20 U.S. C 6011 (i)(1))
Sec. 702.4 When is performance assessed under these standards?
(a) The Secretary will assess the performance of recipients of OERI
grants, contracts, and cooperative agreements subject to these
standards during and at the conclusion of their period of performance.
(b) The Department requires at least one interim assessment by a
peer review panel for all awards.
(c) The Assistant Secretary will approve and require more than one
interim assessment when an award is identified, either by the initial
interim review or by Department of Education staff monitoring the
award, as having difficulty in achieving project objectives.
(d) A final assessment by a peer review panel is required for all
awards.
(e) As used in this part--
(1) Interim assessment is any assessment conducted during a
recipient's period of performance.
(2) Final assessment is one conducted at the conclusion of a
recipient's period of performance.
(Authority: 20 U.S.C. 6011(i)(2)(F))
Sec. 702.5 What definitions apply?
(a) Definitions in the Educational Research, Development,
Dissemination, and Improvement Act of 1994. The following terms used in
this part are defined in 20 U.S.C. 6011(l)(1):
Development
Dissemination
Educational Research
(b) Definitions in the Education Department General Administrative
Regulations. The following terms used in this part are defined in 34
CFR 77.1:
Application
Award
Department
Grant
Project
Secretary
(c) Definitions in the Federal Acquisition Regulation. The
following term used in this part is defined in 48 CFR Chapter 1:
Contract Proposal.
(Authority: 20 U.S.C. 6011(i)(2)(F)
Subpart B--Selection of Peer Review Panels
Sec. 702.10 What are the characteristics of peer reviewers?
(a) The Assistant Secretary selects each peer reviewer. Each peer
reviewer must have the necessary knowledge and expertise in the area of
the project being reviewed to evaluate the performance of a recipient.
This experience may include--
(1) Expert knowledge of subject matter in the area of the
activities to be reviewed;
(2) Expert knowledge of theory or methods or both in the area of
the activities to be reviewed;
(3) Practical experience in the area of the activities or type of
institution or both to be reviewed;
(4) Knowledge of a broad range of education policies and practices;
(5) Experience in managing complex organizations; or
(6) Expertise and experience in evaluation theory and practice.
(b) Each peer reviewer must be free of conflict of interest, as
determined in accordance with Sec. 702.11 or Sec. 702.12.
(c) The Assistant Secretary may solicit nominations for peer
reviewers from professional associations, nationally recognized
experts, and other sources.
(d) OERI and other Department staff who possess the qualifications
in paragraphs (a) and (b) of this section may serve as peer reviewers
only in exceptional circumstances as determined by the Assistant
Secretary.
(Authority: 20 U.S.C. 6011(i)(2)(B))
Sec. 702.11 What constitutes a conflict of interest for grants and
cooperative agreements?
A peer reviewer assessing the performance of the recipient of a
grant
[[Page 57574]]
from or cooperative agreement with OERI is considered an employee of
the Department for the purposes of conflict of interest analysis. As an
employee of the Department, the peer reviewer is subject to the
provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's
policies used to implement those provisions.
(Authority: 20 U.S.C. 6011(i)(2)(B))
Sec. 702.12 What constitutes a conflict of interest for contracts?
A peer reviewer assessing the performance of the recipient of a
contract with OERI is considered an employee of the Department in
accordance with the Federal Acquisition Regulation (FAR), 48 CFR 3.104-
4(h)(2). As an employee of the Department, the peer reviewer is subject
to the provisions of the FAR, 48 CFR Part 3, Improper Business
Practices and Personal Conflict of Interest.
(Authority: 41 U.S.C. 423)
Sec. 702.13 How are peer reviewers selected for panels?
(a) The Assistant Secretary assigns peer reviewers to panels that
conduct the performance assessments.
(b) The Assistant Secretary may establish panels by category of
recipient, such as a panel to review the performance of all Regional
Educational Laboratories. Each recipient is evaluated individually by
reviewers who have been assigned to this type of panel.
(c) In establishing panels, the Assistant Secretary, to the
greatest extent feasible, selects peer reviewers for each evaluation
who represent a broad range of perspectives.
(Authority: 20 U.S.C. 6011(i)(2)(B))
Subpart C--The Evaluation Process
Sec. 702.21 How does a peer review panel evaluate the performance of a
recipient?
(a) In each evaluation, a peer review panel--
(1) Considers relevant information about the recipient's
performance, as described in Secs. 702.22 and 702.23; and
(2) Makes judgments about the recipient's performance, using the
criteria in Sec. 702.24.
(b) Each peer reviewer prepares a report based on the reviewer's
assessment of the quality of the project according to the evaluation
criteria.
(c) After each peer reviewer has evaluated each project
independently, the panel may be convened to discuss the strengths and
weaknesses of the project. Each reviewer may then independently re-
evaluate each project with appropriate changes made to the written
report.
(d) The report of the interim assessment must include any
recommendations the peer reviewer may have for improving the
recipient's performance.
(e) The report of the final assessment must contain each peer
reviewer's evaluative summary of the recipient's performance, from the
beginning of the contract, grant, or cooperative agreement to its
conclusion.
(Authority: 20 U.S.C. 6011(i)(2)(F))
Sec. 702.22 What information does a peer review panel consider for an
interim assessment?
(a) Sources of information for the interim assessment must
include--
(1) The original request for proposals or grant announcement and
the contract proposal or grant application;
(2) Documentation of any changes in the work described in the
contract, grant, or cooperative agreement, including reasons for the
changes;
(3) Any progress reports delivered to the Department or made
available to the public by the recipient;
(4) Examples of products delivered to the Department or made
available to the public by the recipient;
(5) Any relevant reports written by OERI staff, including reports
of site visits by OERI staff;
(6) Any performance evaluations conducted under the FAR or the
Education Department General Administrative Regulations (34 CFR Part
75).
(7) Any relevant information provided by the recipient in response
to Government Performance and Results Act (GPRA) (Pub. L. 103-62)
requirements; and
(8) Any reports from program evaluations commissioned by the
Department.
(b) Sources of information for the interim assessment may also
include--
(1) A self-assessment, prepared by the recipient, addressing the
criteria in Sec. 702.24;
(2) One or more site visits by the peer review panel;
(3) One or more oral or written presentations to the panel by the
recipient describing its performance; or
(4) Other information about the recipient's performance.
(Approved by the Office of Management and Budget under control
number 1850-0746)
(Authority: 20 U.S.C. 6011(i)(2)(F))
Sec. 702.23 What information does a peer review panel consider for a
final assessment?
(a) Sources of information for the final assessment must include--
(1) The original request for proposals or application notice and
the contract proposal or grant application, together with documentation
of any changes in the work described in the proposal or application,
including reasons for the changes;
(2) If consistent with the recipient's contract, grant, or
cooperative agreement with OERI, a written report or oral presentation
or both by the recipient summarizing its activities and
accomplishments;
(3) Any relevant information provided by the recipient in response
to Government Performance and Results Act (GPRA) (Pub. L. 103-62)
requirements;
(4) Any reports from program evaluations commissioned by the
Department; and,
(5) Any relevant information provided by the interim assessment.
(b) The final assessment may also include other sources of
information, such as one or more of those listed in Sec. 702.22.
(Approved by the Office of Management and Budget under control
number 1850-0746)
(Authority: 20 U.S.C. 6011(i)(2)(F))
Sec. 702.24 What evaluation criteria must be used for performance
assessments?
(a) Peer reviewers (and those recipients who conduct self-
evaluations) shall use the criteria in paragraph (b) of this section to
assess performance and, in case of interim assessments, to identify
areas in which the performance of recipients may need improvement.
(b) The following evaluation criteria are to guide the assessment
process undertaken by peer reviewers. The peer reviewers determine the
extent to which recipients meet these criteria:
(1) Implementation and management. (i) Peer reviewers shall
consider the degree to which the recipient has fully executed its
program of work. In doing so, peer reviewers shall consider evidence on
the extent to which the recipient completes the work described in the
approved application or contract, including any approved modifications,
in the time period proposed and in an efficient manner.
(ii) In examining the degree of implementation, peer reviewers may
also consider evidence on the extent to which--
(A) The recipient implements and utilizes a quality assurance
system for its products or services or both; and
(B) The recipient conducts self-assessment or self-evaluation
activities, including periodically seeking out independent critiques
and evaluations of its work, and uses the results to improve
performance.
(2) Quality. (i) Peer reviewers shall consider the degree to which
the
[[Page 57575]]
recipient's work approaches or attains professional excellence. In
determining quality, peer reviewers shall consider evidence on the
extent to which--
(A) The recipient utilizes processes, methods, and techniques
appropriate to achieve the goals and objectives for the program of work
in the approved application; and
(B) The recipient applies appropriate processes, methods, and
techniques in a manner consistent with the highest standards of the
profession.
(ii) In determining quality, peer reviewers may also consider the
extent to which the recipient conducts a coherent, sustained program of
work informed by relevant research.
(3) Utility. (i) In determining the utility of the recipient's
products or services or both, peer reviewers shall consider evidence on
the extent to which the recipient's work (including information,
materials, processes, techniques, or activities) is effectively used by
and is useful to its customers in appropriate settings.
(ii) In determining utility, peer reviewers may also consider the
extent to which the recipient has received national recognition; e.g.,
articles in refereed journals and presentations at professional
conferences.
(4) Outcomes and impact. (i) Peer reviewers shall consider the
results of the recipient's work. In examining outcomes and impact, peer
reviewers shall consider evidence on the extent to which--
(A) The recipient meets the needs of its customers; and
(B) The recipient's work contributes to the increased knowledge or
understanding of educational problems, issues, or effective strategies.
(ii) In examining outcomes and impact, peer reviewers may also
consider the extent to which recipients address issues of national
significance through its products or services or both.
(c) For National Research and Development Centers, peer reviewers
also shall consider evidence on the extent to which recipients meet the
following criteria:
(1) Quality. (i) The recipient uses a well-conceptualized framework
and sound theoretical and methodological tools in conducting
professionally rigorous studies; and
(ii) The recipient conducts work of sufficient size, scope, and
duration to produce sound guidance for improvement efforts and future
research.
(2) Utility. The recipient documents, reports, and disseminates its
work in ways to facilitate the effective use of its work in
appropriately targeted settings.
(3) Outcomes and impact. (i) The recipient's work contributes to
the development and advancement of theory in the field of study,
including its priority area; and
(ii) The recipient addresses issues of national significance
through its products or services or both.
(d) For the Regional Educational Laboratories, peer reviewers also
shall consider evidence on the extent to which recipients meet the
following criteria:
(1) Quality. (i) The recipient utilizes a well-conceptualized
framework and sound theoretical and methodological tools in conducting
professionally rigorous studies;
(ii) The recipient conducts work of sufficient size, scope, and
duration to produce sound guidance for improvement efforts; and
(iii) The recipient's products are well tested and based on sound
research.
(2) Utility. The recipient documents, reports, and disseminates its
work in ways to facilitate its effective use in appropriately targeted
settings, particularly in school improvement efforts of States and
localities.
(3) Outcomes and impact. (i) The recipient assists States and
localities to implement comprehensive school improvement strategies
through the provision of research-based information (including well-
tested models and strategies), materials and assistance; and
(ii) The recipient's work results in widespread access to
information regarding research and best practices, particularly within
its region.
(e) For Field-Initiated Studies, peer reviewers also shall consider
evidence on the extent to which recipients meet the following criteria:
(1) Implementation and management. The recipient's work responds to
the goals, objectives and mission of the National Institute from which
it is funded.
(2) Quality. The recipient utilizes a well-conceptualized framework
and sound theoretical and methodological tools in conducting
professionally rigorous studies.
(3) Utility. The recipient documents, reports, and disseminates its
work in ways to facilitate its effective use in appropriately targeted
settings.
(4) Outcomes and impact. (i) The recipient's work contributes to
the development and advancement of theory and knowledge in the field of
study; and
(ii) The recipient addresses issues of national significance
through its products.
(f) For the ERIC Clearinghouses, peer reviewers also shall consider
evidence on the extent to which recipients meet the following criteria:
(1) Quality. The recipient applies an integrated approach to
acquiring and disseminating significant and high-quality educational
literature and materials to maintain and enhance the ERIC database.
(2) Utility. The recipient contributes to the development of the
ERIC database as a source of literature and materials that reflects
trends and issues within its scope.
(3) Outcomes and impact. (i) The recipient meets the informational
and educational needs of its customers through dissemination and
outreach approaches and the development of an array of print and non-
print materials; and
(ii) The recipient provides national leadership on the use of
current computer, networking, and information technology.
(Approved by the Office of Management and Budget under control
number 1850-0746)
(Authority: 20 U.S.C. 6011(i)(2)(F))
[FR Doc. 98-28729 Filed 10-26-98; 8:45 am]
BILLING CODE 4000-01-U
