2015-12824. Proposed Priority-Technical Assistance Center for Vocational Rehabilitation Agency Program Evaluation and Quality Assurance

[CFDA Number: 84.263B.]

AGENCY:

Office of Special Education and Rehabilitative Services, Department of Education.

ACTION:

Proposed priority.

SUMMARY:

The Assistant Secretary for Special Education and Rehabilitative Services proposes a priority under the Experimental and Innovative Training program. The Assistant Secretary may use this priority for competitions in fiscal year (FY) 2015 and later years. We take this action to focus Federal financial assistance on an identified national need. We intend the priority to support a Training and Technical Assistance Center for Vocational Rehabilitation Agency Program Evaluation and Quality Assurance (PEQA).

DATES:

We must receive your comments on or before June 29, 2015.

ADDRESSES:

Submit your comments through the Federal eRulemaking Portal or via postal mail, commercial delivery, or hand delivery. We will not accept comments submitted by fax or by email or those submitted after the comment period. To ensure that we do not receive duplicate copies, please submit your comments only once. In addition, please include the Docket ID at the top of your comments.

• Federal eRulemaking Portal: Go to www.regulations.gov to submit your comments electronically. Information on using Regulations.gov, including instructions for accessing agency documents, submitting comments, and viewing the docket, is available on the site under “Are you new to the site?”
• Postal Mail, Commercial Delivery, or Hand Delivery: If you mail or deliver your comments about these proposed regulations, address them to Don Bunuan, U.S. Department of Education, 400 Maryland Avenue SW., Room 5046, Potomac Center Plaza (PCP), Washington, DC 20202-2800.

Privacy Note: The Department's policy is to make all comments received from members of the public available for public viewing in their entirety on the Federal eRulemaking Portal at www.regulations.gov. Therefore, commenters should be careful to include in their comments only information that they wish to make publicly available.

FOR FURTHER INFORMATION CONTACT:

Don Bunuan. Telephone: [phone] or by email: [e-mail].

If you use a telecommunications device for the deaf (TDD) or a text telephone (TTY), call the Federal Relay Service (FRS), toll free, at 1-800-877-8339.

SUPPLEMENTARY INFORMATION:

Invitation to Comment: We invite you to submit comments regarding this priority. To ensure that your comments have maximum effect in developing the final priority, we urge you to identify clearly the specific section of the proposed priority that each comment addresses.

We invite you to assist us in complying with the specific requirements of Executive Orders 12866 and 13563 and their overall requirement of reducing regulatory burden that might result from this proposed priority. Please let us know of any further ways we could reduce potential costs or increase potential benefits while preserving the effective and efficient administration of the program.

During and after the comment period, you may inspect all public comments about this notice by accessing regulations.gov. You may also inspect the comments in person in room 5040, 550 12th Street SW., PCP, Washington, DC 20202-2800, between the hours of 8:30 a.m. and 4:00 p.m., Washington, DC time, Monday through Friday of each week except Federal holidays.

Assistance to Individuals with Disabilities in Reviewing the Rulemaking Record: On request we will provide an appropriate accommodation or auxiliary aid to an individual with a disability who needs assistance to review the comments or other documents in the public rulemaking record for this notice. If you want to schedule an appointment for this type of accommodation or auxiliary aid, please contact the person listed under FOR FURTHER INFORMATION CONTACT.

Purpose of Program: This program is designed to (a) develop new types of training programs for rehabilitation personnel and to demonstrate the effectiveness of these new types of training programs for rehabilitation personnel in providing rehabilitation services to individuals with disabilities; and (b) develop new and improved methods of training rehabilitation personnel so that there may be a more effective delivery of rehabilitation services by State and other rehabilitation agencies.

Program Authority: 29 U.S.C. 772(a)(1).

Applicable Program Regulations: 34 CFR parts 385 and 387.

Proposed Priority:

This notice contains one proposed priority.

Training and Technical Assistance Center for Vocational Rehabilitation Agency Program Evaluation and Quality Assurance (PEQA).

Background:

Federal agencies are increasingly being called upon to implement accountability systems designed to assess and improve the effectiveness and efficiency of the programs they administer. Legislation such as the Government Performance and Results Act of 1993 (GPRA) and the GPRA Modernization Act of 2010 have provided a performance management framework that holds Federal agencies accountable for achieving program results.

The recently enacted Workforce Innovation and Opportunity Act (WIOA) made major changes to improve accountability for performance of the core programs of the Federal workforce system, including the State Vocational Rehabilitation (VR) Services program. In particular, WIOA amendments to section 106 of the Rehabilitation Act eliminate the VR program's evaluation standards and indicators and make the program subject to the common performance accountability measures, established in section 116(b) of WIOA, that are applicable to all core programs of the workforce development system.

In addition to required evaluation activities under the Rehabilitation Act, section 116(e)(1) of WIOA requires States, in coordination with local boards and the State agencies responsible for the administration of the core programs, to conduct ongoing evaluations of activities carried out under such programs “in order to promote, establish, implement, and utilize methods for continuously improving core program activities in order to achieve high-level performance within, and high-level outcomes from, the workforce development system.”

To carry out the WIOA performance accountability and evaluation requirements, State VR agencies will need to build their capacity to develop and evaluate methods to achieve high-level performance and program outcomes, including the effective and efficient use of program resources. In particular, State VR agencies will need personnel with the knowledge and skills to improve agency performance management systems through rigorous program evaluation and the implementation of quality assurance systems.

In anticipation of the increased focus on improving performance management, in 2011, the 36th Institute on Rehabilitation Issues (IRI) study group recommended that (1) the Rehabilitation Services Administration (RSA) work with the rehabilitation field to improve performance management systems and tools, and (2) State agencies embrace continuous improvement practices to properly inform public policy development and measurement of effectiveness (IRI, 2011). The 36th IRI described how bolstering program evaluation and quality assurance within State agencies could improve the quality of service delivery and better achieve successful employment for VR consumers. For example, trained evaluators could provide agencies with valuable data and analysis to use in planning and forecasting, to tailor training to meet the needs of staff, to evaluate staff performance, to respond to policy initiatives, and to monitor overall performance of the agency. As a result, State VR agencies will be more accountable, efficient, and successful.

The demand for program evaluation and quality assurance skill development is also evidenced by the growing number of grassroots communities of practice. These communities of practice, which usually consist of VR agency staff, have identified that one of the greatest needs of State VR agencies is structured program evaluation training specifically tailored for existing staff.

For State VR agencies, a workforce with skills focused on performance evaluation and quality assurance is essential. There is a demonstrated interest and need in the field for additional, structured training opportunities for new and existing State VR agency staff, and RSA believes a training and technical assistance center would be ideally suited to meet this need.

Reference:

Institute on Rehabilitation Issues (2011). Performance management: Program evaluation and quality assurance in vocational rehabilitation. Hot Springs, AR: University of Arkansas CURRENTS.

Proposed Priority:

The purpose of this proposed priority is to fund a cooperative agreement for a training and technical assistance center that will assist State vocational rehabilitation (VR) agencies to improve performance management by building their capacity to carry out high quality program evaluations and quality assurance practices that promote continuous program improvement.

The Training and Technical Assistance Center for Program Evaluation and Quality Assurance (PEQA) will assist State VR agencies in building this capacity through professional education and training of vocational rehabilitation evaluators. To this end, PEQA will:

(a) Provide educational opportunities for State VR staff from recognized experts in program evaluation and quality assurance;

(b) Develop interagency collaboration networks and work teams committed to the improvement of quality assurance systems and tools; and

(c) Deliver technical, professional, and continuing educational support to State VR program evaluators.

Project Activities:

To meet the requirements of this priority, the PEQA must, at a minimum, conduct the following activities:

Basic Certification Program:

(a) Develop a one-year certificate program in VR program evaluation that will result in increasing the numbers and qualifications of program evaluators in State VR agencies. At a minimum, this certificate program must:

(1) Be designed to develop key competencies necessary for successful implementation of program evaluation and quality assurance activities, including, but not limited to:

(i) Knowledge of the State-Federal VR program;

(ii) Data collection methodologies;

(iii) Data analysis and interpretation;

(iv) Making evaluative judgments and recommendations;

(v) Effective communication of results (including presentations, drafting reports, and building partnerships); and

(vi) Ethical practice.

(2) Be responsive to the prior knowledge and skills of participants;

(3) Incorporate adult learning principles and opportunities for practice into training;

(4) Be delivered through multiple modalities and in an accessible format;

(5) Assess, at regular intervals, the progress of training participants toward attainment of the key competencies; and

(6) Require the completion of a capstone project in order to successfully complete the program. The capstone project must:

(i) Be completed within one year of the completion of formal coursework for the certificate program;

(ii) Be conducted on a topic responsive to the needs of the State VR agency and agreed to by the PEQA, the participant, and the State VR agency; and

(iii) Be completed as part of the normal work duties of the participant in the State VR agency.

(7) Be provided at no cost to participants, excluding travel and per diem costs, which may be provided by the sponsoring agency.

(b) Provide training through the certificate program to a cohort of eight to ten working professionals in each year of the project.

(c) Select participants for the certificate program based, in part, on the considered recommendation of their employing State VR agencies.

Special Topical Training:

(a) Develop a series of special training opportunities for intermediate-level program evaluators. These training opportunities must, at a minimum:

(1) Be designed to develop higher-level knowledge, skills, and abilities of program participants;

(2) Be focused on a range of topics determined by the PEQA with input from State VR agencies and other relevant groups or organizations;

(3) Provide opportunities for hands-on application of the competencies discussed in the trainings;

(4) Be of sufficient duration and intensity to ensure that participants obtain the competencies discussed in the trainings; and

(5) Assess the progress of program participants in attaining the competencies discussed in the trainings.

Note:

For purposes of this priority, an “intermediate-level program evaluator” is a program evaluator working for a State VR agency with the knowledge, skills, and abilities typically expected of a professional who has been in such a position for at least five years.

(b) Conduct no fewer than four special training opportunities each year of the project.

Coordination Activities:

(a) Establish a community of practice that will act as a vehicle for communication, exchange of information among program evaluation professionals, and a forum for sharing the results of capstone projects that are in progress or have been completed. This community of practice must be focused on challenges facing project evaluation professionals and the development of key competencies to address such challenges;

(b) Maintain a Web site that, at a minimum:

(1) Provides a central location for later reference and use of capstone projects, resources from special training opportunities, and other relevant materials; and

(2) Ensures peer-to-peer access between State VR project evaluation professionals.

(c) Communicate and coordinate, on an ongoing basis, with other relevant Department-funded projects and those supported by the Departments of Labor, Commerce, and Health and Human Services; and

(d) Maintain ongoing communication with the RSA project officer and other RSA staff as required.

Application Requirements:

To be funded under this priority, applicants must meet the application and administrative requirements in this priority. RSA encourages innovative approaches to meet these requirements, which are:

(a) Demonstrate, in the narrative section of the application under “Significance of the Project,” how the proposed project will—

(1) Address State VR agencies' capacity to conduct high quality program evaluation and data analysis activities. To address this requirement, the applicant must:

(i) Demonstrate knowledge of emerging and best practices in program evaluation and quality assurance;

(ii) Demonstrate knowledge of current State VR and other efforts designed to improve evaluation and performance management practices.

(2) Increase the number of program evaluators working in State VR agencies who have obtained a certificate in their field of work and the number and quality of program evaluation activities performed by State VR agencies.

(b) Demonstrate, in the narrative section of the application under “Quality of Project Services,” how the proposed project will—

(1) Achieve its goals, objectives, and intended outcomes. To meet this requirement, the applicant must provide—

(i) Measurable intended project outcomes;

(ii) A plan for how the proposed project will achieve its intended outcomes; and

(iii) A plan for communicating and coordinating with relevant training programs and communities of practice, State VR agencies, and other RSA partners.

(2) Use a conceptual framework to develop project plans and activities, describing any underlying concepts, assumptions, expectations, beliefs, or theories, as well as the presumed relationships or linkages among these variables, and any empirical support for this framework.

(3) Be based on current research and make use of evidence-based practices. To meet this requirement, the applicant must describe:

(i) How the current research about adult learning principles and implementation science will inform the proposed training; and

(ii) How the proposed project will incorporate current research and evidence-based practices in the development and delivery of its products and services.

(4) Develop products and provide services that are of high quality and sufficient intensity and duration to achieve the intended outcomes of the proposed project. To address this requirement, the applicant must describe—

(i) Its proposed curriculum for a certificate program for VR evaluation professionals;

(ii) Its proposed plan for recruiting and selecting trainees for the certification program;

(iii) Its proposed plan for collecting information on the impact of capstone projects;

(iv) Its proposed plan for identifying, selecting and addressing the special topical program evaluation and quality assurance related training needs of State VR agency staff;

(v) Its proposed plan for annual follow-up with participants in special training opportunities;

(5) Develop products and implement services to maximize the project's efficiency. To address this requirement, the applicant must describe—

(i) How the proposed project will use technology to achieve the intended project outcomes; and

(ii) With whom the proposed project will collaborate and the intended outcomes of this collaboration.

(c) Demonstrate, in the narrative section of the application under “Quality of the Evaluation Plan,” how the proposed project will—

(1) Measure and track the effectiveness of the training provided. To meet this requirement, the applicant must describe its proposed approach to—

(i) Collecting data on the effectiveness of training activities;

(ii) Analyzing and reporting data on the effectiveness of training, including any proposed standards or targets for determining effectiveness;

(2) Collect and analyze data on specific and measurable goals, objectives, and intended outcomes of the project, including measuring and tracking the effectiveness of the training provided. To address this requirement, the applicant must describe—

(i) Its proposed evaluation methodologies, including instruments, data collection methods, and analyses;

(ii) Its proposed standards or targets for determining effectiveness;

(iii) How it will use the evaluation results to examine the effectiveness of its implementation and its progress toward achieving the intended outcomes; and

(iv) How the methods of evaluation will produce quantitative and qualitative data that demonstrate whether the project and individual training activities achieved their intended outcomes.

(d) Demonstrate, in the narrative section of the application under “Adequacy of Project Resources,” how—

(1) The proposed project will encourage applications for employment from persons who are members of groups that have traditionally been underrepresented based on race, color, national origin, gender, age, or disability, as appropriate;

(2) The proposed key project personnel, consultants, and subcontractors have the qualifications and experience to achieve the project's intended outcomes;

(3) The applicant and any key partners have adequate resources to carry out the proposed activities; and

(4) The proposed costs are reasonable in relation to the anticipated results and benefits.

(e) Demonstrate, in the narrative section of the application under “Quality of the Management Plan,” how—

(1) The proposed management plan will ensure that the project's intended outcomes will be achieved on time and within budget. To address this requirement, the applicant must describe—

(i) Clearly defined responsibilities for key project personnel, consultants, and subcontractors, as applicable; and

(ii) Timelines and milestones for accomplishing the project tasks.

(2) Key project personnel and any consultants and subcontractors will be allocated to the project and how these allocations are appropriate and adequate to achieve the project's intended outcomes, including an assurance that such personnel will have adequate availability to ensure timely communications with stakeholders and RSA;

(3) The proposed management plan will ensure that the products and services provided are of high quality; and

(4) The proposed project will benefit from a diversity of perspectives, including those of State and local personnel, technical assistance providers, researchers, and policy makers, among others, in its development and operation.

Types of Priorities:

When inviting applications for a competition using one or more priorities, we designate the type of each priority as absolute, competitive preference, or invitational through a notice in the Federal Register. The effect of each type of priority follows:

Absolute priority: Under an absolute priority, we consider only applications that meet the priority (34 CFR 75.105(c)(3)).

Competitive preference priority: Under a competitive preference priority, we give competitive preference to an application by (1) awarding additional points, depending on the extent to which the application meets the priority (34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets the priority over an application of comparable merit that does not meet the priority (34 CFR 75.105(c)(2)(ii)).

Invitational priority: Under an invitational priority, we are particularly interested in applications that meet the priority. However, we do not give an application that meets the priority a preference over other applications (34 CFR 75.105(c)(1)).

Final Priority:

We will announce the final priority in a notice in the Federal Register. We will determine the final priority after considering responses to this notice and other information available to the Department. This notice does not preclude us from proposing additional priorities, requirements, definitions, or selection criteria, subject to meeting applicable rulemaking requirements.

Note:

This notice does not solicit applications. In any year in which we choose to use this priority, we invite applications through a notice in the Federal Register.

Executive Orders 12866 and 13563:

Regulatory Impact Analysis:

Under Executive Order 12866, the Secretary must determine whether this regulatory action is “significant” and, therefore, subject to the requirements of the Executive order and subject to review by the Office of Management and Budget (OMB). Section 3(f) of Executive Order 12866 defines a “significant regulatory action” as an action likely to result in a rule that may—

(1) Have an annual effect on the economy of $100 million or more, or adversely affect a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local, or tribal governments or communities in a material way (also referred to as an “economically significant” rule);

(2) Create serious inconsistency or otherwise interfere with an action taken or planned by another agency;

(3) Materially alter the budgetary impacts of entitlement grants, user fees, or loan programs or the rights and obligations of recipients thereof; or

(4) Raise novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles stated in the Executive order.

This proposed regulatory action is not a significant regulatory action subject to review by OMB under section 3(f) of Executive Order 12866.

We have also reviewed this proposed regulatory action under Executive Order 13563, which supplements and explicitly reaffirms the principles, structures, and definitions governing regulatory review established in Executive Order 12866. To the extent permitted by law, Executive Order 13563 requires that an agency—

(1) Propose or adopt regulations only on a reasoned determination that their benefits justify their costs (recognizing that some benefits and costs are difficult to quantify);

(2) Tailor its regulations to impose the least burden on society, consistent with obtaining regulatory objectives and taking into account—among other things and to the extent practicable—the costs of cumulative regulations;

(3) In choosing among alternative regulatory approaches, select those approaches that would maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity);

(4) To the extent feasible, specify performance objectives, rather than the behavior or manner of compliance a regulated entity must adopt; and

(5) Identify and assess available alternatives to direct regulation, including economic incentives—such as user fees or marketable permits—to encourage the desired behavior, or provide information that enables the public to make choices.

Executive Order 13563 also requires an agency “to use the best available techniques to quantify anticipated present and future benefits and costs as accurately as possible.” The Office of Information and Regulatory Affairs of OMB has emphasized that these techniques may include “identifying changing future compliance costs that might result from technological innovation or anticipated behavioral changes.”

We are issuing this proposed priority only on a reasoned determination that their benefits would justify its costs. In choosing among alternative regulatory approaches, we selected those approaches that maximize net benefits. Based on the analysis that follows, the Department believes that this regulatory action is consistent with the principles in Executive Order 13563.

We also have determined that this regulatory action would not unduly interfere with State, local, and tribal governments in the exercise of their governmental functions.

In accordance with both Executive orders, the Department has assessed the potential costs and benefits, both quantitative and qualitative, of this regulatory action. The potential costs are those resulting from statutory requirements and those we have determined as necessary for administering the Department's programs and activities.

The benefits of a grant under the Rehabilitation Training program have been established over the years through the successful completion of similar training projects funded for the purpose of improving the skills of State VR agency staff. The proposed priority would specifically improve the skills of State VR agency evaluators. A project of this type will be particularly beneficial to State VR agencies in this era of increased emphasis on accountability and program results.

Intergovernmental Review: This program is subject to Executive Order 12372 and the regulations in 34 CFR part 79. One of the objectives of the Executive order is to foster an intergovernmental partnership and a strengthened federalism. The Executive order relies on processes developed by State and local governments for coordination and review of proposed Federal financial assistance.

This document provides early notification of our specific plans and actions for this program.

Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., braille, large print, audiotape, or compact disc) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT.

Electronic Access to This Document: The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available via the Federal Digital System at: www.gpo.gov/​fdsys. At this site you can view this document, as well as all other documents of this Department published in the Federal Register, in text or Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at the site.

You may also access documents of the Department published in the Federal Register by using the article search feature at: www.federalregister.gov. Specifically, through the advanced search feature at this site, you can limit your search to documents published by the Department.

Dated: May 21, 2015.

Sue Swenson,

Acting Assistant Secretary for Special Education and Rehabilitative Services.