01-13666. Agency Information Activities; Proposed Measures; Comments Request; Performance Measures

AGENCY:

Veterans' Employment and Training Service, Labor.

ACTION:

Notice and request for comments.

SUMMARY:

By law, the Assistant Secretary for Veterans' Employment and Training (ASVET) is required to establish performance standards for the provision of services to veterans by State Employment Security Agencies (SESA's). The ASVET in turn, is required to report on these results in the Veterans' Employment and Training Service's (VETS) Annual Report to Congress. This notice requests comments and opinions on the applicability of a proposed means to measure the performance outcomes of the Public Labor Exchange, which provides services to veterans within local workforce investment areas. In response to General Accounting Office recommendations, VETS is moving from process performance measures to outcome measures, and to evaluation of results based on absolute terms rather than the current relative terms. The new measures will provide useful information on the effectiveness of services provided to veterans. These proposed performance measures will also be key components of VETS' five-year Strategic Plan and Annual Performance Plans for the management of services provided to veterans and disabled veterans through the Public Labor Exchange, which includes services provided by Local Veterans' Employment Representative (LVER) and Disabled Veterans' Outreach Program (DVOP) staff.

In addition to Public Labor Exchange employees assigned to the Local Employment Service Offices (LESO) or One-Stop Centers established in the States, DVOP and LVER positions are provided to the States to enhance, not supplant, services provided to veterans by Wagner-Peyser staff.

DATES:

To be assured of consideration, comments on these proposed performance measures must be received on or before July 30, 2001.

ADDRESSES:

Comments should be sent to Robert Wilson, Chief, Division of Employment and Training Programs, Veterans' Employment and Training Service, U.S. Department of Labor, 200 Constitution Avenue NW, Room S-1316, Washington, DC 20210, Facsimile: 202-693-4755, E-mail: [e-mail]. Receipt of submissions, whether by U.S. mail, e-mail or FAX transmittal, will not be acknowledged; however, the sender may request confirmation that a submission has been received by telephoning VETS at [phone] (VOICE) or [phone] (TTY/TDD).

FOR FURTHER INFORMATION:

Contact Robert Wilson, 202-693-4719, or E-mail: [e-mail]. (Please note the above telephone numbers are not toll free.)

SUPPLEMENTARY INFORMATION:

I. Authority

Components of the VETS' performance measures are proposed under the authority of Title 38 United States Code (U.S.C.) section 4107(b), which requires that performance standards be established to assure that SESA's are providing maximum employment and training opportunities to eligible veterans and are in compliance with Chapters 41 and 42 of Title 38, U.S.C. The proposed performance measures are separate from the reporting requirements of Section 4107 (c) of Title 38, U.S.C., and they do not replace these reporting requirements. Section 4212 of Title 38, U.S.C. requires entities awarded Federal contracts or subcontracts of $25,000 or more to take affirmative action to employ and advance in employment qualified Special Disabled Veterans, Vietnam-era Veterans, Recently Separated Veterans (Pub. L. 106-419 added Recently Separated Veterans to the class of veterans receiving emphasis under Federal Contracts), and Campaign Veterans (any other veterans who served on active duty during a war or in a campaign or expedition for which a campaign badge or expeditionary medal has been authorized). Federal contractors and subcontractors are required by law and regulation to list job openings with their local SESA office. The SESA's, through the Public Labor Exchange system, are required to provide priority referrals of qualified targeted veterans to these Federal contractor openings.

II. VETS Performance Measures

The proposed set of four (4) measures are consistent with the VETS Strategic Plan and Annual Plan, the Workforce Investment Act (WIA) of 1998, Title 38 U.S.C. and the requirements of the Government Performance and Results Act (GPRA) of 1993. Two (2) of the proposed measures are similar to the performance measures for the Public Labor Exchange, as proposed in the August 14, 2000 Federal Register Notice, 65 Fed. Reg. 49707 (August 14, 2000). VETS will make every effort to maintain consistency with the Labor Exchange Performance Measures to be implemented by the Employment and Training Administration (ETA).

A. Outcome and Process Performance Measures

VETS is proposing the following annual Program Year (PY) based veterans' performance measures:

• Veteran Job Seeker Entered Employment Rate (VJSEER)
• Veteran Job Seeker Employment Retention Rate at Six Months (VJSERR)
• Veterans' Employment Rate Following Receipt of Staff Assisted Services (VERS)
• Federal Contractor Job Openings Listed with the Public Labor Exchange (FCJL)

1. Identification and Selection of Outcome and Process Performance Measures

In 1996, the ASVET directed VETS staff to work with State partners to provide input regarding the development, testing, and evaluation of new performance systems to replace the current performance measures system, which had been developed more than a decade before with little or no change since then. There is concern that these performance standards are activity- and volume-driven, and that the current performance measures do not provide the most useful information on the impact of services on veterans served. As a result, from September 1998 through December 1999, VETS conducted a study to review current performance measurements for the Public Labor Exchange and DVOP/LVER programs. Based on this research, VETS developed two types of proposed performance measures. The first category was comprised of outcome measures that can be applied at the National, State, and local areas, and that can be adjusted for labor market and other characteristics within the State and local area. These measures, which were intended to measure outcomes rather than processes were: Entered Employment Rate, Employment Rate, Employment Retention at Six Months, and Earnings Gain. The second category addressed processes and these measures were: Employment Rate Following Receipt of Staff Assisted Services by Wagner-Peyser Staff, and Entered Employment Following Referral to a Federal Contractor.

During July through September 2000, VETS conducted a test of the proposed performance measures in six (6) States to determine if data could be collected using Unemployment Insurance (UI) wage records. VETS obtained contractor support to provide technical assistance for the beta test by conducting site visits and facilitating the matching of registration data to wage record data, which was provided by the participating State agencies. Interviews were conducted with WIA, State UI, State Employment Service (ES), and State Information Systems staff to obtain information on data collection and any cost problems associated with using UI wage records, difficulties in calculating the measures or using the data sets, and opinions concerning the proposed measures. Contractor representatives did not participate in the process of making final recommendations.

All six (6) of the test States calculated Public Labor Exchange performance based on the following three (3) outcome measures: Entered Employment Rate, Earnings Gain, and Employment Retention Rate at Six Months. For these three (3) outcome measures, the six (6) States matched registrant data for veterans to State UI wage record data for PY's 1997 and 1998. Four (4) States calculated a fourth outcome measure, Employment Rate, also utilizing the registrant data for veterans, and State UI wage record data for PY's 1997 and 1998. Five (5) of the States also calculated the process measure, Employment Rate Following Receipt of Staff Assisted Services by Wagner-Peyser Staff.

Due to differences in the way States processed data on Earnings Gain, VETS determined that the measure would not provide valid outcome measurements, and thus, decided not to use Earnings Gain as a performance measure. Furthermore, beta test results indicated that sufficient data for the Entered Employment Rate Following Referral to a Federal Contractor were not available. Therefore, VETS decided that this process measure would be changed to the percent of increase in the number of Federal contractor job openings listed with the Public Labor Exchange. VETS has developed tools to improve the ability of the Public Labor Exchange system to identify job openings listed by Federal contractors. This will, in turn, increase the number of Federal contractor job listings reported by the Public Labor Exchange, as well as provide greater employment opportunities for targeted veterans.

After an in-depth analysis of the results of the beta test and comments from participating States, and in coordination with ETA, VETS is proposing to move forward with four (4) of the proposed outcome and process measures that have been shown to be most useful in measuring performance of the Public Labor Exchange system in the delivery of services to veterans.

2. Proposed VETS Performance Measures

VETS requests comments on the applicability of the four (4) proposed performance measures for the services to veterans implemented through the Public Labor Exchange system. Each of the proposed measures will be calculated and reported quarterly, with an annual PY summary. Listed below are the operational definitions for the proposed performance measures.

a. Veteran Job Seeker Entered Employment Rate (VJSEER). The proposed Veteran Job Seeker Entered Employment Rate is defined as: Of Wagner-Peyser Act funded labor exchange applicants who are veterans and who in the first or second quarter following registration (Q₊₁ or Q₊₂), earned wages from a new or different employer than that from which the applicant earned wages in the quarter prior to registration (Q₋₁), divided by the number of applicants registered during the measurement period. Those applicants who earned wages in the first or second quarter following registration (Q₊₁ or Q₂), solely with the same employer from which wages were earned in the quarter prior to registration (Q₋₁), are excluded from the measure.

b. Veteran Job Seeker Employment Retention Rate at Six Months (VJSERR). The proposed Veteran Job Seeker Employment Retention Rate at Six Months is defined as: Of those Wagner-Peyser Act labor exchange applicants age 19 and older at the time of registration who are veterans, and who in the first or second quarter following registration (Q₊₁ or Q₊₂), earned wages from a new or different employer than that from which the applicant earned wages in the quarter prior to registration (Q₋₁); those who also continue to earn wages in the third or fourth quarter (Q₊₃ or Q₊₄) respectively, following registration, divided by the number who earned wages in the first or second quarter after registration (Q₊₁) or (Q₊₂). The examples of situations that result in a positive outcome are: One (1), a veteran earning wages in quarter Q₊₁ from a new or different employer than in quarter Q₋₁, must at a minimum also earn wages in Q₊₃; Two (2), a veteran earning wages in Q₊₂ from a new or different employer than in Q₋₁, must at a minimum also earn wages in quarter Q₊₄.

c. Veterans' Employment Rate Following Receipt of Staff Assisted Services (VERS). The proposed Veterans' Employment Rate Following Receipt of Staff Assisted Services is defined as: Of the Wagner-Peyser Act applicants who are veterans, who registered in a quarter, Q₀, and who received some form of staff assisted labor exchange services from Public Labor Exchange staff, the number who are employed by the end of the first or second quarter after registration, (Q₊₁ or Q₊₂).

The following chart presents the three preceding measures of service.

VETS Proposed Measures of Service For All Veterans and Disabled Veterans

Category	Measures of service 1
	VJSEER	VJSERR	VERS
All Veterans	X	X	X
Disabled Veteran	X	X	X
1 All measures use two quarters of employment (wage record) followup data.

d. Federal Contractor Job Openings Listed with the Public Labor Exchange (FCJL). Federal Contractor Job Opening Listed with the Public Labor Exchange is defined as: The percentage increase in the number of Federal contractor job openings listed annually with the Public Labor Exchange, relative to the number listed in the previous PY.

Measurement and reporting of these four (4) measures are consistent with the requirements under Title 38 U.S.C., GPRA, the WIA, and Public Labor Exchange performance measures. Section 4107(c)(1) of Title 38 requires that the actual number of veterans who registered with the Public Labor Exchange and who entered employment, be reported by VETS to Congress annually, on a State by State basis. The two (2) outcome measures, Veteran Job Seeker Entered Employment Rate, and Veteran Job Seeker Employment Retention Rate at Six Months; and one (1) process measure, Veteran's Employment Rate Following Receipt of Staff Assisted Services, can be adjusted by State for the characteristics of the population, and for the condition of the labor market within the State. This adjustment provides for equitable performance measurement across States. There is no adjustment for the Federal Contractor Job Openings Listed with the Public Labor Exchange process measure.

To the extent feasible and appropriate, VETS plans to parallel its proposed measures for Veteran Job Seeker Entered Employment Rate, and Veteran Job Seeker Employment Retention Rate at Six Months, with the equivalent measures proposed by the ES. Currently, the calculations for these measures are based on a methodology comparable to that set forth in the August 14, 2000 Federal Register Notice, in which ETA proposed the Labor Exchange Performance Measurement System. This notice and request for comments incorporates the most recent revisions to the proposed ETA Labor Exchange Performance Measurement System. If there are any further revisions, VETS intends to modify its proposed Veteran Job Seeker Entered Employment Rate, and Veteran Job Seeker Employment Retention Rate measures to maximize consistency with the Public Labor Exchange Performance Measurement System.

B. Procedures for Establishing Expected Levels of Performance

VETS proposes to emulate the WIA, section 136, Performance Accountability System, and utilize their negotiation process to establish expected performance levels for Labor Exchange services. The use of this framework means that States, in conjunction with their VETS' Director of Veterans' Employment and Training (DVET), will develop baseline data for each of the measures based on historical data, analyze the baseline data, and propose performance levels for each measure based on that analysis. Each State will negotiate with its DVET to obtain mutually agreed upon expected levels of performance. In developing baseline data, States should use two years of data if possible, but not less than one year in determining trends for performance and factors which may influence performance. In establishing expected performance levels for each measure, factors beyond the control of the State, such as economic conditions, should be considered.

C. Baseline Performance

Baselines for each of the measures will be developed by each State and will be a key factor used to determine the expected level of performance that is negotiated with VETS. Baselines are intended to give an indication of the past outcomes of a performance measure. For performance negotiations to be data-driven and reality-based, the development of baselines is a critical aspect of the negotiation process.

D. State Expected Levels of Performance

When submitting their proposed performance levels, States should be prepared to provide support for their proposed levels by providing baseline performance data, the methodology for developing baseline data, and a description of data sources. In addition, States should also include information on appropriate factors used to project expected levels of performance. Examples of possible factors to consider in negotiating expected levels of performance are: Economic conditions such as the unemployment rate; the rate of job creation/loss, new business start-ups; community factors such as availability of transportation and daycare; pursuit of new or enhanced employer partnerships; natural disasters; State legislation; and other factors which might impact performance. This list is not intended to be prescriptive or exhaustive, but rather to suggest the kind of information that might be considered in the negotiation process.

E. Negotiation of Expected Levels of Performance

The Regional Administrator for Veterans' Employment and Training (RAVET) will review the negotiated levels of performance as submitted through the DVET and will compare the expected performance levels with the National averages, baseline information from other States, and the negotiated levels of performance established for other States, taking into account factors including differences in economic conditions and other factors as discussed above. The RAVET will analyze the quality of the data presented by States, including the relevance of the data, the source of the data, the time period from which the data were drawn, and if the data are part of a trend or anomalous. Established GPRA Annual Performance Plan goals for relevant measures will also be an important part of the Regional review and negotiation of performance levels. When the RAVET's analysis is completed, if need be, there will be the opportunity, through the DVET, for negotiations with the State to obtain mutually agreed upon expected levels of performance. Provision will also be made for renegotiation of performance levels if circumstances arise that result in a significant change in the factors used to establish the original levels. It is understood that either a State or VETS may elect to renegotiate performance as new information becomes available. Factors which will be considered for making changes include those discussed above, in paragraph #D.

F. Rules for Application

During the first year of implementation of the new performance measures, performance will be reviewed as described below; however, States will be held harmless from any consequences of failing to meet their performance goals during that first year. Actual performance for each program year will be compared to negotiated performance levels. Incentives may be put in place for States with exemplary performance results. For a State to be designated as “exemplary,” expected levels of performance for all measures must be achieved or exceeded. A possible incentive will be that “exemplary” States would proportionally split a fixed amount of the grant allocation or any unspent funds. If a State's actual performance varies from the expected performance level by minus two percent or more, VETS will have the option of renegotiating new performance levels with the State. VETS will offer technical assistance as well as giving consideration to external factors affecting performance levels. A negative variation of five percent or more would result in the requirement of a State Corrective Action Plan (CAP) to rectify the situation. Failure to submit or comply with a CAP may become the basis for sanctions.

Signed at Washington, DC, this 25th day of May, 2001.

Stanley A. Seidel,

Director, Operations and Programs.