02-9918. Service to Veterans; Final Performance Measures for State Employment Security Agencies

AGENCY:

Veterans' Employment and Training Service, Labor.

ACTION:

Final Notice.

SUMMARY:

By law, the Assistant Secretary for Veterans' Employment and Training (ASVET) is required to establish performance standards for the provision of services to veterans by State Employment Security Agencies (SESA's). The ASVET, in turn, is required to report on these results in the Veterans' Employment and Training Service's (VETS) Annual Report to Congress. This document communicates the establishment of final performance measures for the provision of services to veterans by the public labor exchange, including those services provided by Local Veterans' Employment Representative (LVER) and Disabled Veterans' Outreach Program (DVOP) staff. VETS’ performance measurement system for the public labor exchange consists of three measures: (1) Veteran Job Seeker Entered Employment Rate, (2) Veteran Job Seeker Employment Retention Rate at Six Months, and (3) Veteran Job Seeker Entered Employment Rate Following Receipt of Staff-Assisted Services. These measures are to be calculated for two categories of veterans: (1) Veterans and Eligible Persons and (2) Disabled Veterans. VETS establishes these measures in light of comments received in response to proposed performance measures for services to veterans by the public labor exchange, as published in the Federal Register on May 31, 2001.

EFFECTIVE DATE:

These performance measures for the public labor exchange service to veterans will become effective July 1, 2002.

ADDRESS:

All comments received during the comment period following the publication of the proposed labor exchange measures for services to veterans (66 FR 29602 et seq., May 31, 2001) are available for public inspection and copying during normal business hours at the Veterans' Employment and Training Service, U.S. Department of Labor, 200 Constitution Avenue NW., Room S-1316, Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT:

Robert Wilson, 202-693-4719 (voice), or [phone] (TTY/TDD for the hearing impaired), or E-mail: [e-mail].

SUPPLEMENTARY INFORMATION:

I. Authority

VETS establishes performance measures for the provision of services by the public labor exchange under the following authority:

A. Title 38, United States Code (U.S.C.), Chapter 41—Job Counseling, Training, and Placement Services for Veterans, Sec. 4107(a)(1)

The Secretary shall establish administrative controls for the following purposes: To insure that each eligible veteran, especially veterans of the Vietnam era and disabled veterans and each eligible person, who requests assistance under this chapter (38 USCS 4100 et seq.) shall promptly be placed in a satisfactory job, or job training opportunity or receive some other specific form of assistance designed to enhance such veteran's and eligible person's employment prospects substantially, such as individual job development or employment counseling services.

B. Title 38, U.S.C., Chapter 41—Job Counseling, Training, and Placement Services for Veterans, Sec. 4107 (b)

The Secretary shall establish definitive performance standards for determining compliance by the State public employment service agencies with the provisions of this chapter (38 USCS 4100 et seq.) and chapter 42 of this title (38 USCS 4211 et seq.).

These final performance measures are separate from the reporting requirements of section 4107 (c) of Title 38, U.S.C., and they do not negate these reporting requirements. Section 4212 of Title 38, U.S.C. requires entities awarded Federal contracts or subcontracts of $25,000 or more to take affirmative action to employ and advance in employment qualified Special Disabled Veterans, Vietnam-era Veterans, Recently Separated Veterans, (Pub. L. 106-419 added Recently Separated Veterans to the class of veterans receiving emphasis under Federal Contracts) and Campaign Veterans (any other veterans who served on active duty during a war or in a campaign or expedition for which a campaign badge or expeditionary medal has been authorized). Federal contractors and subcontractors are required by law and regulation to list virtually all job openings with their local SESA office. The SESA's, through the public labor exchange system, are required to provide priority referrals of qualified targeted veterans to these Federal contractor openings.

II. VETS Performance Measures

A. Background

VETS developed its performance measurement system based upon a series of tests of proposed measures and meetings with various stakeholders. Beginning in September 1998, and continuing through the Summer of 1999, VETS reviewed current performance measurements for the public labor exchange and DVOP and LVER programs. Following the finalization of the performance measures for the Workforce Investment Act (WIA) in 1999, and the publication of recommendations of the United States Employment Service's (USES) workgroup on performance measures for the public labor exchange system (65 FR 49708 et seq., Aug. 14, 2000), VETS developed five proposed measures that were consistent with these related workforce development programs. These measures were: Entered Employment Rate, Employment Retention Rate at Six Months, Earnings Gain, Employment Rate Following Receipt of Staff-Assisted Services by Wagner-Peyser Staff, and Entered Employment Rate Following Referral to a Federal Contractor.

During July through September 2000, VETS conducted a beta test of its proposed performance measures in six (6) States. Based on the results of this beta test, the final recommendations of the USES workgroup on performance measures for the public labor exchange, and through coordination with the USES on the Revised Employment and Training Administration (ETA) 9002 Reports and Revised Employment and Training (ET) Handbook 406 (ETA 9002 Data Preparation Handbook), VETS published a set of four proposed performance measures in the Federal Register (66 FR 29602 et seq., May 31, 2001). These measures were Veteran Job Seeker Entered Employment Rate, Veteran Job Seeker Employment Retention Rate at Six Months, Veterans' Employment Rate Following Receipt of Staff-Assisted Services, and Federal Contractor Job Openings Listed with the Public Labor Exchange. The first three of these measure were proposed to apply to all Veterans and to Disabled Veterans. The Federal Register Notice also provided a general framework for establishing expected levels of performance for each of these measures.

During the Summer of 2001, VETS reviewed and analyzed the comments received in response to the publication of the proposed performance measures. In addition, VETS considered reviews and comments from various sources on its proposed measures as well as other related measures, and guidelines. VETS considered the findings and recommendations in a report published in May 2001, by the General Accounting Office on the VETS proposed performance measurement system. VETS then considered commentary on the proposed measures received from SESA representatives who attended a series of four Regional Planning and Management Workshops that VETS hosted across the country earlier in 2001. VETS also reviewed the final performance measures for the public labor exchange established by the ETA on July 1, 2001, and the comments received by USES for its publication of the draft ET Handbook 406 and VETS 200 Reports. Based on these considerations, VETS developed its final performance measures of services to veterans by the public labor exchange.

The final measures are consistent with Title 38, U.S.C., the requirements of the Government Performance and Results Act (GPRA) of 1993, VETS” Strategic Plan and Annual Plan, and the performance measurement systems established by the WIA and the USES.

B. Response to Comments

In response to the four proposed VETS performance measures published in the May 31, 2001, Federal Register, VETS received 31 sets of comments distributed as follows: representatives from state agencies (17), Disabled Veterans' Outreach Program Specialists and Local Veterans' Employment Representatives (9), the National Association of State Workforce Agencies, USES, an ETA Regional Office (3), and current and former employees of VETS (2). The comments are discussed at length as follows:

(1) Veteran Job Seeker Entered Employment Rate

VETS initially proposed the Veteran Job Seeker Entered Employment Rate defined as:

Of Wagner-Peyser Act funded labor exchange applicants who are veterans and who in the first or second quarter following registration (Q₊₁ or Q₊₂), earned wages from a new or different employer than that from which the applicant earned wages in the quarter prior to registration (Q₋₁), divided by the number of applicants registered during the measurement period. Those applicants who earned wages in the first or second quarter following registration (Q₊₁ or Q₊₂), solely from the same employer from which wages were earned in the quarter prior to registration (Q₋₁), are excluded from the measure.

Comment: Five respondents supported the measure as proposed. Eight respondents stated concern that the two quarter period to follow-up veteran registrants' employment outcomes after the registration dates will not be sufficient for those with multiple barriers to employment or who are placed in training. Six respondents suggested that veterans who are enrolled in case management should be excluded from the denominator for the Veteran Job Seeker Entered Employment Rate because they are likely to require extensive services over a period of time that extends beyond two outcome follow-up quarters (Q₊₁ and Q₊₂). Similarly, four respondents suggested that veterans who are enrolled in long-term training should be excluded from the denominator for the Veteran Job Seeker Entered Employment Rate because their employment outcomes are not likely to appear in the first or second quarter after registration.

Response: To maintain consistency with the final measures for the public labor exchange, as published by the USES in the May 31, 2001, Federal Register, VETS will retain the proposed definition where outcomes are determined in the two quarters following the quarter of registration. To ensure consistency with the performance measures for the public labor exchange, VETS will use the registration policies defined in the ET Handbook 406. VETS and USES have jointly discussed the issue of State systems not counting registrants while they are in long-term training or are receiving intensive services such as case management. In order to develop a broad, overall measure of the effectiveness of participation in the public labor exchange system, VETS and USES agree that states are not to exclude from the applicant counts any individuals engaged in long term training or case management. VETS believes the measure of Veteran Job Seeker Entered Employment Rate will help capture the success of the public labor exchange efforts to assist veteran job seekers in achieving the desired outcome of entering employment.

(2) Veteran Job Seeker Employment Retention Rate at Six Months

VETS initially proposed the Veteran Job Seeker Employment Retention Rate at Six Months defined as:

Of those Wagner-Peyser Act labor exchange applicants age 19 and older at the time of registration who are veterans, and who in the first or second quarter following registration (Q₊₁ or Q₊₂), earned wages from a new or different employer than that from which the applicant earned wages in the quarter prior to registration (Q₋₁); those who also continue to earn wages in the third or fourth quarter (Q₊₃ or Q₊₄) respectively, following registration, divided by the number who earned wages in the first or second quarter after registration (Q₊₁ or Q₊₂).

Comment: Six respondents supported the measure as proposed. Four respondents asked for further clarification about why this measure includes the applicants aged 19 years old.

Response: To maintain consistency with the public labor exchange and the similar measure for Title 1 of WIA, the VETS employment retention measure will follow the definition and specifications set forth for the public labor exchange employment retention measure. The exclusion of registrants under the age of 19 at the time of registration will remove from the equation a large portion of those registered job seekers who most likely are seeking only short term employment.

Comment: Five respondents stated that the measure is beyond the control of the public labor exchange because several factors may influence employment retention. Some of the possibilities include seasonal nature of some occupations and changes in the economy of a particular area.

Response: VETS acknowledges that the employment retention measure is blind to labor market conditions. In conjunction with the USES, VETS will be developing methods to adjust for economic conditions and characteristics of registered veteran job seekers to use in establishing performance goals and for interpreting final performance levels compared to the rates established in the baseline year.

Comment: One respondent noted that the measure, as originally proposed, does not measure the same group of veterans as the Veteran Job Seeker Entered Employment Rate measure.

Response: We agree that the Veteran Job Seeker Employment Retention Rate at Six Months measures outcomes for a different pool of applicants than those measured in the Entered Employment Rate. The Employment Retention Rate measures outcomes for job seekers who are age 19 or older at the time of registration while the Entered Employment Rate measures outcomes for applicants of all ages. The intent of the proposed Employment Retention Rate measure is to determine employment retention outcomes for those most likely to be seeking long-term employment, excluding youth, who are most likely seeking short term employment.

Comment: Another respondent noted that the standard does not measure movement from part-time to full-time work. Some respondents also noted that initial placements are often temporary and seasonal.

Response: VETS supports the measure of Veteran Job Seeker Employment Retention Rate at Six Months as a measure that is consistent with those of other workforce development programs. The intent of the measure also supports VETS’ vision of minimizing underemployment, maximizing career employment opportunities, and improving labor market status of veteran job seekers. With respect to the comment about temporary and seasonal work, it is important to note that this measure assesses retention in employment, not job retention with a specific employer. To be counted in this measure, the veteran job seeker must retain some form of employment in the two quarters following initial entry to employment, not necessarily continued employment with the same employer.

Comment: A respondent questioned the effect that the emphasis on self-service and automated systems will have on this standard.

Response: The measure is not intended to specifically assess effectiveness of the use of self-service tools by individuals. Individuals using only self-service tools will not be counted unless a State has defined its registration policy to include these individuals in the registered applicant pool. For States that do register these individuals, the measure will help capture the quality of staff-assisted services, facilitated self-help services, and self-services that provide job seekers with resources to secure and maintain continued employment.

Comment: Another respondent also suggested that the measure will be flawed if it fails to take into account customer satisfaction.

Response: The measure of Veteran Job Seeker Employment Retention Rate at Six Months does not specifically include a customer satisfaction component as that type of information is intended to be collected separately. The public labor exchange performance measures for customer satisfaction will serve as a broad indicator of how well the public labor exchange is serving all job seekers including veterans.

(3) Veterans' Employment Rate Following Receipt of Staff-Assisted Services

VETS initially proposed the Veterans' Job Seeker Employment Rate Following Receipt of Staff-Assisted Services defined as:

Of the Wagner-Peyser Act applicants who are veterans, who registered in a quarter, (Q₀), and who received some form of staff-assisted services from public labor exchange staff, the number who are employed by the end of the first or second quarter after registration, (Q₊₁ or Q₊₂).

Comment: Five respondents supported a performance measure for Veterans' Job Seeker Employment Rate Following Staff-Assisted Services. Several respondents discussed the definition of staff-assisted services, the ETA policies for States to define registration, and the potential impacts of these two issues on this measure. Four respondents suggested that the staff-assisted services should be more specifically defined. Two respondents noted that it is possible that outcomes following this measure are likely to be similar to those of the Veteran Job Seeker Entered Employment Rate because States may choose to implement a registration policy where only those job seekers who receive staff-assisted services are actually registered. Two respondents suggested that states should have greater flexibility to define staff-assisted services. One respondent recommended that referrals to a job not be considered a staff-assisted service because this activity is not staff-intensive. One respondent recommended not implementing this measure unless VETS provides additional funding for programming. Finally, two other respondents suggested that the quality of staff-assisted services can be measured through the Veteran Job Seeker Employment Retention Rate After Six Months measure.

Response: The comments about this measure pertain to two issues: registration policies and definition of staff-assisted services. The new registration policy for the public labor exchange requires that anyone receiving staff-assisted services be registered. Under the WIA, States do have the option to register those using only self-service tools. For States that exercise the option to not register self-service customers, VETS acknowledges that the Veterans' Employment Rate Following Receipt of Staff-Assisted Services measure may parallel the entered employment measure. To maintain consistency, and to ensure that any veteran in any State who received services from the public labor exchange will be counted by the VETS performance measures, VETS will maintain the Veterans' Employment Rate Following Receipt of Staff-Assisted Services measure (this measure will be renamed as Veteran Job Seeker Entered Employment Rate Following Receipt of Staff-Assisted Services).

The elements of the staff-assisted services were developed in consultation with the USES workgroup of State representatives, officials from the National Association of State Workforce Agencies, and staff from VETS and ETA National and Regional offices. The workgroup revisited the definition and specifications for staff-assisted services to ensure that uniformity can be achieved among States. Therefore, the elements of staff-assisted services have been defined in ways that all States can reasonably apply the definition of staff-assisted services to their specific workforce development programs and systems.

VETS believes that this measure will provide effective data regarding the outcomes of more intensive public labor exchange services to veterans. For example, a veteran job seeker receiving staff-assisted services may require a multitude of services any one of which, or combination thereof, may require extensive staff time. Thus, measures of entered employment outcomes after receipt of staff-assisted services will provide an indication of the quality of those services. VETS encourages States to collect and analyze more detailed information on staff-assisted services, as each State deems necessary for its management purposes.

Comment: Two respondents suggested that VETS consider “weighting” services so that those Services that are more staff-intensive (such as case management) are given additional weight, compared to those that are less intensive.

Response: VETS is exploring the development of a weighted measurement system that would further encourage the provision of intensive services to veterans by the public labor exchange system.

Comment: Two respondents suggested that case management by DVOP or LVER staff specifically should be emphasized in this measure.

Response: VETS is focusing its guidance on DVOP services to address the needs of those veteran job seekers, particularly disabled veterans, who could benefit from case management and other employment development activities by DVOP staff. VETS is currently developing specific performance measures as well as prototype performance standards that would apply to DVOP and LVER staff activities.

Comment: Another respondent suggested that restricting measurement for this measure to only two quarters may not fully capture the entered employment outcome, and therefore three follow-up quarters would be preferable.

Response: VETS believes that consistency between this measure and other entered employment measures for the public labor exchange will help provide broad, overall measures of the effectiveness of the public labor exchange system in serving veterans. Therefore, VETS and ETA agree that this measure should use data from the two quarters following registration as the period for determining entered employment outcomes.

(4) Federal Contractor Job Openings Listed With the Public Labor Exchange

VETS initially proposed the Federal Contractor Job Openings Listed with the Public Labor Exchange be defined as:

The percentage increase in the number of Federal contractor job openings listed annually with the public labor exchange, relative to the number listed in the previous Program Year (PY).

Comment: VETS received 27 comments on the proposed measure for Federal Contractor Job Openings Listed (FCJL) With the Public Labor Exchange. Listing of jobs by Federal contractors with the public labor exchange is not under the control of labor exchange staff. Four respondents noted that listings of contractors and subcontractors available to staff are not complete. Seven respondents noted that enforcement of the FCJL program is the responsibility of the Office of Federal Contract Compliance Programs. Nine respondents noted that contractors can meet their Federal contractor award requirements by listing job openings with America's Job Bank, not the public labor exchange. New Federal contracts in any given State do not necessarily imply new job openings. Firms require new contracts to maintain their current workforce. The quantity of new Federal contractor job listings is determined by the amount and location of Federal contracting. No State is guaranteed a quarter to quarter increase in the number of Federal contracts that will be awarded in its jurisdiction.

Response: VETS has chosen not to use the proposed FCJL measure as a performance standard for the public labor exchange. Since VETS is mandated by section 4212 of Title 38, U.S.C. to report to Congress on Federal contractor listing and hiring activity, States will still be required to submit data on the number of Federal contractor job listings received, the number of Federal contractors listing jobs, the number of veterans referred to FCJL jobs, and on the number of veterans placed in FCJL jobs.

(5) General Comments

Comment: Additionally, VETS received a number of comments related to performance measures in general. Several respondents raised concerns about registration policies. Four respondents requested clarification of the definitions of (1) registration, (2) renewal of registration, and (3) reporting periods, as they relate to the rolling four quarter concept. One respondent expressed concern that under the self-service concept, the number of veteran job seekers will drop significantly. Another respondent suggested that in some States, the registration policy will create situations where the only veteran job seekers who will be registered are those who receive intensive services offered principally by staff funded under WIA grants, without regard to the services provided by staff funded under Wagner-Peyser or DVOP or LVER grants. As a result of these registration policies, the labor exchange services to veterans will be evaluated not on the general veteran population served by the public labor exchange, but by those veterans receiving intensives services. Similarly, two respondents expressed concern that the new measures will not measure the success or failure of the self-service and automated systems, unless all veterans who use self-service are registered. Two respondents voiced concerns that without a National, uniform policy for registration, data will not be available for State-to-State comparisons that are valid or reliable. However, another respondent encouraged VETS to allow States to establish their own registration policies for self-service. Another respondent asked for clarification that, as the public labor exchange measures are modified, VETS measures will also be modified to maintain consistency.

Response: States have the option of registering self-service customers. As a result, VETS, in conjunction with the USES, has decided not to implement a policy that will require registration or establish mandatory performance measures for users of self-service tools. To maintain consistency with the performance measures for the public labor exchange, VETS will use the registration policies defined in the ET Handbook 406. A veteran job seeker customer is counted as registered during the quarter in which registration occurs (registration quarter) and the subsequent three quarters. Registration is the date of registration or re-registration, and the registration year is the quarter of registration plus the following three quarters. A veteran job seeker who engages in a labor exchange activity after a registration year expires will begin a new registration year, and will once again be eligible to be counted in the measures.

For purposes of reporting, the rolling four quarter period concept will be used. As specified in the ET Handbook 406, the ETA 9002A (quarterly services report for all applicants), and the ETA 9002B (quarterly services report for veteran applicants) will provide data on persons who either registered or received services within the four quarter reporting period. The 9002C (quarterly outcome report for all applicants), and 9002D (quarterly outcome report for veteran applicants) will report the outcomes available on services provided to all registered job seekers, and to veteran job seekers for the four quarter reporting period. The rolling four quarter reporting period eliminates the concept of a carry-over registration from one program year to the next and provides uniformity for registration across States.

As the performance measurement system for the public labor exchange is modified, VETS will continue to coordinate with USES to ensure as much consistency as possible between the VETS measures and the ETA measures for the public labor exchange.

Comment: Other respondents expressed concern about the use of wage records, specifically the delay in the availability of wage data and the difficulty in obtaining access to wage data for Federal employees, military employees, interstate data, and occupations where employers do not regularly submit wage data to State agencies. One respondent recommended that consideration be given to adding a line item to the United States Department of Labor budget to cover the costs of States' use of the Wage Record Interchange System. Two respondents voiced concerns about the additional costs of reporting that are not borne by the DVOP and LVER grants. A final respondent suggested revisiting the economic model used by the Job Training Partnership Act where considerations such as unemployment rates and labor market conditions are factored into the results to evaluate outcomes.

Response: The three final performance measures rely heavily on wage record data for calculation. VETS acknowledges that the time lags and lack of coverage of all employers that are associated with Unemployment Insurance (UI) wage record data will pose significant challenges as States transition to this new way of measuring the performance of the public labor exchange's services to veterans. VETS continues to support the use of wage record data for performance measurement of the public labor exchange's services to veterans. Use of UI wage records is consistent with the performance measurement systems established for WIA and for the public labor exchange. The use of wage record information will ease the burden of administrative follow-up inherent in the current reporting system for the public labor exchange. In conjunction with the USES, VETS will be developing methods to adjust for economic conditions and characteristics of registered veteran job seekers, to use in establishing performance goals and for interpreting final performance levels compared to the rates established in the base line year. In addition, VETS is working with USES to develop data validation procedures to support quality control in performance measurement and data collection.

Comment: One respondent requested further clarification about the decision not to pursue the earnings gain measure that VETS tested with historical data in six States in the summer of 2000.

Response: During our analysis of the test of proposed measures in the Summer of 2000, VETS determined that results of the test did not sufficiently demonstrate that States could produce this measure validly and reliably due to differences in the way the States processed the data for this measure. In addition, VETS determined that the earnings gain measure was not an effective measure for the public labor exchange since there is no control over which jobs applicants choose. Thus, VETS decided not to use earnings gain as a performance measure.

Comment: Other comments focused on issues pertaining to the implementation of the performance measures. Three respondents asked for clarification that the performance measures are to be applied at the State level, and that States should have the authority to negotiate and establish standards at the sub-State level with local workforce investment boards. Another respondent asked for clarification of whether the five percent variance will be for the total standards or the negotiated performance standards. A final respondent asked for clarification about the time frames for establishing expected performance levels, when performance levels may be renegotiated, and the impact of receiving an incentive or renegotiating downward the initial performance levels.

Response: The VETS performance standards are intended to measure, at the State level, the effectiveness of services provided to veterans. The State Directors of Veterans' Employment and Training (DVETs) and representatives from the State agencies will negotiate the expected levels of performance, based on past performance by the entire public labor exchange system in their State. State agencies retain the authority to negotiate and establish performance standards with the local workforce investment areas in their State. The five percent variance applies to each of the negotiated performance standards, developed through the use of two years of data if possible, but not less than one year, to establish baseline data for each performance measure. In the absence of established baseline data, negotiations between DVETs and State agencies will form the basis of performance standards for the transition year of PY 2002. Data from PY 1999 and PY 2000 should serve as the basis for establishing performance levels for PY 2002. In addition, VETS is seeking several States willing to pilot new performance standards based on data for the past two program years replicating outcomes on the proposed performance measures.

Comment: One respondent also stated concerns about the implications that VETS might withhold grant funds to support incentives for States with “exemplary performance”. Another respondent suggested that VETS include incremental incentives in the definition of performance standards.

Response: VETS is exploring the feasibility of initiating an incentive program for exemplary performance in service to veterans by a State's public labor exchange system. Should this incentive program be implemented, VETS does not intend to withhold any grant funds from State agencies to support this program.

Comment: One respondent suggested that the baselines for each measure should not be developed by each State in negotiation with the DVET. Rather, VETS should establish a National formula and time frames to develop a year of baseline data. Another respondent stated that the process for negotiating baselines using prior data seems unnecessary, and that the first cycle of data following July 2002 (PY 2002) should be the basis for States' negotiations with DVETs.

Response: VETS intends for each State to develop baseline data based on their past performance. This allows for consideration of economic conditions, new business start-ups, State legislation, and other factors which might impact on an individual State's performance. Thus, as with the negotiations for performance measures under Title I of WIA, States are given the flexibility to develop their own baseline data.

Comment: One respondent suggested that comparisons be restricted to comparing veteran outcomes against veterans served, rather than comparing veterans against the total population of job seekers served by the public labor exchange. The respondent also suggested counting the number of transactions and the number of times an individual is served.

Response: Beginning in PY 2002, comparisons of services to veterans versus service to other groups will not be used as a quantitative performance measure. The new VETS performance measures do not include comparisons of veteran job seeker services or outcomes to those of other populations served by the public labor exchange. The intent of the new measures is to encourage a State's public labor exchange system to improve its services to veterans, as compared to its previous years' performance. These improvements will be demonstrated by the three outcome measures: Veteran Job Seeker Entered Employment Rate, Veteran Job Seeker Employment Retention at Six Months, and Veteran Job Seeker Entered Employment Rate Following Receipt of Staff-Assisted Services, rather than counts of the number of services provided. Counts of services provided are more process-oriented, and do not reflect VETS’ emphasis on outcomes.

Comment: One respondent asked for clarification about the definition of Disabled Veteran and Special Disabled Veteran, if the registrant did not serve over 180 days.

Response: The VETS performance measures and other reporting requirements for services to veterans by the public labor exchange will use the legal definition of veteran as established by section 4211 of Title 38, U.S.C. As this relates to disabled veterans, if the applicant did not serve on active duty for over 180 days, then he/she must have been discharged or released with other than a dishonorable discharge, and have been discharged or released because of a service-connected disability.

Comment: One respondent noted that the VETS measures do not distinguish among the One-Stop partners, public labor exchange staff, or other employment security staff such as DVOP or LVER staff who serve veteran job seekers. Additionally, the respondent asked for clarification on how States should determine which VETS 200 Report is credited with service in the event that an LVER and a DVOP both serve the same individual. Another respondent stated that since only two measures are exactly consistent with the public labor exchange measures, States will have a more difficult time integrating veterans services into the entire One-Stop system.

Response: The intent of the VETS performance standards is to measure the services to veterans by the entire public labor exchange system. This system includes One-Stop partners, public labor exchange staff, and DVOP and LVER staff. The measures are not designed to distinguish which particular program provided services to veteran job seekers. The specifications for generating the Veteran Job Seeker Entered Employment Rate measure and the Veteran Job Seeker Employment Retention Rate at Six Months measure are exactly the same as the public labor exchange measures, with the exception that the VETS measures restrict the applicant pool to veterans. The measure of Veteran Job Seeker Entered Employment Rate Following Receipt of Staff-Assisted Services is an outcome that the USES is not measuring. This, however, is an outcome measure on services that are to be provided to veterans by the entire public labor exchange system. The revised ETA 9002B and D Reports are proposed to capture the necessary information about receipt of staff-assisted services and outcomes following these staff-assisted services. VETS, in conjunction with USES, has made every effort to minimize reporting burdens incurred by States while still requesting sufficient information to demonstrate that veteran job seekers receive suitable services from the public labor exchange. The specification for the revised VETS 200C Report provides procedures for generating reports of unduplicated counts of services to veteran job seekers by DVOP and LVER staff.

C. VETS Performance Measures

We establish three performance measures for the provision of services to veterans by the public labor exchange:

• Veteran Job Seeker Entered Employment Rate.
• Veteran Job Seeker Employment Retention Rate at Six Months.
• Veteran Job Seeker Entered Employment Rate Following Receipt of Staff-Assisted Services.

The VETS performance measures apply to public labor exchange services provided to veterans as part of the One-Stop delivery systems implemented by the States. These services include those provided by DVOP and LVER staff and other public labor exchange staff employed through funds under the Wagner-Peyser Act. At their discretion, States may include other publicly-funded labor exchange services in this measurement system for services to veterans.

The VETS performance measures apply to all individuals who meet the definition of eligible veteran or other eligible, as established by sections 4101 and 4211 of Title 38, U.S.C. An eligible veteran is a person who served on active duty for a period of more than 180 days and was discharged or released therefrom with other than a dishonorable discharge; or was discharged or released from active duty because of a service-connected disability; or a member of a reserve component who served on active duty during a period of war or in a campaign or expedition for which a campaign badge is authorized and was discharged or released with other than a dishonorable discharge.

Other eligible persons must fit the following criteria, established in section 4101(5) (A), (B), and (C) of Title 38, U.S.C.:

—The spouse of any person who died of a service-connected disability;

—The spouse of any member of the Armed Forces serving on active duty, who at the time of application, is listed in one or more of the following categories and has been so listed for a total of more than ninety days: (i) Missing in action, (ii) captured in the line of duty by a hostile force, (iii) forcibly detained or interned in the line of duty by a foreign government of power; or

—The spouse of any person who has a total disability permanent in nature resulting from a service-connected disability.

A veteran job seeker is counted as a registered veteran job seeker during the quarter in which the registration occurs (registration quarter), and the subsequent three quarters. This four quarter period constitutes the registration year. A registered veteran job seeker who receives services during the fourth quarter after the registration quarter will begin a new registration year, or will be considered re-registered. This veteran job seeker will be counted again during each of the four reporting periods covering the subsequent registration year. If the veteran job seeker's registration year lapses, and after some time he or she returns to the public labor exchange, that job seeker would begin a new registration year.

The VETS performance measures are defined below.

(1) Veteran Job Seeker Entered Employment Rate (VJSEER)

The count of registered job seekers who are veterans and who, in the first or second quarter following the registration quarter (Q₊₁ or Q₊₂), earned wages from a new or different employer than that from which the registered veteran job seeker earned wages in the quarter prior to registration (Q₋₁), divided by the difference between the count of veteran job seekers who registered or re-registered with the labor exchange during any of the previous four calendar quarters and the count of any of those veteran job seekers whose wages earned in the first and second quarter following registration were exclusively with the same employer from which wages were earned in the quarter prior to registration.

This measure contains the following elements.

Entered Employment with a New Employer: To be counted as successfully entering employment, the veteran applicant must, in the first or second quarter following the quarter of registration (Q₊₁ or Q₊₂), earn wages from a different employer than from whom he/she earned wages in the quarter prior to registration (Q₋₁).

Registered Veteran Job Seeker: Veteran job seekers are considered to be registered for a four quarter period beginning with their registration quarter and the subsequent three quarters (registration year). Veteran job seekers who engaged in a labor exchange activity after their registration year has expired, will be re-registered and will then begin a new registration year.

Quarter of Registration: The calendar quarter in which a veteran job seeker completed an initial registration with the public labor exchange (Q₀) or in which a previously registered job seeker began a new registration year.

A veteran job seeker may be employed or unemployed at the time of registration. The key factor that determines whether the veteran job seeker is counted in the entered employment rate is if he/she earns wages with a new employer in the follow-up quarters after the registration quarter. A successful employment outcome is determined by comparing the Employer Identification Numbers (EIN) of registered veteran job seekers' employers prior to and following registration based on information contained in the UI wage record database, the State Directory of New Hires database, or other available records. Veteran job seekers who remain employed exclusively with the same employer(s) (those found in Q−1) during the measurement period are excluded from the calculation.

(2) Veteran Job Seeker Employment Retention Rate at Six Months (VJSERR)

The count of the number of registered veteran job seekers who are veterans age 19 and older at the time of registration, who in the first or second quarter following registration (Q₊₁ or Q₊₂), earned wages from a new or different employer than that from which the applicant earned wages in the quarter prior to registration (Q₋₁), and who also continued to earn wages in the second quarter following the quarter in which they entered employment (Q₊₃ or Q₊₄), divided by the number of veteran job seekers who entered employment during the reporting period.

This measure contains the following elements.

Entered Employment with a New Employer (age 19 and older): The base, or the denominator, of the VJSERR measure is the number of veteran job seekers age 19 and older at the time of registration who enter employment with a new employer in the first or second quarter after the quarter of registration. The process for determining entered employment is described above in the VJSERR section.

Retained Employment Two Quarters after Entered Employment with a New Employer (age 19 and older): To be counted in this measure, veteran job seekers must earn wages in the second quarter following the quarter in which they entered employment with a new employer. If a veteran job seeker enters employment in the first quarter after registration, then this measure will check to see if he/she is still earning wages in the third quarter after the quarter of registration. If the veteran job seeker enters employment in the second quarter following the quarter of registration, then this measure will check to see if he/she is still earning wages in the fourth quarter after registration.

Successful employment retention is recorded for veteran job seekers, age 19 and older at the time of registration, who are determined to have entered employment according to the entered employment rate measure, and who earned wages with any employer in the second quarter following the quarter in which they were first determined to have entered employment.

(3) Veteran Job Seeker Entered Employment Rate Following Receipt of Staff-Assisted Services (VERS)

The count of registered job seekers who are veterans and who, in the quarter of registration (Q₀) or in the first or second quarters following the quarter of registration (Q₊₁ or Q₊₂) received staff-assisted services and who in the first or second quarter following the registration quarter (Q₊₁ or Q₊₂) earned wages from a new or different employer than that from which the registered job seeker earned wages in the quarter prior to registration (Q₋₁); divided by the difference between the count of job seekers who are veterans who registered or re-registered with the labor exchange during any of the previous four calendar quarters and who in the first or second quarter following registration (Q₊₁ or Q₊₂) received staff-assisted services, and the count of any of those same job seekers whose wages earned in the first or second quarter following registration (Q₊₁ or Q₊₂) were exclusively with the same employer from which wages were earned in the quarter prior to registration (Q₋₁).

This measure contains the following elements.

Received Staff-Assisted Services: The elements of staff-assisted services can be found in the Revised ET Handbook 406. These elements were developed in consultation with the USES workgroup of State representatives, officials from the National Association of State Workforce Agencies, and staff from VETS and ETA National and Regional offices. Draft Specifications for the ET 406 Handbook were published in the Federal Register on June 6, 2001 (66 FR 30487 et seq.).

Entered Employment with a New Employer: To be counted as successfully entering employment, the veteran applicant must meet the entered employment criteria as described in measure (1) above, and have received staff-assisted services during the measurement period (Q₀−Q₊₂), as reported on the ETA 9002 Reports.

D. Levels of Performance and Rules for Application

VETS will use the WIA Title I framework (published in Training and Employment Guidance Letter No. 8-99) which was also used for negotiating and setting expected performance levels for public labor exchange services. Accordingly, States, in conjunction with their DVET, will develop baseline data for each of the measures based on historical data, analyze the baseline data, and propose performance levels for each measure based on that analysis. Each State will negotiate with its DVET to obtain mutually agreed upon expected levels of performance.

In developing baseline data, States should use two years of data if possible, but not less than one year in determining trends for performance and factors which may influence performance. In establishing expected performance levels for each measure, factors beyond the control of the State are also to be considered. When submitting their proposed performance levels, States should be prepared to provide support for their proposed levels by providing baseline performance data, the methodology for developing baseline data, and a description of data sources.

The Regional Administrator for Veterans' Employment and Training (RAVET) will review the negotiated levels of performance as submitted through the DVET and will compare the expected performance levels with the National averages, baseline information from other States, and the negotiated levels of performance established for other States, while taking into account factors including differences in economic conditions and other factors as discussed above. The RAVET will analyze the quality of the data presented by States, including the relevance of the data, the source of the data, the time period from which the data were drawn, and if the data are part of a trend or anomalous. Established GPRA Annual Performance Plan goals for relevant measures will also be an important part of the Regional review and negotiation of performance levels. When the RAVET's analysis is completed, if need be, there will be the opportunity, through the DVET, for negotiations with the State to obtain mutually agreed upon expected levels of performance. Provisions will also be made for renegotiation of performance levels if circumstances arise that result in a significant change in the factors used to establish the original levels. It is understood that either a State or VETS may elect to renegotiate performance as new information becomes available. Factors which will be considered for making changes include those discussed above.

During the first year of implementation of the new performance measures in PY 2002, performance will be reviewed as follows: States will be held harmless from any consequences of failing to meet their performance goals. Actual performance for each program year will be compared to negotiated performance levels. If a State's actual performance varies from the expected performance level by minus two percent or more, VETS will have the option of renegotiating new performance levels with the State. VETS will offer technical assistance as well as giving consideration to external factors affecting performance levels. A negative variation of five percent or more would result in the requirement of a State Corrective Action Plan (CAP) to rectify the situation. Failure to submit or comply with a CAP could become the basis for sanctions.

Signed at Washington, DC, this 16th day of April.

Frederico Juarbe Jr.,

Assistant Secretary, Veterans' Employment and Training Service.