2022-19003. Workforce Innovation and Opportunity Act Title I Non-Core Programs Effectiveness in Serving Employers Performance Indicator

AGENCY:

Employment and Training Administration (ETA), Labor.

ACTION:

Proposed rule.

SUMMARY:

The Workforce Innovation and Opportunity Act (WIOA) established six primary indicators of performance for certain WIOA-authorized programs. Currently, the regulations contain definitions for five of the six performance indicators. In the final rule implementing WIOA, the U.S. Departments of Labor and Education (the Departments) indicated that they would initially implement the sixth indicator of performance—effectiveness in serving employers—in the form of a pilot program to test the feasibility and rigor of three proposed approaches. With the pilot completed, the Departments are engaging in a rulemaking under RIN 1205-AC01 to incorporate a standard definition of the performance indicator for effectiveness in serving employers into the implementing regulations for the six WIOA core programs. In this related rulemaking, the Department of Labor (DOL or the Department) is proposing to incorporate the same definition of the effectiveness in serving employers performance indicator into regulations for title I non-core programs: the Indian and Native American (INA) programs, the Job Corps program, the YouthBuild programs, and the National Farmworker Jobs Program (NFJP).

DATES:

Interested persons are invited to submit written comments on the proposed rule on or before November 14, 2022.

ADDRESSES:

You may submit comments, identified by Docket No. ETA-2022-0005 and Regulatory Identification Number (RIN) 1205-AC08, through the Federal eRulemaking Portal: https://www.regulations.gov. Search for the above-referenced RIN, open the proposed rule, and follow the on-screen instructions for submitting comments.

Instructions: All submissions received must include the agency name and docket number for this rulemaking or “1205-AC08.” Because of the narrow scope of this proposed regulation, the Department encourages commenters to submit, and the Department will consider only comments, regarding the definition of the effectiveness in serving employers performance indicator for WIOA title I non-core programs as set forth herein. The proposed amendments are limited to the sections of the regulations detailed in this rulemaking.

Please be advised that the Department will post all comments received that relate to this notice of proposed rulemaking (NPRM) without changes to https://www.regulations.gov, including any personal information provided. The https://www.regulations.gov website is the Federal e-Rulemaking Portal and all comments posted there are available and accessible to the public. Therefore, the Department recommends that commenters remove personal information (either about themselves or others) such as Social Security numbers, personal addresses, telephone numbers, and email addresses included in their comments, as such information may become easily available to the public via the https://www.regulations.gov website. It is the responsibility of the commenter to safeguard personal information.

Because of the direct relationship between this proposed rule and the Workforce Innovation and Opportunity Act Effectiveness in Serving Employers Performance Indicator; Joint proposed rule (RIN 1205-AC01) and to ensure that comments are reviewed and considered, the Department encourages commenters to submit only comments regarding the proposed amendments to the title I non-core program regulations, which are limited to the sections of the regulations detailed in this proposed rule, to the docket that corresponds to this rulemaking action. Comments on other provisions and aspects of the WIOA regulations will be considered outside the scope of this rulemaking and will not be considered by the Department.

Docket: For access to the docket to read background documents or comments received, go to https://www.regulations.gov (search using RIN 1205-AC08 or Docket No. ETA-2022-0005).

Comments Under the Paperwork Reduction Act of 1995 (PRA): In addition to filing comments on any aspect of this proposed rule with the Department, interested parties may submit comments that concern the information collection (IC) aspects of this NPRM to the Office of Information and Regulatory Affairs (OIRA) at https://www.reginfo.gov/​public/​do/​PRAMain. Find the relevant information collection by selecting “Currently under Review—Open for Public Comments” or by using the search function.

FOR FURTHER INFORMATION CONTACT:

Heidi Casta, Acting Administrator, Office of Policy Development and Research, U.S. Department of Labor, Employment and Training Administration, 200 Constitution Avenue NW, Room N-5641, Washington, DC 20210, Telephone: [phone] (voice) (this is not a toll-free number), 1-877-872-5627, or 1-800-326-2577 (telecommunications device for the deaf).

SUPPLEMENTARY INFORMATION:

Preamble Table of Contents

I. Background and Rulemaking Authority

II. Effectiveness in Serving Employers Performance Indicator Approaches for WIOA Core Programs, as Relevant to WIOA Non-Core Programs

III. Effectiveness in Serving Employers Performance Indicator for WIOA Title I Non-Core Programs

A. Part 684—Indian and Native American Programs

B. Part 685—National Farmworker Jobs Program

C. Part 686—Job Corps Program

D. Part 688—YouthBuild Programs

IV. Regulatory Analysis and Review

A. Executive Orders 12866 (Regulatory Planning and Review) and 13563 (Improving Regulation and Regulatory Review)

B. Regulatory Flexibility Act, Small Business Regulatory Enforcement Fairness Act, and Executive Order 13272 (Proper Consideration of Small Entities in Agency Rulemaking)

C. Paperwork Reduction Act

D. Executive Order 13132 (Federalism)

E. Unfunded Mandates Reform Act

F. Executive Order 13175 (Indian Tribal Governments)

Acronyms and Abbreviations

AEFLA Adult Education and Family Literacy Act

CFR Code of Federal Regulations

Departments U.S. Departments of Labor and Education

DOL or Department U.S. Department of Labor

E.O. Executive Order

ES Employment Service

ETA Employment and Training Administration

FR Federal Register

GPMS Grantee Performance Management System

ICR Information Collection Request

INA Indian and Native American

MSFW migrant and seasonal farmworker

NAETC Native American Employment and Training Council

NFJP National Farmworker Jobs Program

NPRM or proposed rule notice of proposed rulemaking

OIRA Office of Information and Regulatory Affairs

OMB Office of Management and Budget

PIRL Participant Individual Record Layout

PRA Paperwork Reduction Act of 1995

Pub. L. Public Law

PY Program Year

RFA Regulatory Flexibility Act

RIA Regulatory impact analysis

RIN Regulation Identifier Number

Stat. United States Statutes at Large

UI unemployment insurance

UMRA Unfunded Mandates Reform Act

U.S.C. United States Code

TEGL Training and Employment Guidance Letter

VR Vocational Rehabilitation

WIOA Workforce Innovation and Opportunity Act

WIPS Workforce Integrated Performance System

I. Background and Rulemaking Authority

President Barack Obama signed WIOA into law on July 22, 2014. WIOA, the first legislative reform of the public workforce system in more than 15 years, superseded the Workforce Investment Act of 1998 and amended the Wagner-Peyser Act and the Rehabilitation Act of 1973. WIOA reaffirmed the role of the customer-focused one-stop delivery system, a cornerstone of the public workforce system, and enhanced and increased coordination among several key employment, education, and training programs. The law also includes a common performance accountability system, consisting of six statutory primary indicators of performance, applicable to all WIOA core programs: adult, dislocated worker, and youth programs under title I of WIOA; the Adult Education and Family Literacy Act (AEFLA) program under title II; the Employment Service (ES) program authorized under the Wagner-Peyser Act as amended by WIOA title III; and the Vocational Rehabilitation (VR) program authorized under title I of the Rehabilitation Act as amended by WIOA title IV. WIOA also required that the six statutory primary indicators of performance apply to four WIOA title I, DOL-administered non-core programs: INA programs (WIOA sec. 166(e)(5)), the NFJP (WIOA sec. 167(c)(2)(C)), Job Corps (WIOA sec. 159(c)(1)), and YouthBuild (WIOA sec. 171(f)) (hereinafter “title I non-core programs”).

Other DOL-administered WIOA title I non-core programs and projects ( e.g., National Dislocated Worker Grants under WIOA sec. 170, the Reentry Employment Opportunities grants under WIOA sec. 169 and annual appropriations acts) also report on the WIOA sec. 116 primary indicators of performance, as directed by Training and Employment Guidance Letter (TEGL) No. 14-18, “Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” and the DOL-only performance Information Collection Request (ICR), Office of Management and Budget (OMB) Control Number 1205-0521, “DOL-Only Performance Accountability, Information, and Reporting System.” However, unlike the other title I non-core programs that are the subject of this rulemaking, WIOA did not mandate the use of the sec. 116 performance indicators for these other title I programs. Those programs are not the subject of, or addressed in, this rulemaking, but for some of these programs, the Department has chosen to apply the sec. 116 primary indicators to assess performance.^[1] For those programs, the proposed definition of effectiveness in serving employers performance indicator also would be applied.

In WIOA, Congress directed the Department to issue regulations implementing statutory requirements to ensure that the public workforce system operates as a comprehensive, integrated, and streamlined system in order to provide pathways to prosperity and continuously improve the quality and performance of its services to job seekers and employers. On August 19, 2016, the Department issued the Workforce Innovation and Opportunity Act; Final Rule (DOL WIOA Final Rule) to implement WIOA for the title I non-core programs (81 FR 56071). That same day the Departments jointly issued the Workforce Innovation and Opportunity Act; Joint Rule for Unified and Combined State Plans, Performance Accountability, and the One-Stop System Joint Provisions; Final Rule (Joint WIOA Final Rule) to implement WIOA for the six core programs (81 FR 55791).

Under WIOA, there are six primary indicators of performance that apply to the core programs and the title I non-core programs authorized under WIOA. The statute defines five of the six performance indicators. However, the statute did not specify how effectiveness in serving employers should be measured. Instead, WIOA directed the Departments to develop a definition for the effectiveness in serving employers performance indicator (WIOA sec. 116(b)(2)(A)(iv)).^[2] At that time, the Departments concluded that there was not enough evidence to adopt a standard definition. Therefore, in the Joint WIOA Final Rule, the Departments determined that it was prudent to pilot three definitions for the sixth performance indicator to test the feasibility and rigor of three approaches to measure a State's effectiveness in serving employers through its WIOA-authorized programs. As discussed more fully below, during the pilot period the Department, through guidance ^[3] and the “DOL-Only Performance Accountability, Information, and Reporting System” ICR, approved under OMB Control Number 1205-0521, required the title I non-core programs to report on one of the three definitions being piloted.

As detailed later in this NPRM, that pilot, as well as a study of the results from the pilot, are now complete. The Departments are engaging in two rulemakings to incorporate into the WIOA regulations a proposed standard definition of the performance indicator for effectiveness in serving employers. This proposed definition is meant to apply to both WIOA core programs—which are addressed in the concurrently published Workforce Innovation and Opportunity Act Effectiveness in Serving Employers Performance Indicator; Joint proposed rule (RIN 1205-AC01) (herein after referred to as Joint Effectiveness in Serving Employers NPRM)—as well as the four title I non-core programs, which are addressed in this NPRM.

WIOA secs. 159(c)(1) (Job Corps), 166(e)(5) (INA), 167(c)(2)(C) (NFJP), and 171(f) (YouthBuild) specify that performance for these title I non-core programs must be assessed using the primary indicators of performance for WIOA core programs. In this proposed rule, the Department is proposing to codify the approach for evaluating a program's effectiveness in serving employers. When finalized, this rulemaking would result in the codification of all the primary performance indicators for these programs—including the effectiveness in serving employers indicator—just as with the WIOA core programs.

II. Effectiveness in Serving Employers Performance Indicator Approaches for WIOA Core Programs, as Relevant to WIOA Non-Core Programs

Section 677.155 sets forth the primary indicators by which the performance of core programs is evaluated, as required by WIOA sec. 116(b)(2)(A)(i). These primary indicators of performance apply to the core programs described in WIOA sec. 116(b)(3)(A)(ii), as well as to the title I non-core programs. These primary indicators of performance create a common language shared across the programs' performance metrics, support system alignment, enhance programmatic decision making, and help participants make informed decisions related to training. Sections 116(b)(2)(A)(i)(VI) and (iv) of WIOA require the Secretaries of Labor and Education to jointly develop and establish the sixth performance indicator—effectiveness in serving employers—after consultation with representatives of State and local governments, business and industry, and other interested parties.

In the Joint Effectiveness in Serving Employers NPRM, the Departments are proposing to define the effectiveness in serving employers performance indicator in § 677.155(a)(1)(vi) as the percentage of participants with wage records who exited a program and were employed by the same employer in the second and fourth quarters after exit and specifies that this is a statewide indicator reported by one core program on behalf of all six core programs in the State. The Department is proposing this is same language for the WIOA title I non-core programs in this NPRM; however, the statewide aspect of the definition in the proposed Joint Effectiveness in Serving Employers NPRM would not apply to WIOA title I non-core programs. The Department seeks comment in this NPRM on how the proposed definition of effectiveness in serving employers performance indicator would impact the title I non-core programs.

Prior to selecting this single approach to propose, the Departments selected three approaches for measuring effectiveness in serving employers to be piloted by WIOA core programs. The Departments assessed the use of each of the three approaches with a focus on minimizing employer burden and using information that would provide an accurate picture of how well the public workforce system serves employers.

Under the guidance of the Departments,^[4] each State piloted its choice of any two of three definitions for the effectiveness in serving employers performance indicator for WIOA core programs: (1) Retention with the Same Employer: Percentage of participants with wage records who exited from WIOA core programs and were employed by the same employer in the second and fourth quarters after exit; (2) Repeat Business Customer: Percentage of employers who have used WIOA core program services more than once during the last three reporting periods; and (3) Employer Penetration: Percentage of employers using WIOA core program services out of all employers in the State.

The Departments assessed the pilot through a Department of Labor contract that resulted in a final report titled Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act.^[5] Specifically, the study assessed each approach to defining the effectiveness in serving employers performance indicator for validity, reliability, practicality, and unintended consequences.^[6] Though the study did not definitively recommend one approach, in assessing the study's findings for each of the three approaches of the effectiveness in serving employers performance indicator, the Departments concluded that the Retention with the Same Employer approach provides a valid and reliable approach to measuring the indicator, while also placing the least amount of burden on States to implement.

The study authors identified strengths for the Repeat Business Customer approach, including that it serves as a proxy for employer satisfaction. In the study, the authors also identified weaknesses in the Repeat Business Customer approach, including that it: (1) may provide a disincentive to reach out to new employers; (2) is subject to variation in industry and sector economic conditions; and (3) may require a statistical adjustment model to mitigate the weaknesses and improve implementation and interpretation.^[7] The study authors identified strengths for the Employer Penetration approach, including that the dataset used for this measure is comprehensive, covering more than 95 percent of U.S. jobs. The study authors also identified weaknesses in the Employer Penetration approach through the study, including: (1) emphasis on quantity rather than quality or intensity of the employer service provided; (2) reliability issues associated with data entry and the process to count unique establishments; (3) measurement of program output rather than outcome; (4) potential for creation of perverse incentives to prioritize program breadth rather than depth in service and delivery; and (5) lack of sensitivity to industry sectors targeted by State and local workforce agencies.^[8] The Departments considered the study's findings and concurred with its conclusions on the Repeat Business Customer and Employer Penetration approaches.

The study did not identify any significantly advantageous alternatives to defining the effectiveness in serving employers performance indicator outside of the three proposals (Executive Summary, pp. xx-xxi).

Nevertheless, the Departments identified the following advantages regarding the Retention with the Same Employer definition of the effectiveness in serving employers performance indicator:

• Demonstration of Effectiveness: Retention with the Same Employer demonstrates a continued relationship between the employer and participants who have exited WIOA programs. While many circumstances affect an employer's retention of employees, an indication that an employee maintains employment with the same employer in both the second and fourth quarters after exiting from a WIOA program demonstrates a level of success for WIOA customers ( i.e., successfully preparing participants to fill jobs that meet employers' needs). Retention of an employee reduces the costs to the employer associated with employee turnover and retraining. The other two approaches are based only on employer data and fail to capture any level of job match effectiveness.

• Stable Collection Mechanism: Retention with the Same Employer uses data already collected in the WIOA Joint Performance ICR (OMB Control Number 1205-0526). While not all States selected this approach in the pilot, all States collect this information under the existing WIOA Joint Performance ICR. In contrast, the Participant Individual Record Layout (PIRL) in the WIOA Joint Performance ICR does not currently collect data elements used for the Repeat Business Customer and Employer Penetration approaches to the performance indicator.

• Alignment with Employment Performance Indicators: Retention with the Same Employer aligns with the performance indicators for employment in the second and fourth quarters after exit, which are existing performance indicators that all WIOA core programs already report.

Of the three approaches piloted with the States, Retention with the Same Employer is the least burdensome for both States and employers, as noted in the Joint WIOA Final Rule regulatory impact analysis (RIA) (81 FR 55792, 55968). DOL gives particular weight to reporting burden, especially for the competitive grantees with generally less reporting capacity than States, in order to allow grantees to focus on services and improve the accuracy and completeness of the data. However, the Department acknowledges that the limitations for Retention with the Same Employer could include the unintended consequences that this approach may be at odds with an employee seeking a higher paying job or employment benefits, and the possibility that the performance outcome for this indicator might not be the result of an employer receiving a service from the workforce development system. Prioritizing the advantages discussed above ( i.e., stable data collection mechanism, alignment with other employment performance indicators, and demonstrating maintained relationships between employers and employees), the Department has determined Retention with the Same Employer is the preferred approach of measuring effectiveness in serving employers and are proposing that approach in the Joint Effectiveness in Serving Employers NPRM. For further information on the pilot, including the Departments' findings regarding the utility of each pilot definition and reasoning for selecting the Retention with the Same Employer performance indicator definition, please refer to the Joint Effectiveness in Serving Employers NPRM, which is published concurrently with this NPRM elsewhere in this issue of the Federal Register .

III. Effectiveness in Serving Employers Performance Indicator for WIOA Title I Non-Core Programs

Although the four WIOA title I non-core programs in this rulemaking—Job Corps, INA, NFJP, and YouthBuild—did not participate in the core program pilot, these title I non-core program fund recipients ( i.e., Job Corps contractors and INA, NFJP, and YouthBuild grantees) were apprised of the three proposed definitions for the effectiveness in serving employers performance indicator that the pilot studied.^[9] Moreover, the title I non-core program recipients have been required to report on Retention with the Same Employer since at least 2019. In TEGL No. 14-18 the Department implemented WIOA's performance reporting requirements by requiring the non-core programs to use the Retention with the Same Employer definition of the effectiveness in serving employers performance indicator.

Under this proposed rule, the WIOA title I non-core programs would be subject to the same data collection and reporting requirements as they have been under TEGL No. 14-18. The TEGL specified that, starting in Program Year (PY) 2018 (or the point at which wage matching data becomes available to the program), the Job Corps, INA, NFJP, and YouthBuild programs were to begin tracking the effectiveness in serving employers performance indicator using the Retention with the Same Employer definition. Consistent with related guidance issued in PYs 2016, 2017, and 2018,^[10] these programs were required to use the Workforce Integrated Performance System (WIPS), the online performance reporting system for the Department's employment and training grants,^[11] to submit information that would be used for calculating the effectiveness in serving employers performance indicator.^[12] These requirements are all included in an existing information collection, the WIOA PIRL (ETA 9172), in the “DOL-Only Performance Accountability, Information, and Reporting System” ICR, approved under OMB Control Number 1205-0521. By proposing to use the Retention with the Same Employer definition for this indicator, the NPRM would require programs to use already-collected data and the existing performance reporting system, WIPS. Thus, programs would not have additional burden to collect and report on any other type of additional data to calculate and report results for other possible approaches to defining this performance indicator. Finally, TEGL No. 14-18 also put forth program-specific timelines for implementation of the WIOA reporting requirements factoring in data lags associated with the performance indicator as well as known implementation actions such as case management system development, which are further detailed below in each program-specific section. In summary, for these four title I non-core programs (Job Corps, INA, NFJP, and YouthBuild), this NPRM proposes to codify in regulation the existing practice of reporting Retention with the Same Employer in order to measure a program's effectiveness in serving employers.

As discussed above, the Department has concluded that the benefits of this proposed performance indicator definition with regard to the core programs—that, among other things, it places a low burden on the programs and employers, has a stable method of collection through wage records, and demonstrates a level of success for WIOA customers—are also applicable to the title I non-core programs. Using the proposed Retention with the Same Employer definition of the effectiveness in serving employers indicator, which would be the same definition used to assess the core programs, has the advantage of assessing performance consistently across the WIOA programs. This is consistent with one of the central purposes of WIOA: “[t]o support the alignment of workforce investment, education, and economic development systems in support of a comprehensive, accessible, and high-quality workforce development system in the United States.” WIOA sec. 2(2). Additionally, because WIOA applies the effectiveness in serving employers performance indicator to the WIOA core and title I non-core programs, applying the same definition of effectiveness in serving employers for all of these WIOA programs could allow the Department to build a common body of data that can be used to study effectiveness in serving employers across the entire workforce system.

While reporting this performance indicator contributes to the holistic data analysis of the workforce system, the Department recognizes that drawbacks to this proposed definition exist for the title I non-core programs, especially due to the unique nature of programs focused on youth and migrant or seasonal workers. Nevertheless, the Department believes that the benefits of this approach outweigh those drawbacks. Moreover, the Department intends to mitigate these drawbacks, if necessary, by exercising its discretion to place appropriate weight on the effectiveness in serving employers performance indicator. Title I non-core programs that serve youth, for example, focus on employment, career readiness, retention in education, and life skills to support youth participants in obtaining academic and career skills necessary to be successful in the job market, and success for youth is more likely to include progression in jobs. Recognizing the unique circumstances title I non-core programs may face, the Department expects variability in the reported outcomes from program to program, especially for programs serving youth, and intends to take this variability into account when negotiating levels of performance. These considerations are consistent with TEGL No. 14-18 guidance for applicability of primary performance indicators, which specifies that, as a general matter, participants' outcomes on the applicable primary indicators of performance may be relevant for negotiating levels of performance, decisions related to contract awards and renewal, and the award of competitive grants.^[13]

It should be kept in mind that the effectiveness in serving employers performance indicator is unique among all other indicators in that it is employer-focused. Employers are critical partners with title I non-core programs in providing quality services and employment opportunities to program participants.

While WIOA does require an effectiveness in serving employers indicator to be applied to the title I non-core programs that are the subject of this rulemaking, the Department is soliciting comments to better inform implementation of the effectiveness in serving employers performance indicator for these programs, particularly those currently undergoing transition to the Grantee Performance Management System (GPMS). The Department is particularly interested in hearing from the regulated community regarding challenges that they might face in implementing this proposed definition of the effectiveness in serving employers performance indicator; challenges they have faced under TEGL No. 14-18, which serves as the basis for how the performance indicator is proposed to be defined in this NPRM; experiences they have had in considering alternate ways to measure effectiveness in serving employers; and other definitions that might be more suitable.

A. Part 684—Indian and Native American Programs

Part 684 governs the INA programs authorized under WIOA sec. 166, including programs for Native American youth (INA Supplemental Youth Services). The INA programs are intended to support employment and training activities for INA program participants in order to develop more fully academic, occupational, and literacy skills and to serve unemployed and low-income INA populations seeking to achieve economic self-sufficiency consistent with the goals and values of the particular communities. Where active, INA programs are required one-stop center partners. The Department administers these programs to maximize Federal commitment to support the growth and development of INAs and their communities as determined by representatives of such communities while meeting the applicable statutory and regulatory requirements.

WIOA sec. 166(h)(2) requires the Department to reach an agreement with Tribal Governments—and the respective entities administering the programs—as to the levels of performance required for each core indicator, including an effectiveness in serving employers performance indicator. The Department is also required to work with the Native American Employment and Training Council (NAETC) to develop a set of performance indicators and standards for the INA adult and youth programs in addition to the primary indicators used to measure performance (WIOA sec. 166(h)(1)(A)).

Beginning with PY 2018, ETA has applied the effectiveness in serving employers performance indicator to INA adult grants as it is described in TEGL No. 14-18, using the Retention with the Same Employer definition of the performance indicator. Specifically, on March 25, 2019, TEGL No. 14-18, Attachment 2 provided that the definition for effectiveness in serving employers performance indicator for INA program reporting purposes would be consistent with the Retention with the Same Employer approach applicable to DOL-administered WIOA title I non-core programs and described in Appendix I of the TEGL. On November 20, 2019, the ICR approved under OMB Control Number 1205-0521 formally established for INA programs the calculation of effectiveness in serving employers and the collection of required elements for the effectiveness in serving employers performance indicator. The cohort of INA adult program participants who exited after July 1, 2020, is the first that may have effectiveness in serving employers data collected, which will be compiled and analyzed in summer 2022.

For the INA Supplemental Youth Services program, the DOL WIOA Final Rule and TEGL No. 14-18 both acknowledged the significant challenges in implementing the performance indicators in WIOA sec. 116(b)(2)(A)(ii). In implementing these performance indicators in TEGL No. 14-18, the Department gave consideration as to how youth performance indicators can be implemented in a way that is realistic and feasible for INA program grantees while also implementing the requirements in WIOA. INA Supplemental Youth Services program participants will be reported once the INA youth case management system modernization has been completed, at which time it will be at least six additional quarters until the first data on effectiveness in serving employers will be available. INA grantees will eventually report on this performance indicator, but given the complexity of aligning data elements and building new systems to report such data, the Department is using the transition authority found in WIOA sec. 503(b) to work co-operatively with grant program organizations to transition to reporting of the information over time.^[14]

In 2021, as part of the development of this proposed rule, the Department held two events ^[15] to consult with INA program grantees and representatives of Tribal institutions about their experiences with the implementation and operation of the effectiveness in serving employers performance indicator under TEGL No. 14-18. Participants at these two events expressed several concerns and questions, including: (1) how the Retention with the Same Employer performance indicator definition takes into account participants' employer, wage, or position changes; (2) how temporary jobs, such as seasonal or contract-based employment, would be considered; (3) the impact on performance of limited-duration summer employment opportunities for high school students within INA youth programs, (4) data collection and reporting process for INA youth programs, (5) use of and access to wage records that may not account for self-employed participants, and (6) the need for consideration of all Tribal communities and their unique needs. Other commenters suggested other ways to define the calculation of the performance indicator. One commenter asserted that the Department is not required to assess INA grantees on their effectiveness in serving employers. Section IV.F of this document, which pertains to Executive Order (E.O.) 13175 (Indian Tribal Governments), summarizes details from these events and requests further comments to provide the Department with recommendations and suggestions to address the issues identified through this consultation. The concerns raised during the consultation process can be classified into several categories: (1) issues focusing on services to participants (wages and position changes, temporary or contract jobs, and summer employment); (2) administrative and data tracking (data collection and use of wage records); (3) the needs of the Tribal communities.

If this rulemaking is finalized as proposed, the Department intends to work with INA program grantees to mitigate these concerns. First, INA program grantees' services to participants also are measured and assessed through the other five WIOA primary indicators of performance, and the Department recognizes the importance of these indicators in assessing the performance of INA program grantees. The effectiveness in serving employers performance indicator, unlike the other indicators, which are focused on program participants, focuses on how the WIOA programs are serving employers. As explained above, the proposed performance indicator definition of Retention with the Same Employer is one metric by which to ascertain how employers are being served by these programs. Second, the Department acknowledges and understands the challenges related to reporting for INA program grantees and is working to ensure that all INA program grantees have the systems and resources needed to report the information required for this performance indicator. Third, the Department acknowledges the concerns of Tribal communities and their unique needs. WIOA makes provision for the Department to negotiate additional performance indicators and standards taking into account the needs of participants and the economic circumstances of the communities INA program grantees serve. See WIOA sec. 166(h)(1). The Department will negotiate these additional performance indicators keeping these considerations in the forefront of the negotiations process. INA program grantee performance also is assessed based on these outcomes. Effectiveness in serving employers is not the only metric for assessing INA program grantee performance.

While the Department acknowledges the concerns that have been expressed by INA grantees during the Tribal consultation for this proposed rule regarding application of the effectiveness in serving employers to INA adult and youth programs and will work to mitigate the issues such concerns raise, we note that WIOA requires the performance of these programs to be measured using the WIOA sec. 116 six statutory indicators of performance, including effectiveness in serving employers. Specifically, WIOA sec. 166(h)(2) requires the Secretary to reach agreement on the levels of performance for each of the primary indicators of performance described in WIOA sec. 116(b)(2)(A), which includes the effectiveness in serving employers indicator.

Further, as explained above, the benefits of defining this measure using Retention with the Same Employer, including that it minimizes reporting burdens for INA program grantees, outweigh the drawbacks, as well as providing more benefits than the use of either of the other performance indicator definitions piloted by the core programs. To fulfill the intent of WIOA's common performance accountability system, the Department is proposing to define effectiveness in serving employers for the INA programs using the Retention with the Same Employer approach so that the Department can measure effectiveness in serving employers consistently across core programs and the title I non-core programs.

Additionally, the Department notes that WIOA sec. 166(i)(3) and the WIOA regulations at 20 CFR part 684 subpart I allow the Department to waive requirements, including performance requirements, that are inconsistent with the specific needs of INA grantees. Based on consultation with the NAETC, the Department issued guidance TEGL No. 04-19, “Waiver Authority for the INA Program and Implementation of Additional Indicators of Performance,” ^[16] which provides how INA grantees can request waivers of performance indicators, and how grantees with waivers can report on alternative performance indicators for INA adult and youth programs. As consultation commenters discussed, performance reporting can be particularly challenging for smaller grantees. Therefore, if this rulemaking is finalized as proposed, consistent with this waiver guidance, the Department would accept and promptly make determinations on requests submitted by grantees for waivers of performance indicators, including effectiveness in serving employers, so that grantees can structure their performance indicators to best fit the economic circumstances of the communities served and improve positive outcomes.

Section 684.460—What performance indicators are applicable to the supplemental youth services program?

Section 684.460(a) sets out the performance indicators that apply to INA youth programs, including an indicator of the effectiveness of serving employers—specifically in paragraph (a)(6)—as established under WIOA sec. 116(b)(2)(A)(iv). This NPRM proposes to change the language currently found in paragraph (a)(6) to align with the effectiveness in serving employers performance indicator language proposed at § 677.155(a)(1)(vi) in the Joint Effectiveness in Serving Employers NPRM. Specifically, proposed § 684.460(a)(6) would define the required effectiveness in serving employers performance indicator as the percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

Section 684.620—What performance indicators are in place for the Indian and Native American program?

Section 684.620(a) lists the performance indicators used to evaluate the INA programs, including an effectiveness in serving employers performance indicator. Like the proposed changes to § 684.460(a)(6), the Department proposes changing the existing language at § 684.620(a)(6) to define the required effectiveness in serving employers performance indicator as the percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit. This definition of effectiveness in serving employers aligns with the effectiveness in serving employers performance indicator language proposed at § 677.155(a)(1)(vi) in the Joint Effectiveness in Serving Employers NPRM.

B. Part 685—National Farmworker Jobs Program

Part 685 establishes regulations for NFJP, authorized in title I, subtitle D of WIOA. The NFJP is a nationally directed, locally administered program of services for migrant and seasonal farmworkers (MSFWs) and their dependents. Grant recipients help program participants acquire new skills to either stabilize or advance their agricultural careers or obtain employment in a new industry, as well as working to meet the critical need of safe and sanitary permanent and temporary housing for farmworkers and their families.

The NFJP would be impacted by the proposed addition of the definition of the effectiveness in serving employers performance indicator in 20 CFR part 677. Section 167(c)(3) of WIOA (29 U.S.C. 3222) requires the Department to use the six WIOA primary indicators of performance, including the effectiveness in serving employers performance indicator, to assess the performance of the NFJP. In the DOL WIOA Final Rule, the Department implemented this requirement in 20 CFR 685.400(a) and (b), which states that NFJP grantees providing career services and training use the indicators of performance described in WIOA sec. 116(b)(2)(A). NFJP housing grantees, which provide housing assistance rather than training and employment placement services, are required to report a different set of performance indicators as defined in 20 CFR 685.400(c), specifically the total number served of eligible MSFWs, other individuals, eligible MSFW families, and other families. Therefore, if finalized, the proposed definition of the effectiveness in serving employers performance indicator in 20 CFR part 677 in the Joint Effectiveness in Serving Employers NPRM would apply to NFJP career services grantees but not housing grantees, although it would have no noticeable change to procedures for career services grantees as they already report this information in accordance with TEGL No. 14-18. Beginning with PY 2018, NFJP career services grants have applied the effectiveness in serving employers performance indicator as it is described in TEGL No. 14-18, using the Retention with the Same Employer definition of the performance indicator. However, the third quarter of PY 2020 was the first quarter where NFJP generated quarterly performance reports in WIPS with the effectiveness in serving employers performance indicator. No changes to the regulatory text at 20 CFR part 685 are necessary to implement this change, as the regulations currently state that the Department uses the indicators of performance described in WIOA sec.116(b)(2)(A) and do not state a definition directly.

C. Part 686—Job Corps Program

Part 686 establishes regulations for the Job Corps program, authorized in title I, subtitle C of WIOA. Job Corps is a no-cost education and career technical training program administered by the Department, which includes 121 Job Corps centers across the United States. The program aims to help young people—ages 16 to 24—gain academic credentials and career technical training skills and secure quality employment.

Job Corps historically has used post-separation surveys to capture post-program employment results. Job Corps' current surveys (OMB Control Number 1205-0426) are administered to participants immediately following the second and fourth quarters after exit and capture information related to whether they are employed or in an educational or training program during those quarters and if they have attained any additional certifications or credentials after exit from the program. In PY 2018, Job Corps revised the reporting periods in the post-separation surveys to replace program-specific definitions of the second and fourth quarters after exit with the same definitions used by other DOL employment and training programs. This definitional shift created alignment with quarterly wage records and facilitated calculation of common exit and outcomes across WIOA programs. With this change in definition, Job Corps has been able to apply the effectiveness in serving employers performance indicator as it is described in TEGL No. 14-18, using the Retention with the Same Employer definition of the performance indicator. While the post-separation surveys are a supplemental data source for reporting on the primary indicators of performance, Job Corps did not gain access to wage record matches, the primary data source, until the fourth quarter of PY 2020. All reported outcomes for Job Corps prior to this period were based solely on the supplemental data source. Job Corps began certifying its program results in WIPS for all the primary measures of performance, including the Retention with the Same Employer indicator, in the first quarter of PY 2020. Starting with the fourth quarter of PY 2020, Job Corps obtained quarterly wage record matches and, combined with the supplemental data from the surveys, has been able to report fully on the primary measures of performance, including the Retention with the Same Employer indicator.

Section 686.1010—What are the primary indicators of performance for Job Corps centers and the Job Corps program?

Section 686.1010 lists the primary indicators used to measure the performance of Job Corps centers, which includes the effectiveness in serving employers performance indicator. The effectiveness in serving employers performance indicator specifically applies to Job Corps center operators and career transition service providers. The Department proposes to change the existing language found at § 686.1010(f) to align with the effectiveness in serving employers performance indicator language proposed at § 677.155(a)(1)(vi) in the Joint Effectiveness in Serving Employers NPRM. Specifically, proposed § 686.1010(f) would define the required effectiveness in serving employers performance indicator as the percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

D. Part 688—YouthBuild Programs

Part 688 establishes regulations for the YouthBuild programs, authorized in title I, subtitle D of WIOA. YouthBuild is a pre-apprenticeship program that provides educational and job training opportunities for at-risk youth (ages 16-24) who have previously dropped out of high school. Program participants learn vocational skills focused on the construction industry, as well as other in-demand industries including healthcare, information technology, and hospitality. Participants earn their high school diploma while splitting time between the vocational training work site and the classroom, as well as preparing for postsecondary training opportunities, such as Registered Apprenticeships, college, and eventual employment. Community service is required of participants, including through construction and rehabilitation of affordable housing for low-income and homeless families, often in their own neighborhoods. YouthBuild programs include mentoring, follow-up education, employment, and personal counseling services as support systems for program participants as well. YouthBuild grants include a 4-month planning period and run on a cohort model, which spans from 6 to 12 months.

On March 25, 2019, TEGL No. 14-18, Attachment 11, provided that the definition for the effectiveness in serving employers performance indicator for YouthBuild reporting purposes would be consistent with the Retention with the Same Employer approach generally applicable to DOL-administered WIOA programs and described in Appendix I to the TEGL. On November 20, 2019, the ICR approved under OMB Control Number 1205-0521 formally established for YouthBuild programs the calculation of effectiveness in serving employers and the collection of required elements for effectiveness in serving employers. YouthBuild program participants will be reported once the case management system modernization is completed, at which time it will be at least an additional six quarters until the first data on effectiveness in serving employers will be available. The YouthBuild participants from the grant class that began on July 1, 2021, is the first that may have effectiveness in serving employers data available, which would be available in the quarter ending on September 30, 2023.

Section 688.400—What are the performance indicators for YouthBuild grants?

Section 688.400 lists the primary indicators used to measure the performance of YouthBuild programs, which also includes a performance indicator for effectiveness in serving employers. This NPRM proposes to codify current practices by replacing existing language in § 688.400(f) with language that aligns with the effectiveness in serving employers performance indicator language proposed at § 677.155(a)(1)(vi) in the Joint Effectiveness in Serving Employers NPRM. Specifically, proposed § 688.400(f) would define the required effectiveness in serving employers performance indicator as the percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

IV. Regulatory Analysis and Review

A. Executive Orders 12866 (Regulatory Planning and Review) and 13563 (Improving Regulation and Regulatory Review)

Under E.O. 12866, OIRA determines whether a regulatory action is significant and, therefore, subject to the requirements of the E.O. and review by OMB. See 58 FR 51735 (Oct. 4, 1993). Section 3(f) of E.O. 12866 defines a “significant regulatory action” as an action that is likely to result in a rule that: (1) has an annual effect on the economy of $100 million or more, or adversely affects in a material way a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local, or Tribal governments or communities (also referred to as economically significant); (2) creates serious inconsistency or otherwise interferes with an action taken or planned by another agency; (3) materially alters the budgetary impacts of entitlement grants, user fees, or loan programs, or the rights and obligations of recipients thereof; or (4) raises novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles set forth in the E.O. Id. This proposed rule is a significant regulatory action, although not an economically significant regulatory action under sec. 3(f) of E.O. 12866. Accordingly, OMB reviewed this proposed rule.

E.O. 13563 directs agencies to propose or adopt a regulation only upon a reasoned determination that its benefits justify its costs; the regulation is tailored to impose the least burden on society, consistent with achieving the regulatory objectives; and in choosing among alternative regulatory approaches, the agency has selected those approaches that maximize net benefits. E.O. 13563 recognizes that some benefits are difficult to quantify and provides that, where appropriate and permitted by law, agencies may consider and discuss qualitatively values that are difficult or impossible to quantify, including equity, human dignity, fairness, and distributive impacts.

1. Outline of the Analysis

Section IV.A.2 provides a summary of the results of the RIA. Section IV.A.3 describes the need for the proposed rule, and Section IV.A.4 describes the process used to estimate the costs and cost savings of the proposed rule and the general inputs used, such as wages and number of affected entities. Section IV.A.5 explains how the provisions of the proposed rule would result in quantifiable costs and cost savings and presents the calculations the Department used to estimate them. In addition, Section IV.A.5 describes the qualitative benefits of the proposed rule. Section IV.A.6 summarizes the estimated first-year and 10-year total and annualized costs, cost savings, net costs, and transfer payments of the proposed rule. Finally, Section IV.A.7 describes the regulatory alternatives considered when developing the proposed rule.

2. Analysis Overview

The Department estimates that the proposed rule would result in costs and qualitative benefits. As shown in Exhibit 1, the proposed rule is expected to have a one-time cost of $41,551. The Departments estimate that the proposed rule would result in an annualized net quantifiable cost of $5,916 at a discount rate of 7 percent and expressed in 2020 dollars.

Exhibit 1—Estimated Monetized Costs of the Proposed Rule

[2020 dollars]

	Cost
10-Year Total with a Discount Rate of 3%	$41,551
10-Year Total with a Discount Rate of 7%	41,551
10-Year Average	4,155
Annualized at a Discount Rate of 3%	4,871
Annualized at a Discount Rate of 7%	5,916

The cost of the proposed rule is associated with rule familiarization for all 121 Job Corps centers and 1 career transition service provider for a total of 122 Job Corps entities, 53 NFJP career service and training grantees, 69 INA youth grantees, 104 INA adult grantees, and 216 YouthBuild grantees.^[17] See the costs subsections of Section IV.A.5 (Subject-by-Subject Analysis) below for a detailed explanation.

The Department cannot quantify the benefits of the proposed rule; therefore, Section IV.A.5 (Subject-by-Subject Analysis) describes the benefits qualitatively.

3. Need for Regulation

This proposed rulemaking is necessary to complete implementation of the performance accountability requirements as discussed in the Joint WIOA Final Rule and required by statute. WIOA included a common performance accountability system, consisting of six statutory primary indicators of performance, applicable to all WIOA core programs: adult, dislocated worker, and youth programs under title I of WIOA; the AEFLA program under title II; the ES program authorized under the Wagner-Peyser Act as amended by WIOA title III; and the VR program authorized under title I of the Rehabilitation Act, as amended by WIOA title IV. WIOA also required that the six statutory primary indicators of performance apply to four WIOA title I, DOL-administered non-core programs: INA, NFJP, Job Corps, and YouthBuild (“title I non-core programs”). The statute defines five of the six performance indicators. However, WIOA did not specify how effectiveness in serving employers should be measured. Instead, WIOA directed the Departments to develop a definition for the effectiveness in serving employers performance indicator (WIOA sec. 116(b)(2)(A)(iv)). In the Joint WIOA Final Rule, the Departments determined that it was prudent to pilot three definitions for the sixth performance indicator, which measures a State's effectiveness in serving employers through its WIOA-authorized programs. As explained earlier in this proposal, that pilot, as well as a study of the results from the pilot, is now complete. The Departments are engaging in two rulemakings to incorporate into the WIOA regulations a proposed standard definition of the performance indicator for effectiveness in serving employers. This proposed performance indicator definition is meant to apply to both WIOA core programs—which are addressed in the concurrently published Joint Effectiveness in Serving Employers NPRM—as well as the four title I non-core programs, which are addressed in this NPRM. When finalized, this rulemaking would codify the use of all the primary performance indicators for the evaluation of title I non-core program performance—including the effectiveness in serving employers indicator—just as with the WIOA core programs.

4. Analysis Considerations

a. Baseline for Title I Non-Core Programs: Indian and Native American, Job Corps, and YouthBuild

The Department estimated the costs of the proposed rule relative to the existing baseline. The Department determined that the proposed rule would result in no change from the baseline for the title I non-core programs. As a result, the Department estimates only the costs of rule familiarization for the title I non-core programs.

WIOA secs. 159(c)(1) (Job Corps), 166(e)(5) (INA), 167(c)(2)(C) (NFJP), and 171(f) (YouthBuild) specify that performance for these title I non-core programs must be assessed using the WIOA sec. 116 primary indicators of performance for WIOA core programs. In this proposed rule, the Department is codifying the approach for evaluating a program's effectiveness in serving employers, as put into practice through previously issued guidance ^[18] and the “DOL-Only Performance Accountability, Information, and Reporting System” ICR, approved under OMB Control Number 1205-0521 for the title I non-core programs.

All title I non-core programs, except the INA Supplemental Youth Services program, are able to report the Retention with the Same Employer definition of effectiveness in serving employers performance indicator, as required in TEGL No. 14-18, through WIPS or GPMS. Unlike the other title I non-core programs, the INA Supplemental Youth Services program is not currently reporting, and will not immediately be able to report, the effectiveness in serving employers performance indicator. The INA Supplemental Youth Services case management system modernization has not been completed at the time of this rulemaking; therefore, INA youth grantees will, for a period of time, use WIOA transition authority with regard to collecting and reporting on WIOA performance indicators, including the proposed effectiveness in serving employers performance indicator. The Department is planning, independent of this rulemaking, to build a new case management system for INA youth grantees that will provide for the collection and reporting of the effectiveness in serving employers performance indicator. Therefore, this proposed rule does not impose any new cost associated with the case management system. When the case management system is built, the INA youth grantees will use it to collect and report the outcomes for the effectiveness in serving employers performance indicator. The use of the new system to report the effectiveness in serving employers performance indicator would impose a de minimis cost for the INA youth grantees. When the INA Supplemental Youth Services case management system is complete, the INA youth program grantees would face a de minimis cost associated with reporting the effectiveness in serving employers performance indicator in the new system.

Exhibit 2 presents the number of entities the Department expects the proposed rule to affect. The Department provides these estimates and uses them to calculate the cost of rule familiarization for the title I non-core programs.

Exhibit 2—Title I Non-Core Programs Number of Affected Entities by Type

Entity type	Number
Job Corps:
Current centers	121
Career transition service providers	1
NFJP:
Career services and training grantees	53
Indian and Native American:
Number of INA youth grants awarded under WIOA sec. 166	69
Grantees for the Comprehensive Services Program/INA adult program	104
YouthBuild:
Grantees in active grant classes	216

b. Compensation Rates

In Section IV.A.5 (Subject-by-Subject Analysis), the Department presents the costs, including labor, associated with the proposed rule. Exhibit 3 presents the hourly compensation rates for the occupational categories expected to experience a change in level of effort (workload) due to the proposed rule. We use the Bureau of Labor Statistics (BLS) mean hourly wage rate for local government employees.^[19] To reflect total compensation, wage rates include nonwage factors such as overhead and fringe benefits ( e.g., health and retirement benefits). We use an overhead rate of 17 percent ^[20] and a fringe benefits rate of 62 percent,^[21] which represents the ratio of average total compensation to average wages for State and local government workers in March 2021. We then multiply the sum of the loaded wage factor and overhead rate by the corresponding occupational category wage rate to calculate an hourly compensation rate.

Exhibit 3—Compensation Rates [2020 Dollars]

Position	Grade level	Base hourly wage rate	Loaded wage factor	Overhead costs	Hourly compensation rate
		(a)	(b)	(c)	d = a + b + c
Management Analyst	N/A	$41.23	$25.43 ($41.23 × 0.62)	$7.01 ($41.23 × 0.17)	$73.67

5. Subject-by-Subject Analysis

The Department's analysis below covers the estimated cost of the proposed rule.

c. Costs

The following sections describe the costs of the proposed rule.

(1) DOL-Only Non-Core Programs Rule Familiarization

If the proposed rule is finalized, INA, YouthBuild, NFJP, and Job Corps programs would need to familiarize themselves with the new regulation. Consequently, this would impose a one-time cost in the first year.

To estimate the first-year cost of rule familiarization for INA, YouthBuild, NFJP, and Job Corps programs, the Department multiplied the estimated number of management analysts (1) by the time required to read and review the rule (1 hour), and by the applicable hourly compensation rate ($73.67/hour). We multiplied this result by the number Job Corps active centers (122), NFJP grantees (53), INA Youth program grantees (69), INA Adult program grantees (104), and the number of YouthBuild grantees (216). This calculation yields $41,551 in one-time labor costs for Job Corps, NFJP, INA Youth, and INA Adult programs to read and review the rule. Over the 10-year period of analysis, these estimated one-time costs result in an average annual cost of $4,155 undiscounted, or $4,871 and $5,916 at discount rates of 3 and 7 percent, respectively.

d. Qualitative Benefits Discussion

(1) General Benefits of Measuring Effectiveness in Serving Employers

The Department cannot quantify the proposed rule's benefits associated with improving the title I non-core programs' effectiveness in serving employers. Measuring effectiveness in serving employers allows title I non-core programs to set goals, monitor, and learn how to serve employers more effectively.^[22] Reporting a measure of effectiveness in serving employers also helps Federal, State, and local policymakers evaluate program performance and inform future policy changes to better meet program goals, particularly providing employers with skilled workers and other services.

The Department cannot quantify these estimated benefits because we do not have quantitative data on how the effectiveness in serving employers performance indicator has influenced program implementation and how much it would influence future policies.

(2) Specific Benefits of Reporting Retention With the Same Employer

Requiring the calculation and reporting of Retention with the Same Employer as the effectiveness in serving employers performance indicator would make it easier to compare WIOA title I non-core programs' effectiveness in serving employers performance across grant programs. Retention with the Same Employer demonstrates a continued relationship between the employer and participants who have exited WIOA programs. While many circumstances can have an impact on an employer's retention of employees, an indication that an employee is still working for the same employer in both the second and fourth quarters after exiting from a WIOA program demonstrates a level of success for both parties, as retention of an employee reduces the costs to the employer associated with employee turnover and retraining. Thus, reporting Retention with the Same Employer can help inform design and implementation of program services to reduce job turnover and improve employer-employee match quality. Improved matching and reduced turnover allow employees and employers to operate closer to their productive potential and can make it more worthwhile for employers to invest in training its employees and for employees to invest in learning employer-specific skills.

6. Summary of the Analysis

The Department estimates the total net cost of the proposed rule at $41,183 at a discount rate of 7 percent. The Department estimates the annualized net cost of the proposed rule at $5,864 at a discount rate of 7 percent. Exhibit 4 summarizes the estimated cost of the proposed rule over the 10-year analysis period.

Exhibit 4—Estimated Monetized Costs of the Proposed Rule

[2020 dollars]

	Costs
2022	$41,551
2023	0
2024	0
2025	0
2026	0
2027	0
2028	0
2029	0
2030	0
2031	0
10-Year Total with a Discount Rate of 3%	41,551
10-Year Total with a Discount Rate of 7%	41,551
10-Year Average	4,155
Annualized with a Discount Rate of 3%	4,871
Annualized with a Discount Rate of 7%	5,916

7. Regulatory Alternatives

The Department considered two alternatives to the proposed definition of the effectiveness in serving employers performance indicator. First, the Department considered requiring use of the Employer Penetration pilot approach, which reports the percentage of employers using services out of all employers in the State. This approach would have required counts of services provided to employers requiring States and local areas to report unique counts of employer establishments receiving services through WIOA's programs. Employer Penetration would require a more data-intensive analysis than the proposed approach of Retention with the Same Employer. Employer Penetration would have the benefit of capturing the extent to which employers within a State are engaged with WIOA-funded services and would provide State programs an incentive to work with additional employers. The Department, in an Urban Institute study, found weaknesses in this pilot approach including: (1) emphasis on quantity rather than quality or intensity of the employer service provided; (2) reliability issues associated with data entry and the process to count unique establishments; (3) measurement of program output rather than outcome; (4) potential for creation of perverse incentives to prioritize program breadth rather than depth in service and delivery; and (5) lack of sensitivity to industry sectors targeted by State and local workforce agencies.^[23]

The Department considered a second regulatory alternative that would require the use of the Repeat Business Customer approach to the effectiveness in serving employers performance indicator, which reports the percentage of employers receiving services in a year who also received services within the previous 3 years. This approach to the effectiveness in serving employers measure requires counts of services provided to employers through WIOA's programs. Repeat Business Customer requires a more data-intensive analysis than the proposed approach of Retention with the Same Employer. Repeat Business Customer captures the extent to which employers within a State can find workers and the employer's level of satisfaction with the public workforce system services. The Department, in an Urban Institute study, found weaknesses in this pilot approach including that it: (1) may provide a disincentive to reach out to new employers; (2) is subject to variation in industry and sector economic conditions; and (3) may require a statistical adjustment model to mitigate the weaknesses and improve implementation and interpretation.^[24]

The Department prefers the proposed approach of requiring the use of Retention with the Same Employer because it has data more readily available and, therefore, it is less burdensome. The Retention with the Same Employer approach better aligns with workforce system goals of matching employers with job seekers and reducing turnover without the weaknesses associated with the other two approaches to defining the effectiveness in serving employers performance indicator. In addition, because title I non-core programs are already required to report the Retention with the Same Employer measure, the two alternative measures would impose new costs to affected entities associated with collecting data, calculation of, and reporting the alternative measure.

B. Regulatory Flexibility Act, Small Business Regulatory Enforcement Fairness Act, and Executive Order 13272 (Proper Consideration of Small Entities in Agency Rulemaking)

The Regulatory Flexibility Act of 1980 (RFA), 5 U.S.C. 601 et seq., as amended by the Small Business Regulatory Enforcement Fairness Act of 1996, Public Law 104-121 (Mar. 29, 1996), requires Federal agencies engaged in rulemaking to consider the impact of their proposals on small entities, consider alternatives to minimize that impact, and solicit public comment on their analyses. The RFA requires the assessment of the impact of a regulation on a wide range of small entities, including small businesses, not-for-profit organizations, and small governmental jurisdictions. Agencies must perform a review to determine whether a proposed or final rule would have a significant economic impact on a substantial number of small entities. 5 U.S.C. 603 and 604.

The Department finds that this proposed rule would not have a significant economic impact on a substantial number of small entities. Based on this determination, the Department certifies that this proposed rule does not have a significant economic impact on a substantial number of small entities. This finding is supported, in large measure, by the fact that small entities are already receiving financial assistance under WIOA. In addition, the calculated cost of this rulemaking is a one-time per-entity cost of $73.67 associated with rule familiarization and would therefore have a de minimis impact on any on particular entity.

This proposed rule can be expected to impact small entities within the Job Corps, NFJP, and INA programs. These small entities can be, for example, Tribal or non-profit grantees, including regionally focused entities. The Department has estimated costs that are new to this proposed rule. As discussed in Section IV.A, the calculated cost of this rulemaking is a one-time per-entity cost of $73.67 associated with rule familiarization and would, therefore, have a de minimis impact on any one particular entity. Therefore, the Department certifies that this proposed rule does not have a significant economic impact on a substantial number of small entities.

C. Paperwork Reduction Act

The Department previously submitted and received OMB approval for the information collection discussed above (OMB Control Number 1205-0521) in Section I, Background and Rulemaking Authority, and Section III, Effectiveness in Serving Employers Performance Indicator for WIOA Title I Non-Core Programs. See ICR Reference Number 202104-1205-003 (OMB Control Number 1205-0521). This NPRM does not modify any of the content in the exiting OMB Control Number 1205-0521.

D. Executive Order 13132 (Federalism)

E.O. 13132 aims to guarantee the division of governmental responsibilities between the National Government and the States and to further the policies of the Unfunded Mandates Reform Act of 1995 (UMRA). Accordingly, E.O. 13132 requires executive departments and agencies to ensure that the principles of federalism guide them in the formulation and implementation of policies. Further, agencies must adhere to constitutional principles, examine the constitutional and statutory authority supporting a regulation that would limit the policymaking discretion of the States, and assess the need for such a regulation. To the extent practicable, agencies must consult State and local officials before implementing any such regulation.

E.O. 13132 further provides that agencies must implement a regulation that limits the policymaking discretion of the States only where there is constitutional and statutory authority for the regulation and it addresses a problem of national significance. For a regulation administered by the States, the National Government must grant the States the maximum administrative discretion possible to avoid intrusive Federal oversight of State administration, and agencies must adhere to special requirements for a regulation that preempts State law. E.O. 13132 also sets forth the procedures that agencies must follow for certain regulations with federalism implications, such as preparation of a summary impact statement.

Accordingly, the Department has reviewed this WIOA-required NPRM and has concluded that the rulemaking has no Federalism implications. This NPRM has no substantial direct effects on States, on the relationships between the States, or on the distribution of power and responsibilities among the various levels of government as described by E.O. 13132. Therefore, the Department has concluded that this NPRM does not have a sufficient Federalism implication to warrant the preparation of a summary impact statement.

E. Unfunded Mandates Reform Act

UMRA directs agencies to assess the effects of Federal regulatory actions on State, local, and Tribal governments, as well as the private sector. A Federal mandate is any provision in a regulation that imposes an enforceable duty upon State, local, or Tribal governments, or imposes a duty upon the private sector that is not voluntary.

Following consideration of the above factors, the Department has concluded that this NPRM contains no unfunded Federal mandates, which are defined in 2 U.S.C. 658(6) to include either a “Federal intergovernmental mandate” or a “Federal private sector mandate.” No additional burden related to reporting the effectiveness in serving employers performance indicator is being proposed to be placed on State, local, and Tribal governments, as this information already is being collected and reported on. Furthermore, the reporting is a contingent to receiving Federal program funding. Any associated reporting mandate cannot, therefore, be considered “unfunded.” Because the decision by a private training entity to participate as a provider under a WIOA core program is purely voluntary, the information collection burden does not impose a duty on the private sector that is not voluntarily assumed.

F. Executive Order 13175 (Indian Tribal Governments)

The Departments of Labor and Education reviewed this proposed rule, as well as the Joint Effectiveness in Serving Employers NPRM published concurrently with this NPRM elsewhere in this issue of the Federal Register , under the terms of E.O. 13175 and DOL's Tribal Consultation Policy (77 FR 71833 (Dec. 4, 2012)) and have determined that it would have Tribal implications, because the proposed regulations would have substantial direct effects on: one or more Indian Tribes; the relationship between the Federal Government and Indian Tribes; or the distribution of power and responsibilities between the Federal Government and Indian Tribes. Therefore, DOL has prepared a Tribal summary impact statement.

Prior to developing this proposed rule, the Department held two events to consult with INA program grantees and representatives of Tribal institutions about their experiences with the implementation and operation of the effectiveness in serving employers performance indicator. These two events consisted of a town hall meeting attended both in person and virtually and a formal consultation webinar. The town hall, entitled “Town Hall Discussion: Effectiveness in Serving Employers Performance Indicator,” occurred on September 21, 2021, at the 41st National Indian and Native American Employment and Training conference.^[25] The consultation webinar, entitled “Tribal Consultation for WIOA Effectiveness in Serving Employers Indicator Proposed Rulemaking,” occurred on October 19, 2021.^[26] At the consultation webinar, the Department provided an opportunity for stakeholders to submit written feedback through DOL's Tribal consultation email account by October 29, 2021.

At the two events, the Department received feedback from the INA community and the general public that established several areas of interest concerning the definition of the effectiveness in serving employers performance indicator for WIOA programs. These areas of interest are summarized below. The Department did not receive any written feedback through DOL's Tribal consultation email account. The Department received one letter after the consultation period that raised similar issues to those articulated at the consultation event and summarized below. This comment was not considered due to the late nature of its submission, though similar comments made during the feedback sessions were considered.

Employer, Wage, or Position Changes

Many commenters expressed concern about impacts of individuals changing employers for higher wages or different positions. Specifically, several commenters asked how the Retention with the Same Employer definition of the performance indicator would apply to individuals who have continuous employment through the second and fourth quarters, but with different employers. Some commenters expressed concern that this definition of the performance indicator would not consider individuals who advance to better employment opportunities. One commenter expressed concern that the program would be penalized if employees change employers.

Temporary, Seasonal, and Youth Employment

Many commenters expressed concern about how temporary jobs, such as seasonal or contract-based employment, would be considered. Specifically, one commenter gave an example of contractor jobs where individuals may not stay with the same employer and instead change from job to job, such as in construction. Additionally, another commenter stated that employers that regularly lay off and then rehire employees would affect outcomes.

A commenter asked if this measure applies to the INA youth program. Another commenter expressed concern about the impact on performance of limited-duration summer employment opportunities for high school students within INA youth programs. The commenter also questioned DOL's willingness to invest in developing a data collection and reporting process for INA youth programs.

Other commenters expressed concern about how seasonal jobs would be addressed and that certain areas have more seasonal employment than other areas do. Another commenter stated that individuals who participate in the program on a short-term basis while serving time with the Department of Corrections and later return to a different State may impact the performance indicator calculation. A different commenter stated that many participating employers primarily provide entry-level positions focused on gaining work experience.

Performance Indicator Calculation

Many commenters inquired about how the performance indicator is calculated. One commenter asked a question in which the sound quality of the audio was not clear. However, the subject-matter expert interpreted the question to ask if supplemental wages are considered. One commenter stated that unemployment insurance (UI) records may not capture individuals who are self-employed. Another commenter said that certain States do not have access to UI information that would enable them to calculate the performance indicator.

Many commenters suggested other ways to calculate the performance indicator. Examples provided by one commenter included employer satisfaction surveys, number of employers served, number of repeat employers, and number of job fairs coordinated with employers. Another commenter said they measure success when an employer enquires about recent graduates to fill open positions. A different commenter stated that they understood the options DOL considered for how to measure effectiveness in serving employers to include how well programs have assisted employers in hiring new employees through job fairs, work experience to full‐time hires, pre‐screening of candidates, and individual hiring events for specific employers.

Tribal Community Impacts

Some commenters had questions and comments about how the performance indicator would specifically impact INA communities. One commenter expressed the need for consideration of all Tribal communities and their unique needs. The commenter stated that measures used for all INA programs must not only satisfy the intent of the performance indicator but also be meaningful, which is part of the purpose of WIOA sec. 166. The commenter also suggested that grantees should establish a work group within the NAETC to develop information to share with Tribal leaders so that they have background and can communicate what these performance indicators would mean for INA programs.

Another commenter cited the DOL-commissioned third-party study of the performance indicator, “Measuring the Effectiveness of Service to Employers,” and questioned why some States with many INA participants were not included in the pilot study. The commenter also asked if any INA WIOA programs were included in the study. Additionally, a commenter said that DOL is seeking support from Tribes on how to measure a performance indicator they may not want.

Process Questions and Other Observations

Many commenters asked questions about the rulemaking process and how the Department decided on the proposed definition of the performance indicator. Some commenters asked if this performance indicator is required. One commenter asked if the performance indicator can be customized based on the grantee's status, for example with different requirements for rural and urban programs. A different commenter asked if DOL would decide after consultation with Tribes whether or not to apply the performance indicator to INA programs. Other commenters asked if the definition of this performance indicator would be permanent or if it would be re-evaluated in the future. Additionally, a commenter asked if they could review the draft rule with others before it is published, when the proposed rule would be published, and when the final rule would take effect.

A commenter asked if other performance indicator definitions have been submitted for consideration, for example from the NAETC. Another commenter stated that grantees with direct employer relationships differ from grantees that work with American Job Centers to facilitate employment for employers. Additionally, a commenter asked how grantees can assist participants who are facing issues at a new employment site, such as being picked on or treated unfairly, and whether it would be appropriate to act as a mediator between the employer and the participant.

Conclusion

The Department appreciates the valuable feedback received through this Tribal consultation process and has considered this feedback carefully in crafting this proposed rule and its planned implementation, such as use of the waiver process outlined in TEGL No. 04-19, “Waiver Authority for the INA Program and Implementation of Additional Indicators of Performance,” and discussed in Section III.A of this document. The Department invites and encourages submission of public comments that provide further information, including detailed recommendations for program-specific alternatives for the effectiveness in serving employers performance indicator, so that it may take this information under further consideration when making determinations regarding a final rule.

List of Subjects

20 CFR Part 684

• Employment
• Grant programs—labor
• Indians
• Reporting and recordkeeping requirements

20 CFR Part 686

• Employment
• Grant programs—labor
• Job Corps

20 CFR Part 688

• Employment
• Grant programs—labor
• Youth
• YouthBuild

For the reasons discussed in the preamble, the Department of Labor proposes to amend 20 CFR parts 684, 686, and 688 as follows:

PART 684—INDIAN AND NATIVE AMERICAN PROGRAMS UNDER TITLE I OF THE WORKFORCE INNOVATION AND OPPORTUNITY ACT

1. The authority citation for part 684 continues to read as follows:

Authority: Secs. 134, 166, 189, 503, Pub. L. 113-128, 128 Stat. 1425 (Jul. 22, 2014).

Subpart D—Supplemental Youth Services

2. Amend § 684.460 by revising paragraph (a)(6) to read as follows:

§ 684.460

What performance indicators are applicable to the supplemental youth services program?

(a) * * *

(6) The percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

* * * * *

Subpart F—Accountability for Services and Expenditures

3. Amend § 684.620 by revising paragraph (a)(6) to read as follows:

§ 684.620

What performance indicators are in place for the Indian and Native American program?

(a) * * *

(6) The percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

* * * * *

PART 686—THE JOB CORPS UNDER TITLE I OF THE WORKFORCE INNOVATION AND OPPORTUNITY ACT

4. The authority citation for part 686 continues to read as follows:

Authority: Secs. 142, 144, 146, 147, 159, 189, 503, Pub. L. 113-128, 128 Stat. 1425 (Jul. 22, 2014).

Subpart J—Performance

5. Amend § 686.1010 by revising paragraph (f) to read as follows:

§ 686.1010

What are the primary indicators of performance for Job Corps centers and the Job Corps program?

* * * * *

(f) The percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

PART 688—PROVISIONS GOVERNING THE YOUTHBUILD PROGRAM

6. The authority citation for part 688 continues to read as follows:

Authority: Secs. 171, 189, 503, Pub. L. 113-128, 128 Stat. 1425 (Jul. 22, 2014).

Subpart D—Performance Indicators

7. Amend § 688.400 by revising paragraph (f) to read as follows:

§ 688.400

What are the performance indicators for YouthBuild grants?

* * * * *

(f) The percentage of participants with wage records in the second quarter after exit who were employed by the same employer in the second and fourth quarters after exit.

* * * * *

Martin J. Walsh,

Secretary of Labor.

Footnotes

1.   Pages 2 through 5 of TEGL No. 14-18, “Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” provide the current list of DOL-administered non-core programs for which DOL has chosen to apply these performance reporting requirements, which include programs authorized by WIOA, as well as programs authorized by other Federal legislation. TEGL No. 14-18, Mar. 25, 2019, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​docn=​7611. The list of programs may change to reflect policy changes and updates to Federal legislation authorizing DOL's non-core programs.

2.  Section 116(b)(2)(A) of WIOA states the primary indicators of performance: (1) the percentage of participants who are employed during the second and (2) fourth quarters after exit from the program, (3) the median earnings of participants who are employed during the second quarter after exit, (4) the percentage of participants who obtain a recognized postsecondary credential during the program or within 1 year of exit, (5) the percentage of participants who achieve measurable skill gains during a program year, and (6) “indicators of effectiveness in serving employers.” This last indicator is the subject of this NPRM. Definitions of the others were included in the WIOA regulations promulgated in August 2016 (81 FR 55791; see20 CFR 677.155, 34 CFR 361.155, 34 CFR 463.155).

3.  ETA, TEGL No. 14-18, “Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” Mar. 25, 2019, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​docn=​7611.

4.  This joint guidance, “Performance Accountability Guidance for Workforce Innovation and Opportunity Act (WIOA) Title I, Title II, Title III, and Title IV Core Programs,” was concurrently issued on December 19, 2016, as TEGL No. 10-16 by the Department of Labor, and as Office of Career, Technical, and Adult Education Program Memorandum 17-2 and Rehabilitation Services Administration Technical Assistance Circular (TAC) TAC-17-01 by the Department of Education.

5.  S. Spaulding, et al., “Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act,” Jan. 2021, https://wdr.doleta.gov/​research/​FullText_​Documents/​ETAOP2021-17%20Measures%20of%20Effectiveness%20in%20Serving%20Employers_​Final%20Report.pdf.

6.   See id. at 3-6 (stating that validity “is used to assess whether you are measuring what you intend to measure”; that reliability “refers to the ability to maintain consistency in data collection over time and across organizations collecting the data”; that practicality means that the measure “must be relatively uncomplicated and simple to administer to avoid threats to reliability and validity” and “must be practical to use in administrating programs”; and that unintended consequences are “negative consequences or behaviors that result, like the displacement of goals or conflict with other goals.”

7.  S. Spaulding, et al., “Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act,” Jan. 2021, p. 67, https://wdr.doleta.gov/​research/​FullText_​Documents/​ETAOP2021-17%20Measures%20of%20Effectiveness%20in%20Serving%20Employers_​Final%20Report.pdf.

8.  S. Spaulding, et al., “Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act,” Jan. 2021, p. 68, https://wdr.doleta.gov/​research/​FullText_​Documents/​ETAOP2021-17%20Measures%20of%20Effectiveness%20in%20Serving%20Employers_​Final%20Report.pdf.

9.  See Joint WIOA Final Rule, 81 FR 55791, 55845-55846 (discussing the pilot and the three proposed definitions for the effectiveness in serving employers performance indicator); ETA, TEGL No. 10-16, “Performance Accountability Guidance for Workforce Innovation and Opportunity Act (WIOA) Title I, Title II, Title III, and Title IV Core Programs,” Dec. 19, 2016, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​DOCN=​8226;​ ETA, TEGL No. 14-18, “Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” Mar. 25, 2019, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​docn=​7611 (referring the title I non-core programs to TEGL No. 10-16 for a description of the pilot).

10.  ETA, Training and Employment Notice (TEN) No. 08-06, “Implementation of an Integrated Performance Reporting System for Multiple Employment and Training Administration (ETA) and Veterans' Employment and Training Service (VETS) Administered Programs,” Aug. 24, 2016, https://wdr.doleta.gov/​directives/​attach/​TEN/​TEN_​08-16.pdf; ETA, TEN 40-16, “ Workforce Integrated Performance System (WIPS) User Resource Library Information Page,” Apr. 11, 2017, https://wdr.doleta.gov/​directives/​attach/​TEN/​TEN_​40-16_​Acc.pdf.; ETA, TEGL No. 14-18, “Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” Mar. 25, 2019, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​docn=​7611.

11.  ETA, “Workforce Integrated Performance System (WIPS),” https://www.dol.gov/​agencies/​eta/​performance/​wips (last visited Jan. 9, 2022).

12.  Specifically, the programs are required to report the wage records or supplemental wage information, as directed in program-specific guidance, which are used to identify whether a program participant's employer wage record indicates a match of the same establishment identifier ( e.g., Federal Employer Identification Number or State tax identifier) in the second and fourth quarters after exit from the program.

13.   ETA, TEGL No. 14-18, “ Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” p. 8, Mar. 25, 2019, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​docn=​7611.

14.  ETA, TEN No. 8-16, “Implementation of an Integrated Performance Reporting System for Multiple Employment and Training Administration (ETA) and Veterans' Employment and Training Service (VETS) Administered Programs,” Aug. 24, 2016.

15.  The first event was a town hall discussion on September 21, 2021. See NAETC, “41st National Indian and Native American Employment and Training Program,” Sept. 20-23, 2021, http://www.ninaetc.net/​41%20NINAETC%20PROGRAM_​FINAL.pdf. The second event, a consultation webinar, occurred on October 19, 2021. See “Tribal Consultation; Workforce Innovation and Opportunity Act, Implementation of the Effectiveness in Serving Employers Performance Indicator; Notice of Tribal Consultation; Virtual Meeting,” 86 FR 54244 (Sept. 30, 2021).

16.  ETA, TEGL No. 04-19, “Waiver Authority for the INA Program and Implementation of Additional Indicators of Performance,” Aug. 29, 2019 , https://wdr.doleta.gov/​directives/​attach/​TEGL/​TEGL_​4-19_​acc.pdf.

17.  The 216 YouthBuild entities consist of grantees within each of the three currently active grant classes (67 grantees in the 2020 class, 68 grantees in the 2019 class, and 81 grantees in the 2018 class).

18.  ETA, TEGL No. 14-18, “Aligning Performance Accountability Reporting, Definitions, and Policies Across Workforce Employment and Training Programs Administered by the U.S. Department of Labor (DOL),” Mar. 25, 2019, https://wdr.doleta.gov/​directives/​corr_​doc.cfm?​docn=​7611.

19.  BLS, “May 2020 National Industry-Specific Occupational Employment and Wage Estimates: NAICS 999300—Local Government, excluding schools and hospitals (OEWS Designation),” https://www.bls.gov/​oes/​current/​naics4_​999300.htm (last visited Jan. 9, 2022).

20.  U.S. Environmental Protection Agency, “Wage Rates for Economic Analyses of the Toxics Release Inventory Program,” June 10, 2002, https://www.regulations.gov/​document/​EPA-HQ-OPPT-2018-0321-0046.

21.  BLS, “Employer Costs for Employee Compensation—March 2021,” June 17, 2021, Calculated using Table 1. Employer Costs for Employee Compensation by ownership, https://www.bls.gov/​news.release/​archives/​ecec_​06172021.htm.

22.  S. Spaulding, et al., “Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act (Research Report),” Jan. 2021, https://www.urban.org/​sites/​default/​files/​publication/​104160/​measuring-the-effectiveness-of-services-to-employers_​1_​0.pdf.

23.  S. Spaulding, et al., “Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act (Research Report),” Jan. 2021, https://www.urban.org/​sites/​default/​files/​publication/​104160/​measuring-the-effectiveness-of-services-to-employers_​1_​0.pdf.

24.  S. Spaulding, et al., “Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act,” Jan. 2021, https://wdr.doleta.gov/​research/​FullText_​Documents/​ETAOP2021-17%20Measures%20of%20Effectiveness%20in%20Serving%20Employers_​Final%20Report.pdf.

25.  NAETC, “41st National Indian and Native American Employment and Training Program,” Sept. 20-23, 2021, http://www.ninaetc.net/​41%20NINAETC%20PROGRAM_​FINAL.pdf.

26.  DOL, “Tribal Consultation for WIOA Effectiveness in Serving Employers Indicator Proposed Rulemaking,” https://www.workforcegps.org/​events/​2021/​09/​14/​13/​57/​Tribal-Consultation-for-WIOA-Effectiveness-in-Serving-Employers-Indicator-Proposed-Rulemaking (last updated Nov. 3, 2021); see also “Tribal Consultation; Workforce Innovation and Opportunity Act, Implementation of the Effectiveness in Serving Employers Performance Indicator; Notice of Tribal Consultation; Virtual Meeting,” 86 FR 54244 (Sept. 30, 2021).