2011-18355. Notice of Random Assignment Study To Evaluate Workforce Investment Act Adult and Dislocated Worker Programs; Request for Comment

AGENCY:

Employment and Training Administration (ETA), Labor.

ACTION:

Notice.

SUMMARY:

The Department of Labor (DOL or the Department) is prepared to conduct an evaluation to provide rigorous, nationally representative estimates of the net impacts of intensive services and training provided under the Workforce Investment Act (WIA) Adult and Dislocated Worker Programs. The Department has determined that it is in the public interest to use a random assignment impact methodology for the study. In the local workforce investment areas (LWIAs) randomly selected to participate in this evaluation, all applicants for intensive services and training under the WIA Adult and Dislocated Worker programs will be required to participate in the study during a 12-18 month period. The Department is soliciting comments concerning the Department's plan to carry out the study.

DATES:

Written comments on the plan to require consent to participate in the study during the designated LWIAs' study enrollment periods must be received by the office listed in the addresses section below on or before August 4, 2011.

ADDRESSES:

You may submit comments by any one of the following methods:

• Mail or Hand Delivery/Courier: Please submit all written comments (including disk and CD-ROM submissions) to Eileen Pederson, U.S. Department of Labor, Employment and Training Administration, Office of Policy Development and Research, 200 Constitution Avenue, NW., Frances Perkins Bldg., Room N-5641, Washington, DC 20210. Commenters are advised that mail delivery in the Washington area may be delayed due to security concerns. Hand-delivered comments will be received at the above address. All overnight mail will be considered to be hand-delivered and must be received at the designated place by the date specified above.
• Facsimile: Please send comments to Eileen Pederson's attention, at fax number [phone].
• E-mail: Please send comments to [e-mail].

Please submit your comments by only one method. The Department will not review comments received by means other than those listed above or that are received after the comment period has closed.

Comments: All comments on this notice will be retained by the Department and released upon request via e-mail to any member of the public. The Department also will make all the comments it received available for public inspection by appointment during normal business hours at the above address. If you need assistance to review the comments, the Department will provide you with appropriate aids such as readers or print magnifiers. The Department will make copies of this notice available, upon request, in large print, Braille and electronic file on computer disk. The Department will consider providing the notice in other formats upon request. To schedule an appointment to review the comments and/or obtain the notice in an alternative format, contact the Office of Policy Development and Research at [phone] (this is not a toll-free number). You may also contact this office at the address listed above.

The Department will retain all comments received without making any changes to the comments, including any personal information provided. If requested, the comments will be released to the public. The Department cautions commenters not to include their personal information such as Social Security Numbers, personal addresses, telephone numbers, and e-mail addresses in their comments as such submitted information will be released with the comment if the comments are requested. It is the commenter's responsibility to safeguard his or her information. If the comment is submitted by e-mail, the e-mail addresses of the commenter will not be released.

FOR FURTHER INFORMATION CONTACT:

Eileen Pederson, U.S. Department of Labor, Employment and Training Administration, Office of Policy Development and Research, 200 Constitution Avenue, NW., Room N-5641, Washington, DC 20210. Telephone number: [phone] (this is not a toll-free number). Individuals with hearing or speech impairments may access the telephone number above via TTY by calling the toll-free Federal Information Relay Service at 1-877-889-5627 (TTY/TDD).

I. Background

The Workforce Investment Act of 1998 (WIA) brought formerly fragmented public and private employment services together in a single location within each community and made them accessible to a wider population than did prior employment and training service delivery systems. The recent recession, high unemployment rate and limited Federal resources serve as a reminder of the importance of ensuring that the services provided to people who are out of work and desiring to transition to new employment are as effective as possible. In order to improve the management and effectiveness of WIA services and related activities, section 172 of the WIA requires the Department to continually evaluate WIA programs and activities. These evaluations must “utilize appropriate methodology and research designs, including the use of control groups chosen by scientific random assignment methodologies.” Congress, the Office of Management and Budget, and the Government Accountability Office have called on DOL to conduct an evaluation now in order to learn if WIA-funded intensive services and training are as effective as they can be. Accordingly, ETA is conducting a multisite control group evaluation to provide rigorous, nationally representative estimates of the net impacts of WIA intensive services and training provided under the Adult and Dislocated Worker Programs.

Generally speaking, intensive services are services that involve staff assistance and include assessments, counseling, and job placement. Training includes education and occupational skills building. This evaluation will offer policymakers, program administrators, and service providers information about the relative effectiveness of Adult and Dislocated Worker intensive services and training, how the effectiveness varies by target population, and how the services and training are implemented. The study will also produce estimates of the benefits and costs of these services and training. The study's key goal is to generate findings that are applicable to the national WIA Adult and Dislocated Worker programs.

To obtain rigorous, nationally representative estimates of WIA's effectiveness for adults and dislocated workers, the Department plans to use a random assignment impact methodology for the evaluation. The evaluation will take place in approximately 30 randomly selected LWIAs. WIA applicants in the selected LWIAs who are eligible for intensive services will be randomly assigned to one of three groups. The three research groups to which they will be assigned are: (1) The full-WIA group—adults and dislocated workers in this group can receive any WIA services and training for which they are eligible, (2) the core-and-intensive group—adults and dislocated workers in this group can receive any WIA services for which they are eligible other than training, and (3) the core-only group—adults and dislocated workers in this group can receive only WIA core services but no intensive services or training. Overall, 94 percent of all WIA applicants in the participating LWIAs who are eligible for and interested in intensive services or training will be assigned to the full-WIA group.

In the LWIAs randomly selected to participate in the evaluation, all applicants for intensive services and/or training under the WIA Adult and Dislocated Worker programs will be asked to participate in the study during the 12-18 month study enrollment period. They will be informed of the evaluation, provided an opportunity to ask questions or seek clarification of their role and responsibilities should they agree to participate, and then required to give their consent to participate. Applicants who do not consent to participate in the study will be allowed to receive core services only. The sample intake period will range between 12 and 18 months at each site. A total of about 68,000 WIA adult and dislocated worker program applicants will be randomly assigned to the evaluation.

The Department has determined that it is in the public interest to use a random assignment impact methodology. Random assignment is generally viewed as the best and most feasible design for credibly and reliably answering questions about the effectiveness of social programs and policy interventions. This is because when implemented carefully, random assignment creates groups that are, on average, identical in their characteristics before the intervention. Hence, any differences in the employment outcomes of customers in the three research groups can be confidently attributed to differences in the service intervention. Moreover, because of funding limitations, not all people who are eligible for, and could benefit from, WIA services can receive them. As a result, the total number of people who are served will not be affected by the study.

The Department recognizes that this design will assign some applicants to groups that limit their access to WIA services. However, the study was designed to balance two objectives: (1) To fulfill the mandate for a rigorous evaluation of WIA and (2) to maximize the number of customers in the study who have access to the full set of WIA-funded services. Since the three research groups will be identical except for their ability to access different levels of WIA-funded assistance, any differences in outcomes between the groups will be attributable to the WIA services. To meet the second objective, the study design allows most adult and dislocated worker customers to have access to the full set of WIA-funded assistance. Only a small percentage of customers will be restricted to receiving core services or core-and-intensive services. Those customers who are assigned to either the core only or core-and-intensive only research groups will be eligible to apply for intensive services and training 15 months after enrollment into the study.

To protect the rights and welfare of One-Stop Career Center customers who agree to participate in the evaluation, the evaluation team, consisting of researchers from Mathematica Policy Research and MDRC, submitted the WIA Evaluation design to MDRC's Institutional Review Board (IRB) for concurrence. On June 17, 2010, the IRB determined this study to be of minimal risk and unanimously approved it.

II. Desired Focus of Comments

Currently, DOL is soliciting comments concerning the Department's intent to carry out the random assignment study described above: for the limited enrollment period, applicants for WIA intensive and training services would be required to consent to participate in the study, where they would be randomly assigned to one of the three research groups. Applicants who do not consent to participate would receive core services. This requirement would apply only to applicants for intensive services and training provided in the limited number of LWIAs selected to participate in this evaluation.

The Department seeks comments focused on whether there is a methodology that would yield as credible and reliable an evaluation of the WIA program as random assignment, but avoids adverse affect on the study participants. The Department also welcomes comments that suggest ways to more effectively minimize any adverse impact on the study participants who participate in the study described above.

III. Current Actions

Following receipt of comments in response to this request, ETA will adjust, as appropriate, the approach for temporarily requiring applicants for WIA intensive services and training at selected LWIAs to participate in random assignment. Comments submitted in response to this request will also become a matter of public record.

Signed: at Washington, DC this 15th day of July, 2011.

Jane Oates,

Assistant Secretary, Employment and Training Administration.