[Federal Register Volume 64, Number 150 (Thursday, August 5, 1999)]
[Notices]
[Pages 42818-42822]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 99-20119]
[[Page 42817]]
_______________________________________________________________________
Part VI
Department of Labor
_______________________________________________________________________
Employment and Training Administration
_______________________________________________________________________
Consultation Papers on Performance Accountability Under Title I of the
Workforce Investment Act of 1998 (WIA); Notice
Federal Register / Vol. 64, No. 150 / Thursday, August 5, 1999 /
Notices
[[Page 42818]]
DEPARTMENT OF LABOR
Employment and Training Administration, Labor
Consultation Papers on Performance Accountability Under Title I
of the Workforce Investment Act of 1998 (WIA)
AGENCY: Employment and Training Administration, Labor.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: The purpose of this notice is to disseminate consultation
papers for interested parties on the performance accountability system
for title I of the Workforce Investment Act. There are two papers. The
first paper provides a framework regarding the approach and processes
for continuous improvement under title I of the Workforce Investment
Act of 1998. The second paper provides a framework for the approach and
processes for customer satisfaction measures under title I of the
Workforce Investment Act. Interested parties have 30 days to provide
comments on these papers.
FOR FURTHER INFORMATION CONTACT: Mr. Eric Johnson, Workforce
Implementation Taskforce Office, U.S. Department of Labor, 200
Constitution Avenue, NW, Room S-5513, Washington, D.C., Telephone:
(202) 219-0316 (voice) (This is not a toll free number), or 1-800-326-
2577 (TDD). Information may also be found, or comments provided, at the
website--http://usworkforce.org.
SUPPLEMENTARY INFORMATION: The Workforce Investment Act (WIA or Act)
Pub. L. 105-220 (August 7, 1998) provides the framework for a reformed
national workforce investment system designed to meet the needs of the
nation's employers, job seekers and those who want to further their
careers. One of the key reforms contained in the Act is the
establishment of a comprehensive accountability system to assess the
effectiveness of State and local areas in providing employment and
training services. The Act requires:
A focus on results defined by core indicators of
performance;
Measures of customer satisfaction with programs and
services;
A strong emphasis on continuous improvement;
Annual performance levels developed as a result of
negotiations among Federal, State and local partners;
Incentive awards and financial sanctions based on State
performance; and
Reporting and dissemination of performance results.
The two papers contained in this notice focus on two of these
requirements--continuous improvement and customer satisfaction.
The Department is approaching the development of this new
performance accountability system on two tracks. First, definitions of
the core measures of performance and temporary reporting instructions
have been developed and disseminated for those States who are
implementing WIA in Program Year (PY) 1999. Second, the Department is
working with States and local governments to develop definitions and
reporting requirements for use in PY 2000 and beyond. Part of this
process will include using the lessons learned from the early
implementing States and working with the Department of Education and
other Federal agencies to develop common definitions for performance
measures across programs. In general, the Department is considering PY
1999 to be a transition year. The comments received on these two papers
will be used in developing the performance accountability system for PY
2000 and beyond.
Comments are solicited on the overall framework and approaches
being proposed for customer satisfaction and continuous improvement
under title I of WIA.
Signed at Washington, D.C., this 30th day of July 1999.
Raymond L. Bramucci,
Assistant Secretary of Labor, Employment and Training Administration.
Attachment 1--Continuous Improvement Under Title I of the Workforce
Investment Act of 1998
I. Introduction
A. Legal Framework
The Workforce Investment Act of 1998 envisions a high performance
workforce investment system in this country--a system that is customer-
driven, results-oriented, flexible, and continuously improving. The
Act's purpose is clearly stated as: To provide workforce investment
activities that increase participants' employment, retention, earnings,
and skill attainment and as a result:
Improve the quality of the workforce;
Reduce welfare dependency; and
Enhance the productivity and competitiveness of the
nation.
The Act envisions a workforce investment system that strives for
high performance rather than settling for compliance levels of
performance, and that delivers unparalleled levels of services to
customers--job seekers, workers, and employers. Although WIA has
numerous references to continuous improvement, this consultation paper
focuses on three major provisions contained in Section 136 of the
legislation:
A comprehensive performance accountability system will
include an assessment of the effectiveness of state and local areas in
achieving continuous improvement of workforce investment activities.
Section 136(a).
The Governor/Secretary agreement on State adjusted levels
of performance must take into account the extent to which those levels
promote continuous improvement in performance. Section
136(b)(3)(A)(iv)(III).
States must conduct ongoing evaluations of workforce
investment activities to promote and implement methods for continuously
improving them. Section 136(e)(1).
B. Guiding Principles
The U.S. Department of Labor (DOL) is using the following as
guiding principles in designing a system-wide approach for continuous
improvement.
DOL's role in continuous improvement is primarily one of
leadership carried out through an effective technical assistance
effort.
For the workforce investment system to strive toward
performance excellence, continuous improvement practices must be
embraced at all levels--local, State, and DOL Regional and National
Offices.
DOL will integrate existing quality initiatives to drive
continuous improvement through a technical assistance strategy that
includes award and recognition efforts, access to information on best
practices, and the availability of a variety of tools.
The Malcolm Baldrige Criteria for Performance Excellence
will be used as the framework for continuously improving performance in
the workforce investment system.
C. Malcolm Baldrige Criteria
The Malcolm Baldrige Criteria for Performance Excellence and the
Baldrige Scoring Guidelines are proposed as the framework for enabling
organizations within the workforce investment system to advance toward
high performance. This framework is widely accepted as the standard for
defining performance excellence in public and private organizations.
The Criteria and Scoring Guidelines are excellent diagnostic
instruments that can help leaders identify organizational strengths and
key areas for improvement and work to achieve higher levels of
performance. DOL will provide resources and technical assistance to
[[Page 42819]]
state and local organizations that are interested in using the Baldrige
Criteria to help improve performance. The following is excerpted from
the 1999 ``Criteria for Performance Excellence,'' and includes for
informational purposes the relative point value assigned to each
category:
------------------------------------------------------------------------
Point
Award category value
------------------------------------------------------------------------
Leadership................................................... 125
Strategic Planning........................................... 85
Customer and Market Focus.................................... 85
Information and Analysis..................................... 85
Human Resource Focus......................................... 85
Process Management........................................... 85
Business Results............................................. 450
----------
Total.................................................. 1,000
------------------------------------------------------------------------
II. Approach to Continuous Improvement
A. Overview of the Approach
According to leading Baldrige experts, continuous improvement is
the systematic and ongoing improvement of products, programs, services
and processes by small increments and major breakthroughs. Continuous
improvement is the process of building dynamic, high achieving systems
within every organization, and becomes embedded in the way the
organization conducts its daily activities.
DOL's role in the continuous improvement process is primarily based
on providing leadership and technical assistance. In striving to
improve performance as measured by the performance and customer
satisfaction indicators, states and localities will need resources,
information and technical assistance to help them continuously improve
organizational effectiveness. The approach to continuous improvement
proposed in this consultation paper envisions that DOL will play a
strong, proactive role in providing States and localities with
information, resources, tools, training and technical assistance to
help them enhance their performance. DOL will also apply these tools to
continuously improve the effectiveness of ETA National and Regional
Offices.
DOL's Continuous Improvement Strategy is aimed at improving
outcomes for the customers of the workforce investment system by
enhancing system-wide performance. The objectives of the strategy are
to:
Effectively align system-wide resources to achieve
performance excellence.
Recognize and award top performers within the system.
Provide organizations and individuals with learning
opportunities to acquire the skills needed to operate in a high
performance mode.
B. Continuous Improvement in State Workforce Investment Plans
A rigorous approach to continuous improvement must be applied at
all levels of the workforce investment system in order for that system
to achieve the high levels of performance envisioned in the Workforce
Investment Act. For States to develop a Statewide workforce investment
system that incorporates a rigorous approach to continuous improvement,
each State needs to start with a snapshot or baseline of its system
capacity--its ``as is'' capacity at the point in time when the State
Plan is developed. Ideally, States would establish both an ``as is''
state for each organization's capability to become a high performance
organization (organizational effectiveness), as well as the ``as is''
state for each organization's current program results and outcomes.
In the spirit of partnership and shared accountability, State
officials and DOL officials would have this data before them as the
basis for establishing the baseline. From the State's perspective, the
baseline or starting point for continuous improvement is simply defined
as, ``where you are now.'' (This process has been further defined in
the consultation paper on Reaching Agreement on State Adjusted Levels
of Performance.)
The State's continuous improvement strategy becomes its approach
for closing the gap between the current ``as is'' capacity and a time-
sensitive ``desired state'' set forth in the State's plan. This
approach addresses both the voluntary ``organizational improvement''
strategy and the more traditional compliance-oriented strategies for
meeting minimum WIA specified performance measures. This offers states
the opportunity to propose a rigorous and comprehensive approach to
continuous improvement--one that establishes an effort to develop and
improve organizational capacity (systems and processes) thus enabling
committed organizations to deliver high performance, customer-focused
services, as well as meeting all other requirements of the Act.
C. Voluntary Approach to Assessment and Benchmarking
DOL's role is to make available to States the resources, tools and
services that will help them advance toward high performance through a
rigorous continuous improvement strategy. The basic tools and services
would include organizational assessment tools, resources to aid in the
development of improvement plans, best practices, and benchmarking for
continuous improvement services. ``Benchmarking'' is the use of
information and data on processes and results that represent best
practices and the highest levels of performance.
As part of its continuous improvement strategy, DOL would gather
and make easily accessible to States and local organization benchmarks
of the highest levels of performance both in processes and results
within the workforce investment system, and for similar processes and
results for organizations outside the system. Benchmarks represent the
very essence of high performance business practices--comparing your
organization to the very best in class and striving continuously to
attain that level of performance. It is a voluntary practice carried
out by the best organizations as a fundamental component of their
continuous improvement strategy.
D. Supporting Continuous Improvement Activities
Under WIA, States are to ensure that the principle of continuous
improvement is embedded in Statewide workforce investment activities.
Again, this would represent the regimen for achieving the systematic
and ongoing improvement of workforce investment programs, services, and
processes by small increments and major breakthroughs. This continuous
improvement regimen will foster enhancements in performance levels
desired by each level of the system.
The State's Workforce Investment Plan must include a description of
the State's strategy for developing and operating this continuous
improvement approach. While each State has latitude to use a range of
resources, tools and approaches for accomplishing this, the States are
encouraged to work with the Employment and Training Administration
(ETA) Regional Office to take advantage of resources available from or
brokered through ETA's Continuous Improvement Strategy.
Generally, DOL is seeking comment on the following strategy to
support the local, State, Regional and National organizations in
continuous improvement--
Establish a system of organizational and individual
learning to acquire skills needed to support high performance within
the workforce investment system.
Utilize the Malcolm Baldrige Criteria for Performance
Excellence as a
[[Page 42820]]
proven and rigorous methodology to transform local, State, and National
workforce investment organizations.
Establish an award and recognition system in support of
high performing organizations at all levels.
Work closely with early implementing States as partners to
begin the system-wide transformation process toward performance
excellence.
Provide local and State organizations, Regional Offices
and National Office with easily accessible information on benchmarks
and best practices, as well as affordable and effective assessment
tools.
Attachment II: Customer Satisfaction Under Title I of the Workforce
Investment Act of 1998
I. Introduction
A. Legal Framework
In addition to the core measures, the Workforce Investment Act of
1998 [WIA Section 136(b)(2)(B)] states that ``the customer satisfaction
indicator of performance shall consist of customer satisfaction of
employers and participants with services received from the workforce
investment activities authorized under this subtitle.'' The Act
[Section 136(b)(3)(A)(i)] also requires that there be State-adjusted
levels of performance for customer satisfaction and that ``the levels
of performance established * * * shall, at a minimum--
(1) Be expressed in an objective, quantifiable, and measurable
form; and
(2) Show the progress of the State toward continuously improving in
performance.''
WIA draws a clear link between the core indicators of performance
and customer satisfaction. The levels of performance attained for the
core indicators must ``assist the State in attaining a high level of
customer satisfaction'' (WIA Section 136(b)(3)(a)(iv)(I)). WIA further
states that ``customer satisfaction may be measured through surveys
conducted after the conclusion of participation in workforce investment
activities'' (WIA Section 136(b)(2)(B)).
Effective high performance organizations listen to their customers
and build their organization around meeting their customers'
expectations. Determining a customer's expectations and satisfaction is
an integral part of a continuous improvement strategy. Under the
Workforce Investment Act, customer satisfaction is both a process of
identifying and listening to customers, as well as an outcome for
measuring program success.
WIA emphasizes the importance of a customer-driven workforce system
by including customer satisfaction as a required measure, along with
the core indicators of performance. Customer satisfaction measures
provide feedback to supervisors and staff about how their actions
affect customers, giving them critical information to motivate and
guide continuous improvement. Customer satisfaction feedback also sends
a clear message to staff, management, and customers that customers
matter.
B. Guiding Principles
DOL is using the following guiding principles in designing a
system-wide approach for measurement of customer satisfaction:
Customer satisfaction is the foundation of an
organization's strategy for continuous improvement.
Customer satisfaction should be measured after completion
of the service and should be quantifiable.
Customer satisfaction surveys need to contain a set of
required questions to form a customer satisfaction indicator.
Comparability is an important element in negotiating
customer satisfaction performance levels and in providing opportunities
for benchmarking and sharing best practices.
States and local organizations are encouraged to add
customized questions to inform their efforts to align resources or
redesign processes to achieve better results.
II. Approach
A. Overview of the Approach
The Act, in requiring a customer satisfaction indicator for
employers and participants, presents a general framework for developing
a national approach. Customer satisfaction indicators are a specific
part of the performance accountability system and are the foundation of
an organization's strategy for continuous improvement. They provide a
guide to achieving the vision and goals of the Act, and provide a
focused and structured process for listening to and learning from
customers.
To meet the customer satisfaction requirements for Title I, DOL
proposes the use of customer satisfaction surveys. There are two
purposes for surveying customers. The first is to produce an outcome
measure for each State as part of the performance accountability
system. This will be accomplished by a small set of required questions
that will form a customer satisfaction index. The second purpose is to
gain customer feedback to help in improving processes and services.
This will be accomplished through a set of recommended questions
addressing each service component and any additional questions that the
State and local areas choose to ask, depending on their particular
needs and service mix.
DOL will provide guidelines for collecting customer satisfaction
data that will lay out the strategy and standards (e.g., sample size,
response rate) for implementing the survey while providing as much
flexibility for the states and localities as possible. The survey will
contain the required questions that form the indicator. In addition, to
cover many of the most commonly delivered services, the guidelines will
suggest sets of questions that States may choose to use along with the
indicator questions. The advantage of using these questions will be
that they provide additional opportunities for benchmarking and
learning from the best practices of others.
B. Proposed Customer Satisfaction Strategy
Consistent with the Workforce Investment Act, measures of customer
satisfaction:
Must address participants and employers;
Must be quantifiable;
Must be able to track progress toward improvement;
Must be comparable across states;
May be measured at the conclusion of participation; and
Must promote continuous improvement in performance along
with the core measures.
The Act calls for assessment of two customer categories: (1)
participants, and (2) employers. Consistent with the approach taken for
core measures, two options are presented. The first option is to report
the participant indicator for each of the four groups:
Adults
Dislocated Workers
Youth 19-21 served with youth funds, and
Youth 14-18.
The second option is to aggregate the four groups to provide a
single indicator of participant satisfaction.
The advantages of reporting each of the four groups separately are
to:
1. Allow for a more comprehensive analysis of results. An analysis
by group will provide an assessment of the degree to which core
indicator performance contributes to customer satisfaction.
2. Allow program managers to evaluate the degree to which they are
satisfying different customer segments.
The advantage of the second option is that it will simplify
customer surveying
[[Page 42821]]
and reporting, and will emphasize high expectations for all of the
groups. It should be noted that, under both options, the four groups
identified above would also include those participating in incumbent
worker training. Customer surveying for other services that are not
covered under Title I would be at the discretion of other one-stop
partner programs.
For employers, it is proposed that services to employers be grouped
into the following three service categories: (1) informational, (2)
labor exchange, and (3) special services such as rapid response.
Capturing customer satisfaction within each of these three categories
will allow a clearer picture of service to employers and is one way to
expand the system's ability to be accountable for services to a
significant customer base. While States would not be required to report
the three customer indicators for employers at a National level, they
may utilize this method as a way to better understand their employer
customers.
C. Collecting Customer Satisfaction Information
There are a number of different methods to collect customer
satisfaction information.
The simplest approach is to train staff to listen to the
customers they serve and to ask questions that elicit customer needs
while they are providing service.
Focus groups and group interviews are another strategy.
A trained manager or staff person can circulate in the
resource center where people are waiting and ask questions informally
to gain a better understanding of customer needs and concerns.
Suggestion boxes are also a way of gathering information.
Telephone surveys of customers are used to gather specific
information.
To meet the WIA customer satisfaction requirements for Title I, the
method proposed in this paper is customer satisfaction surveys. This is
the most effective method that allows state and national aggregation of
comparable, quantifiable data.
As part of a comprehensive continuous improvement strategy,
organizations will use a combination of strategies in addition to the
proposed surveys, since each serves a somewhat different purpose and
provides different types of information.
D. Proposed Measures
The customer satisfaction indicator will be derived from surveys
that must have a minimum set of common questions asked in a common
format to assure comparability. These common questions are used to form
an index, which is a single score. An index has the advantage of
addressing different dimensions of the customer's experience, and is
more reliable than a single question. The creation of an index provides
a proven methodology to capture common customer satisfaction
information across programs and organizations that can be aggregated to
a State and National level. The responses of the embedded questions
will be rolled up to the State level and reported annually at a
specified time. This approach will continue to be modified as the
Department receives feedback and validation through consultation with
the workforce investment system.
Satisfaction for all customers in all service categories will be
measured through a set of 3-5 questions that together form the
indicator. We propose that the surveys include these three questions:
``Overall, how satisfied were you with the services
received?'' (Ranging from 1--Very Dissatisfied to 10--Very Satisfied)
``How likely would you be to refer others to these
services?'' (Ranging from 1--Not Very Likely to 10--Very Likely)
``If you were in a similar situation again, how likely
would you be to use these services?'' (Ranging from 1--Not Very Likely
to 10--Very Likely)
The above questions provide an indicator sensitive enough to record
change but less prone to random fluctuations common to indicators that
are composed of a single question. [This protects States from being
sanctioned when random error depresses the indicator's performance
level and prevents states from being rewarded for high performance
resulting solely from random error.] The satisfaction score will be
reported on a 0-100 scale. To simplify reporting to the Federal level,
scores for each service category can be aggregated into two
satisfaction indices, one for participants and one for employers.
E. Comparability Across States
Comparability is important for several reasons. First, customer
satisfaction performance levels are negotiated along with the core
measures. One of the factors affecting those negotiations are ``how the
levels compare with state adjusted levels of performance established
for other States * * *.''
Comparability also provides for fairness in determining incentives
and sanctions. Additionally, comparability contributes to continuous
improvement across the system. Having comparable measures will allow
benchmarks to be developed to promote continuous improvement.
Comparability will also facilitate the sharing of best practices within
and among the States.
F. When To Measure
Consistent with WIA, it is proposed that customer satisfaction be
measured at completion of the service. For continuous improvement
purposes, it is particularly important to measure customer satisfaction
as close to the point of service for the following reasons:
The immediacy of a person's impression makes a significant
difference in terms of what he/she will remember;
The highest response rate is obtained at point of service;
Due to the time delay to track outcome-related data (e.g.,
the core indicators), this immediate customer feedback provides much
needed real time data for staff and program managers.
The point in time will vary based on the type of customer and level
of service received.
Participant Customers
For self-help/information and core services, the survey will be
conducted at the point of contact, immediately after the service is
provided. For intensive and training services, the participant will be
surveyed after the completion of services (this does not mean
necessarily that they have ``exited'' or been ``terminated'' from a
program). Additional surveying may be conducted as part of follow-up to
determine other aspects of satisfaction. Such surveys are proposed to
be optional, given the additional reporting burden they would create.
Employer Customers
For informational services, the survey will be conducted at the
point of contact, immediately after the service is provided. For labor
exchange and special services, the employer will be surveyed after the
completion of services.
G. Using Customer Satisfaction in a Continuous Improvement Process
The customer satisfaction indicators, in addition to being a
specific part of the performance accountability system, are also the
foundation of an organization's strategy for continuous improvement.
The indicators provide a guide to achieving the vision and goals of the
Act. Additional questions of local importance to customers, program
[[Page 42822]]
operators and service providers deepen the understanding of how to
reach these goals.
By adding customized questions, organizations can use customer
satisfaction as part of an integrated continuous improvement approach.
They can determine where to focus more resources, or redesign programs
or sequences of services in order to achieve better results. This use
of customer satisfaction will not be federally mandated in order to
maintain local flexibility, and to recognize differing approaches in
program designs that vary depending upon the service mix and each
area's economic and demographic conditions.
H. Definition of Measures
Measurement of Participant Customers
The degree to which participant customers are satisfied with the
core, intensive and training services provided by the workforce
investment system.
Measurement of Employer Customers
The degree to which employer customers are satisfied with the
informational, labor exchange, and special services provided by the
workforce investment system.
I. Pilot Testing
DOL will work with a number of pilot sites to better determine the
range of customer satisfaction levels (i.e., baseline data), and to
explore technical issues of survey timing, methodology, and
questionnaire construction. The sites will be selected based on
interest and previous experience with customer satisfaction surveys.
DOL will use the results of the pilot testing and the feedback from
this consultation paper to issue guidance or technical standards for
the survey methodology.
[FR Doc. 99-20119 Filed 8-4-99; 8:45 am]
BILLING CODE 4510-30-P
