2012-12168. Agency Information Collection Activities: Proposed Collection; Comment Request

AGENCY:

Agency for Healthcare Research and Quality, HHS.

ACTION:

Notice.

SUMMARY:

This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Workflow Assessment for Health IT Toolkit Evaluation.” In accordance with the Paperwork Reduction Act, 44 U.S.C. 3501-3521, AHRQ invites the public to comment on this proposed information collection.

This proposed information collection was previously published in the Federal Register on March 9th, 2012 and allowed 60 days for public comment. One comment was received. The purpose of this notice is to allow an additional 30 days for public comment.

DATES:

Comments on this notice must be received by June 20, 2012.

ADDRESSES:

Written comments should be submitted to: AHRQ's OMB Desk Officer by fax at [phone] (attention: AHRQ's desk officer) or by email at [e-mail] (attention: AHRQ's desk officer).

Copies of the proposed collection plans, data collection instruments, and specific details on the estimated burden can be obtained from the AHRQ Reports Clearance Officer.

FOR FURTHER INFORMATION CONTACT:

Doris Lefkowitz, AHRQ Reports Clearance Officer, [phone], or by email at [e-mail].

SUPPLEMENTARY INFORMATION:

Proposed Project

Workflow Assessment for Health IT Toolkit Evaluation

AHRQ is a lead Federal agency in developing and disseminating evidence and evidence-based tools on how health IT can improve health care quality, safety, efficiency, and effectiveness. Understanding clinical work practices and how they will be affected by practice innovations such as implementing health IT has become a central focus of health IT research. While much of the attention of health IT research and development had been directed at the technical issues of building and deploying health IT systems, there is growing consensus that deployment of health IT has often had disappointing results, and while technical challenges remain, there is a need for greater attention to sociotechnical issues and the problems of modeling workflow.

The implementation of health IT in practice is costly in time and effort and less is known about these issues in small- and medium-sized practices where the impact, of improved or disrupted workflows may have especially significant consequences because of limited resources. Practices would derive great benefit from effective tools for assessing workflow during many types of health IT implementation, such as creating disease registries, collecting quality measures, using patient portals, or implementing a new electronic health record system. To that end, in 2008, AHRQ funded the development of the Workflow Assessment for Health IT toolkit (Workflow toolkit). Through this toolkit, end users should obtain a better understanding of the impact of health IT on workflow in ambulatory care for each of the following stages of health IT implementation: (1) Determining system requirements, (2) selecting a vendor, (3) preparing for implementation, or (4) using the system post implementation. They should also be able to effectively utilize the publicly available workflow tools and methods before, during, and after health IT implementation while recognizing commonly encountered issues in health IT implementation. In the current project AHRQ is conducting an evaluation to ensure that the newly developed Workflow toolkit is useful to small- and medium-sized ambulatory care clinic managers, clinicians, and staff.

The evaluation will consist of field assessments of use of the Workflow toolkit in 18 small- and medium-sized practices and gathering feedback from two Health IT Regional Extension Centers (RECs) who are providing support to some of these practices. The evaluation will address the issues of system validation as classically defined in software engineering: Determining whether the software or system actually meets the requirements of the user to perform the relevant tasks. The evaluation will answer the following questions:

• Are results correct? Are individual tools included in the Workflow toolkit accurate? Does workflow assessment with the Workflow toolkit provide accurate information the practice can act upon?
• Does knowledge change? Does user knowledge and capacity change? Does user knowledge of workflow in their own practice change?
• Do decisions change? Do user decisions about workflow assessment change? Do user decisions about health information technology (health IT) implementation change?
• Do outcomes change? Are changes in workflow favorable? Are changes in clinical practices favorable? Are changes to the practice favorable? Are changes for patients favorable?

To answer these questions the proposed evaluation will be conducted to examine usefulness of the Workflow toolkit in small- and medium-sized practices. The evaluation will be conducted with 18 practices affiliated with one of two Practice-based Research Networks (PBRNs) in Oregon and Wisconsin, and with the Health IT Regional Extension Centers (RECs) in those States. Participants will be recruited who agree to use the Workflow toolkit in their specific health IT project for a minimum of 10 weeks. This will provide an opportunity to observe use of the Workflow toolkit amongst its intended end users, who are best positioned to provide critical feedback to improve the functionality of the Workflow toolkit.

This study is being conducted by AHRQ through its contractors, the Oregon Rural Practice-based Research Network (ORPRN) and the Wisconsin Research & Education Network (WREN), pursuant to AHRQ's statutory authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of health care services and with respect to health care technologies, facilities, and equipment. 42 U.S.C. 299a(a)(1) and (5).

Method of Collection

To achieve the goals of this project the following activities and data collections will be implemented:

(1) Creation of Clinic Study Team: Each participating practice will form small teams, referred to as Clinic Study Teams, who will participate in the Pre-Workflow Toolkit Interview, use the Workflow toolkit and participate in Observations, and participate in the Post-Workflow Toolkit Interview. Each team will include a maximum of 14 individuals and may represent the following types of respondents: clinicians, office managers, front office staff, medical assistant or nurse, nurse care manager, social worker, health educator, information technology specialist, and/or quality improvement director.

(2) Pre-Workflow Toolkit Interview: these will consist of semi-structured interviews with practice staff and with three specialists from each Health IT Regional Extension Center. These interviews are designed to examine the knowledge, attitudes, and barriers to and facilitators of workflow assessment for implementation of health IT. Respondents will be asked to define workflow, to rate its importance to the practice or REC and to health IT implementation, to describe factors motivating use of the Workflow toolkit, to describe previous experience with assessing or redesigning workflow, and to describe previous experience with health IT implementation and the effect of this implementation on work processes in their practice (practices) or for their clients (RECs).

(3) Observations: Participating practices will form small teams (Clinic Study Teams) who will use the Workflow toolkit. A member of the project staff will join each Clinic Study Team or the three specialists at each of the two RECs, as participant-observer and will meet with the team at times to be determined by the teams, but at least every two weeks after the Pre-Workflow Toolkit Interview for at least four visits. During these visits project staff will participate in and keep field notes regarding the practice's or REC's workflow assessment activities.

(4) Usage Logs: As part of their workflow assessment process, Clinic Study Teams, and REC staff, will be asked to meet weekly. For weekly meetings at which a project staff member is not present, Clinic Study Teams and REC staff will keep a record of workflow assessment activities including use of the workflow assessment toolkit, recording in a free-form journal the purpose and results of the activity as well as issues that arose in the process.

(5) Post-Workflow Toolkit Interview: This final interview will consist of individual semi-structured interviews of practice staff and three specialists from each Health IT Regional Extension Center. These interviews will (a) Re-examine their knowledge and attitudes about workflow assessment; (b) revisit the barriers to and facilitators of workflow assessment; (c) discuss changes that have taken place as a result of the process; (d) explore outcomes in terms of: (d.1) for practices, the perceived impacts on clinicians, the practice staff, the practice, and the patients; and (d.2) for RECs, technician confidence in guiding affiliated clinics in understanding workflow; and finally (e) assess the overall impressions about the usefulness of the Workflow toolkit as well as any suggested changes.

The outcome of the evaluation will be a report including recommendations for enhancing and improving the Workflow toolkit. The report will provide results about the perceived usefulness of the Workflow toolkit. Results will be produced separately for practices and RECs as well as for both user groups as a whole. The report will also include specific suggestions on how to revise Workflow toolkit to make it more useful to its intended audiences.

Estimated Annual Respondent Burden

Exhibit 1 shows the estimated annual burden hours for each respondent's time to participate in this evaluation. Each practice will convene a “Clinic Study Team” consisting of no more than 14 individuals; this process will take approximately 8 hours per practice, or about 35 minutes per person. The Pre-Workflow interview will be completed by a total of up to 258 persons (about 14 per practice and 3 per REC) and requires one hour. Up to four observations will be conducted for up to 258 persons and they are each estimated to take two hours. Ten usage logs will be completed by a total of up to 258 persons (one per week of study activity) and completion of a single usage log should take no longer than 15 minutes. The Post-Workflow interview will be completed by a total of up to 258 persons and requires one hour.

The total annual burden is estimated to be 3,372.

Exhibit 2 shows the estimated annual cost burden associated with the organizations' time to participate in this research. The total annual burden is estimated to be $104,813.

Exhibit 1—Estimated Annualized Burden Hours

Data collection	Maximum number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Creation of Clinic Study Team	252	1	35/60	147
Pre-Workflow Toolkit Interview	258	1	1	258
Observations	258	4	2	2,064
Usage Logs	258	10	15/60	645
Post-Workflow Toolkit Interview	258	1	1	258
Total	1,284	NA	NA	3,372

Exhibit 2—Estimated Annualized Cost Burden

Data collection	Maximum number of respondents	Total burden hours	Average hourly rate *	Total cost burden
Creation of Clinic Study Team	252	147	32.28	4,745
Pre-Workflow Toolkit Interview	258	258	32.28	8,005
Observations	258	2,064	32.28	64,044
Usage Logs	258	645	32.28	20,014
Post-Workflow Toolkit Interview	258	258	32.28	8,005
Total	1,284	3,372	NA	104,813
* The hourly wage for the participants across the four data collections (pre-workflow toolkit interviews, observations, usage logs, and post-workflow toolkit interview) is based upon a weighted mean of the average hourly wages for Family and General Practitioners (1.5; $87.84 per hour); office managers (1.0; $35.18 per hour); front office staff (1.0; $15.15 per hour); medical assistants or nurses (2.5; $24.36 per hour); nurse care managers (0.5; $33.57); social workers (0.1; $24.44 per hour); health educators (0.1; $25.12 per hour); information technology specialists (0.25; $23.43 per hour); quality improvement directors (0.25; 25.12 per hour); and technical staff (1.0; $33.14 per hour) for Oregon and Wisconsin from the U.S. Department of Labor, Bureau of Labor Statistics, May 2010 National Occupational Employment and Wage Estimates for the United States, Occupational Employment Statistics (OES), Washington, DC (Feb. 2009), http://bls.gov/​oes/​2010/​may/​www.bls.govoessrcst.htm (accessed November, 2011).

Estimated Annual Costs to the Federal Government

The estimated total cost to the Federal Government for this project is $793,456 over a 27-month period from September 23, 2011 to December 22, 2013. The estimated average annual cost is $352,646. Exhibit 3 provides a breakdown of the estimated total and average annual costs by category.

Exhibit 3—Estimated Total and Annual Cost * to the Federal Government

Cost component	Total cost	Annualized cost
Project Management and Coordination Activities	$96,449	$42,866
Develop Research and Recruitment Plans	78,383	34,837
Compliance with PRA	12,267	5,452
Obtaining IRB approval	10,254	4,557
Develop Data Analysis Plan	18,246	8,109
Conduct Evaluation	534,401	237,512
Data analysis and Final Report	23,554	10,468
Ensure 508-compliant deliverables	19,902	8,845
Total	793,456	352,646
* Costs are fully loaded including overhead and G&A.

Request for Comments

In accordance with the Paperwork Reduction Act, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ healthcare research and healthcare information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.

Dated: May 3, 2012.

Carolyn M. Clancy,

Director.