2015-10982. Agency Information Collection Activities: Proposed Collection; Comment Request

AGENCY:

Agency for Healthcare Research and Quality, HHS.

ACTION:

Notice.

SUMMARY:

This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Pilot Test of the Proposed Hospital Survey on Patient Safety Culture Version 2.0.” In accordance with the Paperwork Reduction Act, 44 U.S.C. 3501-3521, AHRQ invites the public to comment on this proposed information collection.

DATES:

Comments on this notice must be received by July 6, 2015.

ADDRESSES:

Written comments should be submitted to: Doris Lefkowitz, Reports Clearance Officer, AHRQ, by email at [e-mail].

Copies of the proposed collection plans, data collection instruments, and specific details on the estimated burden can be obtained from the AHRQ Reports Clearance Officer.

FOR FURTHER INFORMATION CONTACT:

Doris Lefkowitz, AHRQ Reports Clearance Officer, [phone], or by email at [e-mail].

SUPPLEMENTARY INFORMATION:

Pilot Test of the Proposed Hospital Survey on Patient Safety Culture Version 2.0

Proposed Project

In 2004, AHRQ developed and published a measurement tool to assess the culture of patient safety in hospitals (OMB control no. 0935-0115). The Hospital Survey on Patient Safety Culture (HSOPS) is a survey of providers and staff that can be implemented by hospitals to identify strengths and areas for patient safety culture improvement as well as raise awareness about patient safety. When conducted routinely, the survey can be used to examine trends in patient safety culture over time and evaluate the cultural impact of patient safety initiatives and interventions. The data can also be used to make comparisons across hospital units. AHRQ also produced a survey user's guide to assist hospitals in conducting the survey successfully. The guide addresses issues such as which providers and staff should complete the survey, how to select a sample of hospital providers and staff, how to administer the questionnaire, and how to analyze and report on the resulting data.

Since 2004, thousands of hospitals within the U.S. and internationally have implemented the survey. In response to requests for comparative data from other hospitals, AHRQ funded the development of a comparative database on the survey in 2006 (OMB control no. 0935-0162). The database is currently compiled every two years, using the latest data provided by participating hospitals (and retaining submitted data for no more than 2 years). Reports describing the findings from analysis of the database are made available on the AHRQ Web site to assist hospitals in comparing their results. The 2014 database contains data from 405,281 hospital provider and staff respondents within 653 participating hospitals. The 2014 User Comparative Database Report presents results by hospital characteristics (e.g., number of beds, teaching status, geographic location) and respondent characteristics (e.g., position type, work area/unit).

The survey constructed in 2004 remains in use today, more than 10 years after its initial launch. Since the launch of HSOPS, AHRQ has funded development of patient safety culture surveys for other settings. In 2008, surveys were published for outpatient medical offices (OMB control no. 0935-0131) and nursing homes (OMB control no. 0935-0132). In 2012, a survey for community pharmacies (OMB control no. 0935-0183) was released. Surveys for each setting built upon the strengths of HSOPS but improved and updated items where appropriate.

Users of HSOPS have provided feedback over the years suggesting that changes to the instrument would be valuable and welcomed. The comparative database registrants provided feedback about potential changes in 2013, and telephone interviews were conducted with 8 current survey users and vendors to gain an in-depth understanding of their thoughts on the current survey and possible changes. As a result of this feedback, the Hospital Survey on Patient Safety Culture Version 2.0 (HSOPS 2.0) is being constructed with the following 8 objectives in mind.

(1) Shift to a Just Culture framework for understanding responses to errors. In the original HSOPS, questions around responses to errors were negatively worded to detect a “culture of blame” in organizations. For example, respondents evaluated the extent to which errors were held against them and whether it felt as though the person was being written up rather than the problem. In contrast, a Just Culture framework emphasizes learning from mistakes, providing a safe environment for reporting errors, and utilizing a balanced approach to errors that considers both system and individual behavioral reasons as causes for errors. New items will be constructed in HSOPS 2.0 to capture the extent to which positive responses to error consistent with a Just Culture framework are present in an organization. For example, respondents will be asked to evaluate the extent to which the organization tries to understand the factors that lead to patient safety errors.

(2) Reduce the number of negatively worded items. The original HSOPS has negatively worded items. For example, respondents are asked whether there are “patient safety problems in this unit” (negatively worded). Using some negatively worded items was intended to reduce social desirability and acquiescence biases and identify individuals not giving the survey their full attention (e.g., “straight-lining,” or providing the same answer for every item, regardless of positive or negative wording). However, many users have indicated that respondents sometimes had difficultly correctly interpreting and responding to the negatively worded items. Therefore, many survey users recommended that the number of negatively worded items should be reduced, but they did not recommend removing all of these items as they felt a mixture of items helps keep respondents engaged.

(3) Add a “Does not apply/Don't know” response option. Analysis of the Comparative Database data found that a percentage of respondents selects “neither agree nor disagree” on many items when they really should have answered “Does not apply/Don't know”. While some portion of respondents will always have neutral feelings about a statement, in some cases a respondent will select a neutral response to an item because they do not have experience in that area or the item does not apply to their position. Addition of a “does not apply/don't know” response option should reduce neutral responses to an item in cases where the item is not relevant for a respondent, providing more statistical variability in responses. Recognizing these issues, the other AHRQ Surveys on Patient Safety Culture all have a 5th “Does not apply/Don't know” response option.

(4) Reword unclear or difficult-to-translate items. HSOPS was originally designed for use in U.S. hospitals, but it has since been translated into languages other than English. Some HSOPS items use idiomatic expressions that do not translate well, such as “things fall between the cracks” and “the person is being written up.” Other items have words that are complex or may mean different things to different people, such as “sacrifice” and “overlook.” HSOPS 2.0 uses more universal phrases which can be accurately translated and have more consistent meaning across respondents, some of whom are non-clinical staff. A related change across many items is use of the word “we” rather than “staff.” It may be unclear to respondents whether providers such as physicians, residents, and interns qualify as “staff,” while “we” invites a more inclusive view of those in the hospital or unit.

(5) Reword items to be more applicable to physicians and non-clinical staff. Users have indicated that the wording of some of the items makes it awkward for physicians to answer. For example, the section that asks about “Your Supervisor/Manager” does not apply well to physicians who report to a clinical leader but not to a manager per se. In addition, some items were difficult for non-clinical staff to answer. For example, the item “We have patient safety problems in this unit” may not be relevant for staff who do not have direct interaction with patients (e.g., IT staff).

(6) Align the HSOPS survey with AHRQ patient safety culture surveys for other settings. The development of patient safety culture surveys for other settings provided opportunities to test new items and refinements of original HSOPS items. Many of these items have performed well for other settings and are relevant to the hospital setting. In addition, standardizing items across the patient safety culture surveys would allow cross-setting comparisons that are not currently possible.

(7) Reduce survey length. To increase response rates and reduce the survey administration burden for hospitals, the revised survey is intended to be shorter than the original instrument. Some of the original items have relatively low variability and therefore contribute little to discrimination between positive and negative assessment of patient safety culture. However, the need for careful testing of alternative questions means that the initial draft of the revised or 2.0 survey is slightly longer than the original. Through cognitive interviewing, pilot testing, and expert review, we will identify items that can be deleted, resulting in a shorter final instrument.

(8) Investigate supplemental items/composites. Develop a set of supplemental items for the HSOPS 2.0 survey pertaining to Health Information Technology (Health IT).

Further details about the specific changes by composite and at the item level can be found on the AHRQ Web site at: http://www.ahrq.gov/​professionals/​quality-patient-safety/​patientsafetyculture/​hospital/​update/​index.html.

The draft 2.0 version of the instrument has undergone preliminary cognitive testing with 9 hospital physicians and staff members as well as review by a Technical Expert Panel (TEP).

This research has the following goals:

(1) Cognitively test with individual respondents the items in a) the draft HSOPS 2.0 survey and b) HSOPS 2.0 supplemental item set assessing Health IT Patient Safety. Cognitive testing will be conducted in English and Spanish.

(2) Conduct data collection as follows:

a. A combined pilot test and bridge study for the draft HSOPS 2.0 in 40 hospitals and modify the questionnaire as necessary. The pilot test component will entail administering the draft 2.0 version to determine which items to retain. The bridge study component will entail administering the original HSOPS in addition to the draft HSOPS 2.0 version to provide guidance to hospitals in understanding changes in their scores resulting from the new instrument versus changes resulting from true changes in culture.

b. The pilot testing of the supplemental item set will be conducted with the same hospitals and respondents as the pilot test for the draft HSOPS 2.0. These supplemental items will be added to the draft HSOPS 2.0 survey for pilot testing.

(3) Engage a TEP in review of pilot results and finalize the questionnaire and supplemental item set.

(4) Make the final HSOPS 2.0 survey and the supplemental items publicly available.

This work is being conducted by AHRQ through its contractor, Westat, pursuant to AHRQ's statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

Method of Collection

Cognitive interviews—The purpose of these interviews is to understand the cognitive processes respondents engage in when answering each item on the survey, which will aid in refining the survey instrument. These interviews will be conducted with a mix of hospital personnel, including physicians, nurses, and other types of staff (from dietitians to housekeepers).

Draft HSOPS 2.0—Cognitive interviews have already been conducted with 9 respondents to inform development of the current draft HSOPS 2.0. Up to three additional rounds of interviews will be conducted by telephone with a total of 27 respondents (nine respondents each round). The instrument will be translated into Spanish and another round of cognitive interviews will be conducted with nine Spanish-speaking respondents for a total of up to 36 respondents across all four rounds. A cognitive interview guide will be used for all rounds.

Supplemental Items—Up to three rounds of interviews will be conducted by telephone for a total of 27 respondents (nine respondents each round). The supplemental items will be translated into Spanish and another round of cognitive interviews will be conducted with nine Spanish-speaking respondents for a total of up to 36 respondents across all four rounds. A cognitive interview guide will be used for all rounds.

Feedback obtained from the first round of interviews for the draft HSOPS 2.0 and the supplemental items will be used to refine the items. The results of Round 1 testing, along with the proposed revisions, will be reviewed with a TEP prior to commencing with Rounds 2 and/or 3 testing. In total, up to 72 cognitive interviews will be conducted to refine the draft HSOPS 2.0 and supplemental items for pilot testing.

(2) Pilot test and bridge study—There will be one data collection effort which will provide data for the pilot test and the bridge study. The pilot test of the draft HSOPS 2.0 and supplemental items will allow the assessment of the psychometric properties of the items and composites. We will assess the variability, reliability, factor structure and construct validity of the draft HSOPS 2.0 and supplemental items and composites, allowing for their further refinement. The draft HSOPS 2.0 survey and supplemental items will be pilot tested with hospital personnel in approximately 40 hospitals to facilitate multilevel analysis of the data. Approximately 500 providers and staff will be sampled from each hospital, with 250 receiving HSOPS 2.0 with supplemental items for the pilot test and 250 receiving the original HSOPS for the bridge study comparisons. A hospital point of contact will be recruited in each hospital to publicize the survey and assemble a list of sampled providers and staff. Providers and staff will receive notification of the survey and reminders via email and the web-based survey will be fielded entirely online.

The goal of the bridge study will be to provide users with guidance on how their new results will compare with results from the original HSOPS survey. Although users have requested that the HSOPS survey be revised, they are also concerned about their ability to trend results with data from prior years. A similar bridge study was conducted during the 1994 redesign of the Census Bureau's Current Population Survey (CPS). In the CPS bridge study, an additional 12,000 households were added to the survey's monthly rotation schedule between July 1992 and December 1993. The added households received the redesigned version of the instrument. Thus, the CPS fielded both the revised and the original versions of the instrument simultaneously. One of the most important results of the CPS bridge study was the development of metrics that allowed estimates of change that were due to the changes in the instrument. These metrics were used to adjust the estimates produced by the revised CPS instrument. As a result of the study, key labor force metrics such as the unemployment rate could be trended accurately after the instrument's redesign.

We propose to conduct a similarly constructed bridge study in which sampled providers and staff take either the draft HSOPS 2.0 or original versions of HSOPS. As noted above, a split ballot design will be used in which half of sampled providers and staff in each hospital receive the original HSOPS (N=250) and the other half receive the draft HSOPS 2.0 (N=250). This bridge study is designed to produce metrics of change that are attributable to the changed survey instrument. The number of hospitals and sampled providers and staff for this data collection effort was calculated to ensure the statistical power needed to detect relatively small differences in scores (3 percentage points).

(3) TEP feedback—A TEP has been assembled to provide input to guide patient safety culture survey product development and has been convened to discuss the proposed changes to the HSOPS survey and supplemental items. Upon completion of the pilot test, results will be reviewed with the TEP and the survey will be finalized. This TEP activity does not impose a burden on the public and is therefore not included in the burden estimates in Exhibits 1 and 2.

(4) Dissemination activities—The final HSOPS 2.0 instrument and supplemental items will be made publicly available through the AHRQ Web site. A report from the bridge study will also be made public as a resource to hospitals making the transition to the new survey. This dissemination activity does not impose a burden on the public and is therefore not included in the burden estimates in Exhibits 1 and 2.

Estimated Annual Respondent Burden

Exhibit 1 shows the estimated annualized burden hours for the participants' time to take part in this research. Cognitive interviews for the draft HSOPS 2.0 will be conducted with 36 individuals and will take about one hour and 30 minutes to complete. Cognitive interviews for the supplemental items will be conducted with 36 individuals and take about one hour to complete. We will recruit 40 hospitals for the pilot test and bridge study, sampling approximately 500 staff members in each (250 taking the original survey and 250 taking the HSOPS 2.0 and supplemental item set). Because we require such a large sample within each hospital, we will target only hospitals with 49 or more beds. For hospitals with fewer than 500 providers and staff, we will conduct a census in the hospital (assuming on average 375 providers and staff in these hospitals this will yield a total of 18,375 sample members assuming all 40 hospitals participate. Assuming a response rate of 50 percent, this will yield a total of 9,188 completed questionnaires. The total annualized burden is estimated to be 2,387 hours.

Exhibit 2 shows the estimated annualized cost burden associated with the participants' time to take part in this research. The total cost burden is estimated to be $83,533.26.

Exhibit 1—Estimated Annualized Burden Hours

Form name/activity	Number of respondents	Hours per response	Total burden hours
Cognitive interviews—HSOPS 2.0	36	1.5	54
Cognitive interviews—Supplemental Items	36	1.0	36
Pilot test and bridge study	9,188	0.25	2,297
Total	9,260	na	2,387

Exhibit 2—Estimated Annualized Cost Burden

Form name/activity	Total burden hours	Average hourly wage rate *	Total cost burden
Cognitive interviews (HSOPS 2.0 and supplemental items)	90	a $35.38	$3,184.20
Pilot test and bridge study	2,297	b 34.98	80,349.06
Total	2,387	na	83,533.26
a Based on the weighted average hourly wage in hospitals for one physician (29-1060; $101.53), one registered nurse (29-1141; $30.22), one general and operations manager (11-1021; $52.64), and six clinical lab techs (29-2010; $22.34) whose hourly wage is meant to represent wages for other hospital employees who may participate in cognitive interviews.
b Based on the weighted average hourly wage in hospitals for 1,981 registered nurses, 209 clinical lab techs, 176 physicians and surgeons, and 21 general and operations managers.
* National Industry-Specific Occupational Employment and Wage Estimates, May 2013, from the Bureau of Labor Statistics (available at http://www.bls.gov/​oes/​current/​naics4_​621100.htm [for general medical and surgical hospitals, NAICS 622100]).

Request for Comments

In accordance with the Paperwork Reduction Act, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.

Dated: April 27, 2015.

Sharon B. Arnold,

Deputy Director, AHRQ.