2022-09026. Agency Information Collection Activities: Proposed Collection; Comment Request

AGENCY:

Agency for Healthcare Research and Quality, HHS.

ACTION:

Notice.

SUMMARY:

This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “TeamSTEPPS® Stakeholder Surveys for AHRQ's ACTION III Diagnostic Safety Capacity Building Contract Task.” This proposed information collection was previously published in the Federal Register on February 9th, 2022 and 60 days for public comment. AHRQ did not receive substantive comments from members of the public during this period. The purpose of this notice is to allow an additional 30 days for public comment.

DATES:

Comments on this notice must be received by May 31, 2022.

ADDRESSES:

Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to www.reginfo.gov/​public/​do/​PRAMain. Find this particular information collection by selecting “Currently under 30-day Review—Open for Public Comments” or by using the search function.

FOR FURTHER INFORMATION CONTACT:

Doris Lefkowitz, AHRQ Reports Clearance Officer, [phone], or by email at [e-mail].

SUPPLEMENTARY INFORMATION:

Proposed Project

TeamSTEPPS® Stakeholder Surveys for AHRQ's ACTION III Diagnostic Safety Capacity Building Contract Task 3

AHRQ awarded a contract to the MedStar Health Research Institute (MHRI) in 2019 and received OMB fast track clearance (OMB control number 0935-0179, expiration date of 11/30/23), to provide program support and expertise related to improving diagnostic safety and quality across five distinct contract tasks. Task 3 of the contract is to develop, pilot test and promote a TeamSTEPPS® Course to improve communication among providers related to diagnosis. TeamSTEPPS® to Improve Diagnosis provides communication strategies, including methods to improve intra-professional communication and communication during the referral process and to practice mutual support and situation monitoring during the diagnostic process. TeamSTEPPS® to Improve Diagnosis includes an educational module for leaders on strategies to facilitate improved communication with and among providers related to diagnosis. This module also includes a Team Assessment Tool for Improving Diagnosis (the “Team Assessment Tool”).

The Team Assessment Tool is an instrument developed as a method of self-assessment, with the goal of helping teams reflect on their current diagnostic and teamwork practices. In addition, it orients them to the repertoire of tools available within the TeamSTEPPS for Improving Diagnosis course that are available to support improvement efforts. The Team Assessment Tool asks participants to complete self-assessment ratings as a mechanism to identify strengths and opportunities for improvement in unit-based teamwork. The unit level aggregate results of the assessments help unit leaders identify priorities for training via use of course modules and specific interventions with their diagnostic improvement teams.

AHRQ would like to further develop this Team Assessment Tool into a measurement instrument, expanding on its intended use as an educational activity and formative assessment. The opportunity to provide evidence (via publication in peer reviewed journals) that the tool is both valid and reliable will strengthen its acceptance in the care delivery community and provide a scientifically sound method for teams to assess changes in performance overtime. The Team Assessment Tool requires psychometric testing in order to ensure validity and reliability.

Psychometrics is the construction and validation of measurement instruments and assessing if these instruments are reliable (have consistency in measurement) and valid (have accuracy in measurement). Reliability and validity indicate how well a method, technique, test, or instrument is truly measuring what it intends to measure.

The contractor has conducted precursor psychometric testing on the Team Assessment Tool, which included the following: (1) Item wording and scale refinement, (2) Project Team Subject Matter Expert content review, (3) Non-Project Team Subject Matter Expert review, (4) End-user feedback, and (5) Instrument refinement. This work puts the reliability and validity of the indicators of the instrument at an optimal starting point for full psychometric testing.

Full psychometric testing of this instrument means the scaling must be evaluated extensively, which will require a sample of at least 359 individual care team members (physicians, nurses, ancillary staff, etc.,) from diverse clinical settings to participate in a 15-minute, anonymous, online survey distributed via a shared electronic survey link. Individual care team members will be recruited from across 9 health systems or care settings. The survey will ask participants to read through and complete the questions; participants will not be privy to the results of the survey.

The contractor will examine this sample of results via analyses to determine the stability of the instrument and its indicators, ensuring parallel measurements, homogeneity among indicators, concurrent, convergent, and discriminant validity, latent constructs of the tool, the extent to which measures of the same concept correlate and diverge, and the degree of that correlation in evaluating the instrument's ability to discriminate between different groups with various levels and familiarity with safety culture. It is important to note the responses on the surveys are not being evaluated, but rather the consistency with which the questions are answered is being evaluated ( i.e., determining whether the questions are being interpreted the same by all the users), despite diverse healthcare settings and varying levels of experience and familiarity with TeamSTEPPS. The combination of these psychometric methods will allow for internal and external validity and reliability to be assessed, to create a psychometrically sound instrument vetted for potential widespread adoption.

The Team Assessment Tool instrument will undergo remote usability testing of a survey to refine questions. To execute this task, the contractor has assembled an interprofessional team to execute any or all of the following methods for generating reliability and validity evidence that would be applicable to this specific tool: (1) Parallel forms reliability, (2) internal consistency reliability, (3) inter-rater reliability, (4) content validity, and (5) construct validity, using a multitrait-multimethod matrix and/or known groups testing.

This information collection has the following goal:

1. To determine the stability of the Team Assessment Tool instrument and its indicators in improving communication to reduce diagnostic errors, by quantitatively examining the correlation among responses of each indicator.

This study is being conducted by AHRQ through its contractor, MedStar Health Research Institute, pursuant to AHRQ's statutory authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C 299a(a)(1) and (2).

Method of Collection

To achieve the goal of this project the following information collection instruments will be completed using individual surveys:

(1) Setting Demographics Survey: Prior to testing of the instrument, each health system will take a brief survey to describe the characteristics of the sites engaged in pilot testing ( e.g., size, diagnostic team member role diversity, and familiarity with patient safety and quality improvement activities).

(2) TeamSTEPPS® Team Assessment Tool for Improving Diagnosis: This is collected from individual survey respondents, who are diverse staff members in a diagnostic team. The consistency with which the questions are interpreted and answered among respondents will be evaluated to determine the stability among indicators on the instrument.

AHRQ will use the information collected through this Information Collection Request to assess and enhance the feasibility of adopting a course to improve communication among providers related to diagnosis. AHRQs' ability to publicly share a Team Assessment Tool that has been scientifically validated is expected to be of great interest to the health care community and important in helping organizations prioritize improvement efforts.

Estimated Annual Respondent Burden

Exhibit 1—Estimated Annualized Burden Hours

Form name	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Setting Demographics Survey	9	1	0.25	2.25
TeamSTEPPS® Team Assessment Tool for Improving Diagnosis	350	1	0.25	87.5
Total	359			89.75

Exhibit 2—Estimated Annualized Cost Burden

Form name	Number of respondents	Total burden hours	Average hourly wage rate	Total cost burden
Setting Demographics Survey	9	2.25	a  $57.12	$128.52
TeamSTEPPS® Team Assessment Tool for Improving Diagnosis	265	66.25	b  103.06	$6,827.73
TeamSTEPPS® Team Assessment Tool for Improving Diagnosis	85	21.25	c  15.50	$329.38
Total	359	89.75		7,285.63
a  Based on the mean wages for Medical and Health Services Managers (Code 11-9111) .
b  Based on the mean wages for Family Medicine Physicians (Code 29-1215) .
c  Based on the mean wages for HC Support Occupations (Code 31-0000) .
Occupational Employment Statistics, May 2020 National Occupational Employment and Wage Estimates United States, U.S. Department of Labor, Bureau of Labor Statistics. https://www.bls.gov/​oes/​current/​oes_​nat.htm#b29-0000 .

Request for Comments

In accordance with the Paperwork Reduction Act, 44 U.S.C. 3501-3520, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ's health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.

Dated: April 22, 2022.

Marquita Cullom,

Associate Director.