2014-18972. Agency Information Collection Activities: Proposed Collection; Comment Request

AGENCY:

Agency for Healthcare Research and Quality, HHS.

ACTION:

Notice.

SUMMARY:

This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Evaluation of the AHRQ Healthcare Horizon Scanning System.” In accordance with the Paperwork Reduction Act of 1995, Public Law 104-13 (44 U.S.C. 3506(c)(2)(A)), AHRQ invites the public to comment on this proposed information collection.

DATES:

Comments on this notice must be received by October 14, 2014.

ADDRESSES:

Written comments should be submitted to: Doris Lefkowitz, Reports Clearance Officer, AHRQ, by email at [e-mail].

Copies of the proposed collection plans, data collection instruments, and specific details on the estimated burden can be obtained from the AHRQ Reports Clearance Officer.

FOR FURTHER INFORMATION CONTACT:

Doris Lefkowitz, AHRQ Reports Clearance Officer, [phone], or by email at [e-mail].

SUPPLEMENTARY INFORMATION:

Proposed Project

“Evaluation of the AHRQ Healthcare Horizon Scanning System”

The American Recovery and Reinvestment Act (ARRA) appropriated $1.1 billion for comparative effectiveness research (CER), of which $300 million was made available to the Agency for Healthcare Research and Quality (AHRQ). The goal of CER is to improve patient outcomes by providing clinicians and patients the information they need to choose between preventive and diagnostic treatments, and other health care options to identify the options that best fit an individual patient's needs and preferences. The EHC Program was created in response to Section 1013 of the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003.

To better inform comparative effectiveness research investments at the EHC Program, AHRQ used some of the ARRA funds to develop a horizon scanning system to identify and monitor emerging health care technologies and innovations. While horizon scanning systems exist in other countries, these systems do not take into account the unique political, regulatory, cultural, and economic context of the U.S. health care system. To meet this need, the AHRQ Healthcare Horizon Scanning System was implemented in November 2010. The AHRQ Healthcare Horizon Scanning System provides a systematic process to identify and monitor target technologies and innovations in health care and to create an inventory of target technologies that have the highest potential for impact on clinical care, the health care system, patient outcomes, and costs. It is also a tool for the public to identify and find information on new health care technologies and interventions. Additionally, the AHRQ Healthcare Horizon Scanning System serves as a resource for those involved in decision making about adoption, implementation, and coverage of new health care interventions.

To fulfill its purpose, the AHRQ Healthcare Horizon Scanning System performs three functions: (1) Identification and prioritization of interventions in late phase development for tracking and monitoring; (2) monitoring of target interventions through the development of detailed information on interventions in late phase development; and (3) assessment of potential impact of target interventions through the gathering and synthesizing the perspectives of experts from various areas of the health care community about the potential impact those target interventions may have on the health care system, clinical care, patient outcomes, and health care costs.

As the first and only U.S. horizon scanning system, it is important to understand whether the AHRQ Healthcare Horizon Scanning System is implementing its functions effectively. This evaluation is also essential to determining whether the AHRQ Healthcare Horizon Scanning System is meeting the needs of patients, clinicians, private industry, and policymakers and how it can be improved to better meet those needs. The evaluation will address the following research questions:

1. How successfully did the AHRQ Healthcare Horizon Scanning System identify and prioritize interventions for monitoring?

2. How successfully did the AHRQ Healthcare Horizon Scanning System monitor the selected target interventions?

3. How accurately did the AHRQ Healthcare Horizon Scanning System assess the potential impact of the interventions?

4. How can the processes for identification, prioritization, monitoring, and assessment of potential impact of the interventions be improved?

This research has the following goals:

1. To assess the performance of the AHRQ Healthcare Horizon Scanning System in the identification and prioritization of interventions which are important topics for further assessment.

2. To assess the performance of the AHRQ Healthcare Horizon Scanning System in terms of the quality of information provided on the topics selected, and the accuracy of the assessment of potential impact.

3. To identify which, if any, of these areas of performance may require improvement so as to strengthen the effectiveness of the AHRQ Healthcare Horizon Scanning System.

This evaluation is being conducted by AHRQ through its contractor, ECRI Institute, and ECRI's subcontractor, Mathematica Policy Research, pursuant to AHRQ's statutory authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of health care services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

Method of Collection

To achieve the goals of this project the following data collections will be implemented:

1. Expert Survey—The purpose of this survey, completed by domain experts, is to measure the accuracy and completeness of the AHRQ Healthcare Horizon Scanning System Potential High Impact reports and to collect their assessment of the potential for high impact for the included Potential High Impact interventions.

2. Expert Consultation—The purpose of this consultation with experts is to confirm the cases of inaccurate or missing information identified by a sole expert in the Expert Survey.

3. Stakeholder Survey—The purpose of this survey, completed by stakeholders and likely users of the reports issued by the AHRQ Healthcare Horizon Scanning System, is to rate the relevance, clarity, and usefulness of the Potential High Impact reports.

4. Key Informant Interview—The purpose of these interviews of the AHRQ Healthcare Horizon Scanning System staff is to learn about areas and suggestions for improvement in the identification, monitoring, and impact assessment processes.

The data collected by the Expert Survey will be used to measure the accuracy and completeness of the Potential High Impact reports and the accuracy of the potential for high impact assessments. If the expert survey identifies cases of inaccurate or missing information that are not reported by multiple experts, we will conduct an Expert Consultation with another expert to confirm these cases. Accuracy of the potential for high impact assessments will be measured by the level of sensitivity (if experts agree that the Potential High Impact interventions identified by the AHRQ Healthcare Horizon Scanning System are high impact interventions) and specificity (if experts agree that the No Potential High Impact interventions identified by the AHRQ Healthcare Horizon Scanning System should be excluded from the group of Potential High Impact interventions).

The Stakeholder Survey will collect data to measure the usability of the Potential High Impact reports and the specific report sections that include the potential high impact assessment, summary, and synthesis of expert comments. These data will be used to inform the improvement of the format and content of the report. The survey will also collect information on the sources and media these stakeholders use to find CER information to help AHRQ better target distribution of these reports to stakeholders.

A series of semi-structured Key Informant Interviews will be conducted with staff and domain experts at ECRI Institute and other organizations that participate in the AIIRQ Healthcare Horizon Scanning System in order to identify opportunities for improvements to the AHRQ Healthcare Horizon Scanning System process. Qualitative interviews are the main vehicle for gathering data to (1) learn which elements of the AHRQ Healthcare Horizon Scanning System Protocol are working well and the reasons why they are working well; and (2) understand which elements of the AHRQ Healthcare Horizon Scanning System Protocol can be improved, how they might be improved, and the relative importance of suggested improvements.

All of these information collection activities will allow for an evaluation of the AHRQ Healthcare Horizon Scanning System, thereby creating the opportunity to both maintain and improve this important national resource. The findings will be presented in a report to ECRI Institute and AHRQ.

Estimated Annual Respondent Burden

Mathematica expects a response rate of 80 percent from the sample of 67 experts for the Expert Survey 54 completed surveys. The Expert Survey is expected to require about 20 minutes, on average, to complete. Mathematica expects that Expert Consultation with 15 experts will be needed to confirm cases of inaccurate or missing information identified in the Expert Survey. The follow-ups should be about 10 minutes.

For the Stakeholder Survey, Mathematica expects that 30 percent of the sample of 700 stakeholders will be ineligible (i.e. will not find any of the presented reports relevant and therefore unable to rate a report) and that 65 percent of the eligible sample will complete, resulting in 319 completes. It should take about 30 minutes to complete the Stakeholder Survey. Mathematica will conduct senii-structured Key Informant Interviews, on average lasting 50 minutes, with 23 respondents.

Exhibit 1—Estimated Annualized Burden Hours

Form name	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Expert Survey	54	1	.33	18
Expert Consultation	15	1	.17	3
Stakeholder Survey	319	1	.50	160
Key Informant Interviews	23	1	.83	19
Total	411			200

Exhibit 2—Estimated Annualized Cost Burden

Form name	Number of respondents	Total burden hours	Average hourly wage rate *	Total cost burden
Expert Survey	54	17.8	$92.25	$1,642
Expert Consultation	15	2.5	** 92.25	231
Stakeholder Survey	319	59.5	*** 48.72	7,771
Key Informant Interviews	23	19.1	38.68	739
Total	411			10,383
* May 2013 National Occupational Employment and Wage Estimates, U.S. Department of Labor, Bureau of Labor Statistics.
** Based on average wage for physicians and surgeons.
*** Based on average wage for medical and health services managers.
**** Based on average wage for social scientists and related workers.

Request for Comments

In accordance with the above-cited Paperwork Reduction Act legislation, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.

Dated: July 30 2014.

Richard Kronick,

AHRQ Director.