2020-12513. Agency Information Collection Activities: Proposed Collection; Comment Request

AGENCY:

Agency for Healthcare Research and Quality, HHS.

ACTION:

Notice

SUMMARY:

This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Evaluation of Learning Health Systems K12 Training Program.” This proposed information collection was previously published in the Federal Register on March 12, 2020, and allowed 60 days for public comment. No comments were received from the public during this period. The purpose of this notice is to allow an additional 30 days for public comment.

DATES:

Comments on this notice must be received by 30 days after date of publication of this Notice.

ADDRESSES:

Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to www.reginfo.gov/​public/​do/​PRAMain. Find this particular information collection by selecting “Currently under 30-day Review—Open for Public Comments” or by using the search function.

FOR FURTHER INFORMATION CONTACT:

Doris Lefkowitz, AHRQ Reports Clearance Officer, [phone], or by email at [e-mail].

SUPPLEMENTARY INFORMATION:

Proposed Project

Evaluation of Learning Health Systems K12 Training Program

AHRQ, in partnership with the Patient-Centered Outcomes Research Institute (PCORI), supports an innovative institutional mentored career development program (K12) to train clinician and research scientists to conduct patient-centered outcomes research within learning health systems (LHSs). LHSs provide an environment where science generated from health services research, patient-centered outcomes research (PCOR), and clinical research; informatics; incentives; and culture are aligned for continuous improvement and innovation. In addition, in an LHS, best practices are seamlessly embedded in the care process, in which stakeholders (i.e., providers, patients, and families) are active participants in all elements, and new knowledge is captured as an integral by-product of the care experience. The following are the LHS K12 training program objectives:

• Develop and implement a training program that includes both didactic and experiential learning and embeds the scholars in training at the interface of research, informatics, and clinical operations within LHSs.
• Identify, recruit, and train clinician and research scientists who are committed to conducting PCOR in healthcare settings that generate new evidence to facilitate rapid implementation of practices that will improve quality of care and patient outcomes.
• Establish Centers of Excellence (COEs) in LHS Research Training, focusing on the application and mastery of the newly developed core LHS researcher competencies.
• Promote cross-institutional scholar-mentor interactions, cooperation on multisite projects, dissemination of project findings, methodological advances, and development of a shared curriculum.

The purpose of this evaluation is to assess the overall achievement of the LHS K12 training program's objectives, outcomes, and impact, as well as the program's value to its stakeholders. The information collected through this data collection will allow AHRQ to improve the LHS K12 program and identify whether results correspond to intentional changes in program strategy and implementation.

This study is being conducted by AHRQ through its contractor, 2M Research, pursuant to AHRQ's statutory authority to “build capacity for comparative clinical effectiveness research by establishing a grant program that provides for the training of researchers in the methods used to conduct such research.” 42 U.S.C. 299b-37(e).

Method of Collection

The evaluation will include two types of data collection: (1) Semi-structured interviews with scholars who are close to completing the LHS K12 training program, their health system advisors, and program directors of each of the 11 institutions; and (2) surveys with health system advisors. The proposed data collection spans three years (2020-2023).

To achieve the goals of this project the following data collections will be implemented.

1. Scholar Interview: Interviews with LHS K12 scholars assess the degree of scholar embeddedness in their respective health systems and query which aspects of the training program were most and least successful. Telephone interviews will be conducted one time with scholars who are currently enrolled but close to (within 2 to 3 months of) completing the LHS K12 training program. The total estimated number of scholars interviewed will be approximately up to 123 (or approximately 41 scholars annually).

2. Health System Advisor Interview: Interviews with scholars' health system advisors assess the perceived value of the LHS K12 training program to the health system and the role of health system advisors in supporting the research conducted by LHS K12 scholars. One health system advisor from each scholar's advisory committee will be interviewed by telephone. Health system advisors selected for interviews will include those with direct involvement with or knowledge of the LHS K12 scholars' research projects. Health system advisors will be interviewed once around the same time that the scholar is interviewed. The total estimated number of health system advisors interviewed will be approximately up to 116 (or approximately 39 health system advisors annually).

3. Program Director Interview: Interviews with LHS K12 program directors assess the perceived value of the LHS K12 training program to the health system and the role of health system advisors in supporting the LHS K12 training program. The program director of each of the grantee institutions participating in the LHS K12 program will be interviewed by telephone in the final year of the LHS K12 program. The total number of program directors interviewed will be 10 (or approximately 4 program directors annually).

4. Health System Advisor Survey: Pre-post surveys with scholars' health system advisors measure change in attitudes toward the role of health systems research and the importance of patient, family, and other stakeholder engagement in research. A brief survey will be administered electronically to health system advisors at two time points: Once at the beginning and conclusion of their respective scholar's training. The total number of health system advisors surveyed will be approximately up to 190 (or approximately 63 health system directors annually).

AHRQ will use the information collected through this Information Collection Request to assess the program progress of the LHS K12 training program, and impact to its LHS stakeholders in a prospective manner. The information collected will facilitate program planning.

Estimated Annual Respondent Burden

Table 1 shows the estimated annualized burden hours required for respondents to participate in this evaluation. Interviews will be conducted with a total of 123 scholars, 116 health system advisors, and 10 program directors (approximately 41 scholars, 39 health system advisors, and 4 program directors each year). Each interview is expected to last approximately 60 minutes. Surveys will be conducted with a total of 190 health system advisors (or approximately 63 health system advisors each year). The survey is expected to take less than 10 minutes. The total hour burden is expected to be 284.13 hours (or approximately 94.71 hours each year) for this participant data collection effort.

Table 1—Estimated Annualized Burden Hours

Instrument	Estimated number of respondents	Frequency of response	Average time per response (hours)	Total annual burden estimate (hours)
Scholar Interviews	41	1	1.00	41.00
Health System Advisor Interviews	39	1	1.00	39.00
Program Director Interviews	4	1	1.00	4.00
Health System Advisor Surveys	63	1	0.17	10.71
Estimated Annual Total	147			94.71

Table 2 shows the estimated annualized cost burden based on the time required for respondents to participate in this project. This cost was calculated using average hourly earnings for May 2018, obtained from the Bureau of Labor Statistics' estimates for occupational employment wages. The total estimated cost burden for this data collection is $19,580.75 (or approximately $6,526.92 each year). The following hourly wages were used in the annualized cost calculations: $37.38 per hour for a scholar, $96.22 per hour for a health system advisor, and $52.81 per hour for a program director.

Table 2—Estimated Annualized Cost Burden

Instrument	Estimated number of respondents	Total annual burden estimate (hours)	Hourly rate	Total cost
Scholar Interviews *	41	41.00	$37.38	$1,532.58
Health System Advisor Interviews **	39	39.00	$96.22	3,752.58
Program Director Interviews ***	4	4.00	$52.81	211.24
Health System Advisor Surveys **	63	10.71	$96.22	1,030.52
Estimated Annual Total	147	94.71		6,526.92
Bureau of Labor Statistics (BLS), U.S. Department of Labor. (2018). Occupational employment statistics May 2018 national wages. https://www.bls.gov/​oes/​home.htm
* The hourly wage for scholars varies depending on the scholar's degree. AHRQ averaged hourly wages using the following occupations code to develop an estimate that represents the mix of medical and academic degrees: 29-0000, 29-1000, 21-0000.
** AHRQ anticipates that many health system advisors will be C-suite leaders. The hourly wage for BLS's occupation code 11-1010 (chief executive) was used for this estimate.
*** Program directors hold various roles and responsibilities and, therefore, have varied salaries. For the purpose of this estimate, the hourly wages for the following managerial and post-secondary occupational codes were averaged: 11-3131,11-1021,11-9030,11-9033,11-9039, and 11-9199.

Request for Comments

In accordance with the Paperwork Reduction Act, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ's health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.

Dated: June 4, 2020.

Virginia L. Mackay-Smith,

Associate Director.