2022-12501. Submission for Review: CAHPS Enrollee Survey 3206-0274-RENEWAL

AGENCY:

Office of Personnel Management.

ACTION:

30-Day notice with request for comment.

SUMMARY:

The Office of Personnel Management (OPM) offers the general public and other Federal agencies the opportunity to comment on a proposed information collection (ICR) 3206-0274, Consumer Assessment of Healthcare Providers and Systems (CAHPS®). As required by the Paperwork Reduction Act of 1995, as amended by the Clinger-Cohen Act, OPM is soliciting comments for this collection. The information collection was previously published in the Federal Register on February 25, 2022 allowing for a 60-day public comment period. OPM received twenty-nine comments in response to this information collection. The organizations that submitted comments are the American Association of Nurse Practitioners and the Association of Federal Health Organizations (AFHO). The comments and OPM's responses are in the table.

DATES:

Comments are encouraged and will be accepted until July 11, 2022. This process is conducted in accordance with 5 CFR 1320.1.

ADDRESSES:

Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to www.reginfo.gov/​public/​do/​PRAMain. Find this particular information collection by selecting, “Currently under 30-day Review—Open for Public Comments” or by using the search function.

FOR FURTHER INFORMATION CONTACT:

To request more information on the proposed project or to obtain a copy of the data collection plans and instruments, contact: Michael Kaszynski, Senior Policy Analyst at [e-mail]. Formal requests must be in writing.

SUPPLEMENTARY INFORMATION:

The Office of Management and Budget is particularly interested in comments that:

1. Evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information will have practical utility;

2. Evaluate the accuracy of the agency's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions used;

3. Enhance the quality, utility, and clarity of the information to be collected; and

4. Minimize the burden of the collection of information on those who are to respond, including through the use of appropriate automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submissions of responses.

Analysis

Frequency: Annually.

Affected Public: Federal Employees and Retirees.

Number of Respondents: 73,505.

Estimated Time per Respondent: 15 minutes.

Total Burden Hours: 18,376 hours.

U.S. Office of Personnel Management.

Kellie Cosgrove Riley,

Director, Office of Privacy and Information Management.

OPM Response to Comments for the CAHPS Enrollee Survey 3206-0274—Renewal

Public/individual comments	Section/issue	Comment	Decision	Reasoning
American Association of Nurse Practitioners (AANP)		Requested that OPM amend the survey by changing the word “doctor” to “health care provider” throughout the instrument and clarify that nurse practitioners are included in that definition	OPM determined that this feedback does not necessitate a change to the 30-day notice	As the Agency for Healthcare Research and Quality (AHRQ) is the survey steward, comments related to survey format or questioning clarity should be directed to https://www.ahrq.gov/​cahps/​surveys-guidance/​index.html.
The following section includes recommendations and responses from the Association of Federal Health Organizations (AFHO).
The Association of Federal Health Organizations (AFHO)	Time Burden Estimate	• AFHO members generally agree with the assessment of 15 minutes per response time adding to 18,376 hours for 73,505 total respondents. AFHO members note that the CAHPS survey response time is evidence‐based; however, response time may vary by the respondent, by administration method, and by a respondent's decision to not respond at all. In addition, the burden assessment may not reflect the experience of members with a language barrier	OPM determined that this feedback does not necessitate a change to the 30-day notice	The comments indicate a general agreement with the estimate.
	The CAHPS Survey Instrument	• OPM should support refining and streamlining questions to be more direct and clearer, as well as, actionable for managed care companies/PPOs ○ For example, questions evaluating provider interface with members are not directly actionable by plans, while questions related to member website interface with plan operations or tools, such as the provider directory or member website are actionable by plans	• OPM determined that this feedback does not necessitate a change to the 30-day notice. As OPM is not the survey steward, we cannot make changes to the CAHPS Health Plan Survey 5.1H, Adult Version	• OPM asks Carriers to implement the CAHPS survey as part of OPM's Plan Performance Assessment (PPA) annual process to assess the customer experience. As AHRQ is the survey steward, comments related to survey format or question clarity should be directed to https://www.ahrq.gov/​cahps/​surveys-guidance/​index.html. • OPM understands that Carriers have the best information available in relation to the language needs of their members. Questions related to survey translation should be directed to AHRQ. AHRQ provides additional information on CAHPS translation services Guidelines for Translating CAHPS® Surveys | Agency for Healthcare Research and Quality ( ahrq.gov ).
		• Some AFHO members assert that CAHPS is a lengthy survey which may contribute to a drop in response rates. As many of the topical areas in CAHPS are unlikely to change year-over-year, it may be possible to shorten the CAHPS survey by administering only one topical area per year to boost response rates, lessen burden, and generate a cycle for measuring both relative levels of satisfaction and impact of improvement activities. The questions would be rotated from year to year to allow all questions to be surveyed cyclically
		• OPM should identify and share with carriers the preferred language of each FEHB member to support survey translation prioritization and to tailor the distribution of appropriate surveys through CAHPS vendors
		• OPM should arrange for standardizing translations of the CAHPS surveys to avoid potential inconsistencies in messaging (i.e., having AHRQ perform translations for consistency in structure and messaging across health plans distributing surveys in other languages versus health plans following recommendations to generate surveys in a variety of languages themselves)
		• Several AFHO members service plan members living in other countries where language and terminology used in CAHPS may not be as common making it challenging to track actionable responses via CAHPS data
		• The anonymity of CAHPS data, while purposeful for the survey intent, does impose a challenge to implementing targeted member experience improvement; therefore, some plans assume an added burden in coordinating independent target assessments. In terms of the actionability of CAHPS information for the health plan
		• Some AFHO members recommend integrating open-ended comments in the CAHPS survey as the high-level nature of the CAHPS survey presents a challenge to actionability
		• Several of the questions in the survey are related to provider behavior versus health plan behavior, which is not as directly actionable
	CAHPS Survey Data Collection	• OPM should encourage efforts to promote email and text-based member outreach to increase digital survey responses and improve response rates for more complete data collection • FEHB members are more receptive to completing a paper or online survey as opposed to a telephone-administered survey which may take longer to complete	• OPM determined that this feedback does not necessitate a change to the 30-day notice. As OPM is not the survey steward, we cannot make changes to the CAHPS Health Plan Survey 5.1H, Adult Version administration guidelines	• Multiple survey methodologies are allowed under AHRQ and NCQA guidelines and OPM does not have any jurisdiction in this arena. Feedback related to survey administration should be directed to NCQA and AHRQ as they Oversee the survey data collection guidelines.
		• AFHO recommends inviting sampled individuals to participate in CAHPS via email or text, when possible
		• If outreach transitions to digital and text, then carriers should have the option of forgoing mail outreach as this is cost prohibitive
		• More research is still needed to better understand the effect of email or text outreach on burden. In terms of CAHPS administrative burden on carriers, AFHO members have experienced challenges with survey distribution and collection
		• In terms of CAHPS administrative burden on carriers, AFHO members have experienced challenges with survey distribution and collection
		• AFHO members shared COVID-19 pandemic-related supply chain disruptions in acquiring mailing materials for CAHPS
		• The two-week telephone interview field period presents a challenge as it requires a large volume of interviews to be conducted in a short timeframe, and at times a member may have already completed a paper survey when they are called, but the health plan had not received it yet, due to mail delays
	Feedback Related to Data Use	• CAHPS provides a standardized method of measuring and understanding member experience which is a key component of health plan performance, as well as any opportunities to improve member experience. AFHO members have indicated that information collected from CAHPS has limited practical utility • The FEHB serves an aging demographic. To ensure the accessibility of CAHPS for older populations, OPM should take FEHB demographics into account as changes to survey processes or guidelines may pose a challenge to members' ability to complete the survey • OPM should provide carriers with an understanding of how OPM's Plan Performance Assessment (PPA) program uses CAHPS data to improve carrier performance, given that CAHPS is a randomly sampled survey with subject responses provided based on each respondent's interpretation of the questions	• OPM determined that this feedback does not necessitate a change to the 30-day notice	• OPM understands that the practicality of CAHPS data is determined individually by each FEHB Carrier as they use customer satisfaction data for planning in conjunction with other measures that they collect to assess the customer experience. OPM continues to explore other strategies to measure customer service. At this time, OPM has not found a suitable replacement that meets the PPA methodology criteria. • OPM reviews CAHPS demographics during the annual data evaluation. This information is reviewed internally and not available for public distribution. • OPM relies on Carriers to make determinations on improving Carrier performance as they have the best understanding of their member population, benefit design, and operating environment. • OPM has hosted a PPA Best Practice Workgroup presentation on methods FEHB Carriers have employed to improve on select CAHPS measures. Workgroup presentations are intended to give insight into successful quality improvement efforts on specific topics for FEHB Carriers. It is not meant to dictate any business activities. The PPA Best Practices Workgroup is a forum to allow for open dialogue and idea sharing among FEHB Carriers.
	Feedback Related to Data Distribution	• To improve the actionability of collected data, OPM should create and share a demographic analysis of all CAHPS surveys collected by OPM from year to year. This analysis would allow OPM, carriers and the public to understand better FEHB member demographics, such as geography and enrollment type. Specifically, the analysis would offer an opportunity for carriers to identify the demographic aggregations that would be most meaningful in assessing member experience for actionable quality improvement	• OPM determined that this feedback does not necessitate a change to the 30-day notice. As OPM is not the survey steward, we cannot make changes to the CAHPS Health Plan Survey 5.1H, Adult Version administration guidelines	• OPM understands that FEHB Carriers have access to the demographic breakdown of their own member population. At this time, OPM does not intend to share demographic data from CAHPS across FEHB Carriers.
		• OPM should share FEHB member race, ethnicity, and gender identity with carriers to optimize diverse member data representation in the CAPHS surveys
	Feedback Related to PPA Methodology	• OPM should consider whether there is an opportunity to better align the PPA with National Committee for Quality Assurance (NCQA) Ratings and Accreditation where similar information is also being collected • AFHO members also encourage better alignment with NCQA and OPM in tracking CAHPS improvement as part of NCQA accreditation	• OPM determined that this feedback does not necessitate a change to the 30-day notice	• OPM continues to explore other strategies to measure customer service. At this time, OPM has not found a suitable replacement that meets the PPA methodology criteria. • The PRA request was not a vehicle to provide feedback on PPA methodology. To provide methodology feedback, please email [e-mail] .
		• Reporting on the percentage of members who indicate a rating of 9-10 on rating items versus the percentage of members who indicate a rating of 8-10, to better align with NCQA ratings or transition to the use of NCQA ratings for OPM purposes to reduce duplication of efforts
		• Extending the review timeline for CAHPS performance data
		• Allowing for the review of all FEHB plan CAHPS reports from the past two measurement years rather than one measurement year to better assess the actual impact of improvement plans using CAHPS survey data
		• Will the Net Promoter Score (NPS) referenced in Carrier Letter 2018-07 will receive further discussion as a potential replacement for CAHPS?