03-19462. Request for Additional Hospital CAHPS® (HCAHPS) Test Sites

AGENCY:

Agency for Healthcare Research and Quality (AHRQ), HHS.

ACTION:

Notice of Request.

SUMMARY:

The Agency for Healthcare Research and Quality (AHRQ) in partnership with the Centers for Medicare and Medicaid Services (CMS) is soliciting volunteer sites for the additional testing of a draft Hospital CAHPS (HCAHPS) instrument. The mutual goals of AHRQ and CMS are to develop a standardized survey that is reliable and valid, that will reside in the public domain, and that will make comparative non-identifiable information on hospital patients' perspectives on care widely available. While there are many good survey tools available to hospitals, there is currently no nationally used or universally accepted survey instrument that allows comparisons across all hospitals. To this end, at the request of CMS, AHRQ and the CAHPS®II grantees developed a draft HCAHPS instrument with input from the various stakeholders in the industry. Initial testing of the survey is being done as part of a CMS three-State pilot by hospitals in Arizona, Maryland, and New York.

On June 27th, 2003, CMS published a Federal Register Notice (vol. 68, no. 124, pages 38346-38358) soliciting input into implementation options and the draft HCAHPS survey. To take advantage of the input received, AHRQ and CMS are interested in conducting additional testing of the HCAHPS instrument, sampling and data collection processes to assess issues involved in survey implementation. To accomplish these tasks we plan to conduct very quickly, additional testing at a limited number of hospitals that volunteer to work with AHRQ and CMS. This notice describes an opportunity for volunteering for additional testing for refining the HCAHPS survey and the nature of the implementation procedures. Most of this additional testing will occur between September and December 2003.

This is a separate initiative from “The Quality Initiative: A Public Resource on Hospital Performance”. (For more information about that initiative, please see the Supplemental Information section of this notice).

Types of Studies To Be Conducted During Additional Testing

A. Survey Method Issues

The following are some examples of the type of Methodological studies that AHRQ and CMS would like to conduct. These will be finalized once we have received public input at the close of the comment period for the June 27th Federal Register Notice.

1. Test of mode effects (mail versus telephone) within hospitals using randomly assigned split samples. Because HCAHPS, when finally implemented, may be fielded in by both mail and telephone it is important to test and revise the instrument so that there is comparability across these modes.

2. Test the effect of intervening stays in other facilities. A proportion of patients will be discharged from the hospital in question to another facility before they go home. It is important to test the effect of intervening stays in other facilities on HCAHPS scores for the acute care hospital from which the patient was sampled.

3. Psychometric analyses to evaluate the equivalence of English and the Spanish HCAHPS forms.

4. Test the effect of lag time on HCAHPS scale scores. Due to logistical delays in obtaining up-to-date discharge lists, there will almost certainly be a lag between hospital discharge and survey administration. Long time lags between discharge and survey administration may bias survey responses. For example, time since discharge may affect the survey scores because respondents' memories of their experience degrade with time or because the composition of the respondent pool changes with time (the less healthy patients may be less likely to respond or can not respond because they are too ill or have expired).

5. Test the effect on response rate of different survey materials, taking into account incremental changes in cost. There is some evidence in the survey research literature that response rate can be influenced by the type of survey materials that are sent out. In general impersonal materials from lower status sources will result in lower response rates than personalized materials sent out by higher status sources. However, personalized materials may cost more to produce.

6. Evaluate the covariation of HCAHPS scores with clinical indicators of hospital performance, such as those required by CMS in the 7th scope of work for Quality Improvement Organizations (QIOs) i.e., see www.cms.gov for more information on QIOs and the 7th scope of work statement. These analyses would constitute an expansion of the construct validity analyses for HCAHPS scores in the CMS Pilot.

7. Evaluate HCAHPS instrument psychometrically including the following characteristics:

• Quality of item responses (missing item rates, skip pattern errors);
• Factors associated with item response rates;
• Factors associated with survey response rates;
• Construct validity of composites and ratings;
• Internal consistency;
• Hospital level reliability;
• Components of variance; and
• Case mix adjustment.

B. Implementation Issues

AHRQ and CMS also seek to study the processes for implementation of HCAHPS. Topics include:

1. Sampling procedures that worked well or caused problems.

2. Survey operations procedures that worked well or caused problems.

3. How easily or readily can HCAHPS be integrated into a hospital's existing sampling and survey operations procedures.

Criteria for Additional Site Selection

While AHRQ and CMS would like to provide wide access to the survey for testing, there are limited agency resources for coordinating the testing and analysis of data. Therefore, it is necessary to seek volunteer acute care sites that are able to provide the resources for data collection using the HCAHPS survey, and agree to submit the data to a central repository for analysis. Hospitals may volunteer to participate in the testing program individually or in a group in cooperation with a hospital association, chain or other coalition. Potential testing sites will be selected that best meet the analytic needs of the HCAHPS development effort. Thus, selection from among potential candidate sites will be made using the practical criteria set out below. Criteria for selection of the additional sites are designed to achieve additional diversity in the characteristics of test sites, obtain the most reliable and valid data possible, and to maximize the use of resources allotted for this work.

The criteria for selection are as follows. The test site must:

1. Be able to pay the full cost of data collection and database creation using specifications provided by AHRQ;

2. Be able to field the survey within the timeframe specified by AHRQ to be determined at the time of selection (Most of the testing will occur between September and December 2003, but additional testing may be done in early 2004. Applicants should indicate their ability to carry out the work during those periods in response to Information Item 12 below);

3. Employ a survey vendor with an established record of hospital patient survey experience;

4. Be able to provide an adequate sample size to meet the needs of analyses;

5. Be able to adapt survey implementation as requested by AHRQ, to meet the needs of the experimental design; and

6. Be able to provide a person to coordinate the test site work with AHRQ.

Selection of additional test sites will be determined at the sole discretion of the AHRQ and CMS.

Information Requirements: To volunteer to participate as an additional testing site, please provide the following information:

1. Volunteer site9s) name(s) and location(s).

2. Contact person information including name and title, address, telephone number, fax number and e-mail address.

3. Coordinator for site data collection information (if different from contact person) including name and title, address, telephone number, fax number and e-mail address.

4. The following are examples of the types of studies that may be useful. Additional ones may be identified as a result of the June 27th Federal Register Notice. Please provide information about what types of studies you will or will not be willing to participate in.

a. Test of mode effects (mail versus telephone).

b. Test the effect of intervening stays in other facilities.

c. Psychometric analyses to evaluate the equivalence of English and Spanish HCAHPS forms.

d. Test the effect of lag time on HCAHPS scale scores.

e. Test the effect on response rate of different survey materials (e.g., personalized cover letters), taking into account incremental changes in cost.

f. Evaluate the covariation of HCAHPS scores with clinical indicators of hospital performance.

g. Evaluate HCAHPS instrument psychometrically.

h. Sampling procedures that worked well or caused problems.

i. Survey operations procedures that worked well or caused problems.

j. How easily or readily HCAHPS is integrated into a hospital's existing sampling and survey operations procedures.

5. Number of hospitals proposed for inclusion in the testing.

6. Evidence that hospital(s) is/are willing to participate (i.e., acknowledgement or confirmation from CEO).

7. Average number of discharges per month from participating hospitals, and the average number of discharges for each of the following services: medical, surgical, and obstetrics.

8. Name of current surveys being used in the site and modes of administration of each survey used.

9. Name of current survey vendors working with the site(s).

10. Name of survey vendor who has the largest share of the hospital market in which the volunteer facility, organization or association operates.

11. Statement or affidavit indicating authorization to commit the organization(s) to pay the specific estimated cost of sample selection, data collection, database preparation and coordination with AHRQ.

12. Current schedule for data collection of patient survey data, if you have one.

13. Process and schedule for selecting a vendor for the proposed testing or name of vendor already selected.

dates:

Please submit requested information on or before September 2, 2003.

addresses:

Submissions should include a brief cover letter and the requested information about the potential site(s). They may be in the form of an e-mail with attachments, or a letter, preferably with an electronic file in a standard word processing format, (e.g., Microsoft Word or Word Perfect) on a 31/2 inch diskette. E-mail submissions are preferred and will be acknowledged upon receipt.

E-mail Responses to this request should be submitted to: [e-mail]. Written or faxed responses should be submitted to:

Charles Darby, Agency for Healthcare Research and Quality, Center for Quality Improvement @ Patient Safety, 540 Gaither Road, Rockville, MD 20850. Phone: [phone]; Fax: [phone].

In order to facilitate handling of submissions, please include all requested information about the candidate facilities. Please do not use acronyms. Electronic submissions are strongly encouraged.

for further information contact:

Charles Darby, Center for Quality Improvement and Patient Safety, Agency for Healthcare Research and Quality; Phone: [phone]; Fax: [phone]; E-mail [e-mail].

supplementary information:

The Agency for Healthcare Research and Quality (AHRQ) has been a leading proponent and supporter of the development of instruments for measuring patient experiences within the healhcare system of the United States. Through prior CAHPS® patient survey development efforts, AHRQ has been able to provide valuable information to consumers and purchasers alike. The CAHPS survey for health plans is widely accepted as an industry standard. Therefore, as the research partner of CMS, AHRQ is charged with the development of a hospital patient experience of care instrument as well as the development of reporting strategies to maximize the utility of the survey results. In an effort to provide a firm foundation of evidence-based research for the HCAHPS instrument, the AHRQ is requesting voluntary participation from acute care facilities as potential sites for additional field-testing of the draft HCAHPS survey instrument to provide analytic data that will complement the results of the pilot testing done by QIOs for CMS.

Once HCAHPS is finalized, it will be made available to “The Quality Initiative: A Public Resource on Hospital Performance”, which is a public/private partnership that includes the major hospital associations, government, consumer groups, measurement and accrediting bodies, and other stakeholders interested in reporting on hospital quality. In the first phase of the partnership (which has already begun), hospitals are voluntarily reporting the results of their performance on ten clinical quality measures for three medical conditions: acute myocardial infarction, heart failure, and pneumonia. HCAHPS reporting will comprise the second phase of the effort.

For more information or to participate in the Quality Initiative please visit http://www.aha.org under “Quality and Patient Safety, Quality Initiative,” or at http://www.fah.org,, under “Issues/Advisories,” or at http://www.aamc.org by going to “Government Affairs,” “Teaching Hospitals” and then “Quality.”

Dated: July 24, 2003.

Carolyn M. Clancy,

Director.