03-1294. Medicaid Program; External Quality Review of Medicaid Managed Care Organizations

AGENCY:

Centers for Medicare & Medicaid Services (CMS), HHS.

ACTION:

Final rule.

SUMMARY:

This final rule establishes requirements and procedures for external quality review (EQR) of Medicaid managed care organizations (MCOs) and prepaid inpatient health plans (PIHPs). It defines who qualifies to conduct EQR and what activities can be conducted as part of EQR. In addition, under certain circumstances, this rule allows State agencies to (1) use findings from particular Medicare or private accreditation review activities to avoid duplicating review activities, or (2) exempt certain Medicare MCOs and PIHPs from all EQR requirements. Also, this rule allows the payment of enhanced Federal financial participation (FFP) at the 75 percent rate for the administrative costs of EQRs or EQR activities that are conducted by approved entities.

EFFECTIVE DATE:

These regulations are effective on March 25, 2003. Provisions that must be implemented through contracts with MCOs, PIHPs, and external quality review organizations (EQROs) are effective with contracts entered into or revised on or after 60 days following the publication date. States have up until March 25, 2004 to bring contracts into compliance with the final rule provisions.

FOR FURTHER INFORMATION CONTACT:

Kristin Fan, [phone].

SUPPLEMENTARY INFORMATION:

To order copies of the Federal Register containing this document, send your request to: New Orders, Superintendent of Documents, P.O. Box 371954, Pittsburgh, PA 15250-7954. Specify the date of the issue requested and enclose a check or money order payable to the Superintendent of Documents, or enclose your Visa or Master Card number and expiration date. Credit card orders can also be placed by calling the order desk at [phone] or by faxing to [phone]. The cost for each copy is $10. As an alternative, you can view and photocopy the Federal Register document at most libraries designated as Federal Depository Libraries and at many other public and academic libraries throughout the country that receive the Federal Register.

This Federal Register document is also available from the Federal Register online database through GPO access, a service of the U.S. Government Printing Office. The Website address is http://www.access.gpo.gov/​nara/​index.html.

I. Background

A. The Balanced Budget Act of 1997

The Balanced Budget Act of 1997 (BBA) added to the Social Security Act (the Act) a new section 1932 that pertains to Medicaid managed care. Most of the provisions of section 1932 of the Act will be implemented in accordance with the Medicaid managed care final rule that was published in the Federal Register on June 14, 2002 (67 FR 40988).

Section 1932(c) of the Act, added by section 4705 of the BBA, describes how quality measurement and performance improvement methods should be applied to Medicaid managed care programs through two specific approaches:

• All State agencies must develop and implement a quality assessment and improvement strategy that includes—(1) Standards for access to care; (2) examination of other aspects of care and services related to improving quality; and (3) monitoring procedures for regular and periodic review of the strategy. (This requirement was addressed in the Medicaid managed care final rule published June 14, 2002.)
• State agencies that contract with Medicaid managed care organizations (MCOs) must provide for an annual external, independent review of the quality outcomes, timeliness of, and access to the services included in the contract between the State agency and the MCO. (This requirement is addressed in this rule.)

Section 1932(c) of the Act also requires the Secretary—

In consultation with the States, to establish a method for identifying entities qualified to conduct external quality review (EQR) (section 1932(c)(2)(A)(ii) of the Act); and

In coordination with the National Governors Association (NGA), to contract with an independent quality review organization to develop the protocols to be used in EQRs (section 1932(c)(2)(A)(iii) of the Act).

Two other provisions of section 1932(c) of the Act are pertinent to this rule. They are (1) the requirement that the results of EQRs be made available to participating health care providers, enrollees and potential enrollees (section 1932(c)(2)(A)(iv) of the Act), and (2) the provision that a State agency may, at its option—

• Take steps to ensure that an EQR does not duplicate a review conducted either by a private independent accrediting organization or as part of an external review conducted under the Medicare program (section 1932(c)(2)(B) of the Act); and
• Exempt an MCO from EQR under certain specified conditions (section 1932(c)(2)(C) of the Act).

Section 4705(b) of the BBA amended section 1903(a)(3)(C) of the Act to provide for increased Federal financial participation (FFP) (75 percent) for the administrative costs the State incurs for EQR or EQR activities performed by specified entities under section 1932(c)(2)(A) of the Act.

B. Proposed Rule

On December 1, 1999 we published a proposed rule in the Federal Register (64 FR 67223) to implement the EQR statutory provisions. A summary of the specific provisions of the proposed regulations precedes each section of the comments and responses below. In the proposed rule, we discussed the two major purposes we had in developing the rule: (1) To provide flexibility for State agencies, and (2) to reflect the well-accepted advances in the technology of quality measurement and improvement. For a more detailed discussion of our basis and purpose for the approach taken in the December 1, 1999 proposed rule, see the preamble to that document at 64 FR 67223.

We received 29 comments from States, national and State organizations, health plans, advocacy groups, and other individuals on the December 1, 1999 proposed rule. The comments generally pertained to the types of entities that can be EQROs, EQR activities, nonduplication and exemption provisions, and dissemination of EQR rules. We carefully reviewed and considered all the comments we received.

C. Agency Information Collection Activities

On November 23, 2001 we published a notice in the Federal Register (66 FR 58741) to comply with the requirement of section 3506 (c)(2)(A) of the Paperwork Reduction Act of 1995. We invited public comment regarding the burden estimate or any other aspect of the EQR protocols we developed in accordance with section 1932(c)(2)(A)(iii) of the Act. This provision required that we contract with an independent quality review organization to develop protocols to be used with respect to EQRs required by statute. In response to the requirement under section 1932(c)(2)(A)(iii) of the Act, we contracted with the Joint Commission on Accreditation of Health Care Organizations (JCAHO) which developed nine protocols and one appendix to several of the protocols in six quality improvement areas. We received 13 comments on the November 23, 2001 Federal Register notice. We carefully reviewed and considered all the comments we received.

II. Provisions of the Proposed Rule and Discussion of Public Comments

A. Basis, Scope and Applicability. (Formerly § 438.1), (Now § 438.310)

In this section we proposed to apply provisions to MCOs, prepaid health plans (PHPs), and entities with comprehensive risk contracts that are exempted by statute from the requirements in section 1903(m) of the Act, health insuring organizations (HIOs).

Comment: Many commenters supported the application of this rule to all three of the above types of entities. One commenter, though not opposed to the inclusion of PHPs, expressed concern about the cost of this requirement when applied to entities that provide services to small populations. The commenter suggested that the regulation apply only to entities to the extent feasible for the study being performed. Another commenter did not agree that the provisions should apply to PHPs and stated that there is no specific reference in Federal law to these organizations and that we have gone beyond the explicit language in section 1932(c) of the Act.

Response: We continue to believe these provisions should apply to most capitated health plans that are not MCOs, but that provide inpatient services. The Medicaid managed care final rule eliminated the term PHP and replaced it with two types of entities—prepaid inpatient health plans (PIHPs) and prepaid ambulatory health plans (PAHPs). That rule, under the authority of section 1902(a)(4) of the Act, which authorizes the Secretary to establish requirements necessary “for proper and efficient operation of the plan,” applies the provisions related to a State's quality strategy to PIHPs but not to PAHPs. It does not apply these quality provisions to PAHPs because these entities provide a more limited array of services (for example, transportation or dental), and we do not believe it appropriate to require States to include these entities in their State quality strategies due to the burden it would impose. We, therefore, are revising this rule to be consistent with the Medicaid managed care final rule (§ 438.204(d)) and apply the EQR provisions to PIHPs as specified at § 438.310. We have also made changes to clarify the applicability of this rule to HIOs to be consistent with the Medicaid managed care final rule.

We do not agree with the commenter that we should exempt entities that have smaller enrolled populations from these requirements. Sections 1932(c)(2)(B) and (C) of the Act specifically identify the circumstances under which an entity may be fully or partially exempt from EQR.

Comment: One commenter asked if we intend to hold Indian Health Services (IHS) and 638 Tribal Facilities to the same standard as MCOs to ensure the quality of care provided to Native Americans.

Response: If an IHS entity or 638 Tribal Facility meets the definition of an MCO or PIHP, it would be subject to these provisions.

Comment: One commenter does not believe that primary care case management (PCCM) programs should be subject to these requirements. Another commenter believes that the activities in the December 1, 1999 proposed rule should be applied to PCCM programs.

Response: The statute does not extend the EQR requirement to PCCMs and the Conference Report, pages 859-860, makes clear that PCCMs were specifically excluded from the requirements. We have used the authority of section 1902(a)(4) of the Act to extend the EQR provision to PIHPs because, like MCOs, PIHPs provide inpatient services and are capitated. If a PCCM meets the definition of a PIHP, then it would be subject to the provisions of this rule. However, traditional PCCMs are reimbursed on a fee-for-service (FFS) basis along with a case management fee. Under that reimbursement arrangement, the PCCM would not be subject to the EQR requirements.

Comment: Many commenters recommended that external review also examine subcontracting managed care entities. One commenter suggested that the definition of quality be expanded to include services provided through subcontracts with MCOs.

Response: The MCO or PIHP is fully responsible (§ 438.230 of the Medicaid managed care final rule) for all activities delegated to another entity. Therefore, the EQR should include information on all beneficiaries and the structure and operations of all entities that provide Medicaid services under either the prime contract or subcontract. At § 438.320, we revised our definition of EQR to clarify our intent that the EQR provisions apply to all services received by Medicaid beneficiaries regardless of whether those services are provided by the MCO or PIHP directly or through a subcontract.

Comment: One commenter is concerned that this rule applies the EQR requirement to PHPs despite the BBA's statutory reference only to organizations under section 1903(m) of the Act. The commenter asked us to clarify whether we intend to apply these requirements to any entity that is paid on a prepaid capitation basis for services furnished to enrollees, even if the PHP is not at any financial risk for those services.

Response: As noted in an earlier response, the EQR provisions will apply to a PIHP defined in the Medicaid managed care final rule as an entity that “provides medical services to enrollees under contract with the State agency, and on the basis of prepaid capitation payments, or other payment arrangement that do not use State plan payment rates and that provides, arranges, or otherwise has the responsibility for the provision of any inpatient hospital or institutional services for its enrollees * * *” We do not apply these quality provisions to PAHPs because these entities provide a more limited array of services (for example, transportation or dental), and we do not require States to include these entities in their State quality strategies due to the burden it would impose. The application of this rule to PIHPs is not based on section 1903(m) of the Act. It is based on section 1902(a)(4) of the Act that authorizes the Secretary to establish requirements necessary “for the proper and efficient operation of the plan.” We believe this is consistent with congressional intent.

PIHP and PAHP designation is not based on whether an entity is at financial risk for services provided. Designation is based on prepaid capitation payments for a scope of services. Even though there will be few PIHPs that are not at financial risk, due to the scope of services these entities provide (for example, inpatient services), we believe they should be subject to EQR provisions.

B. Definitions (Formerly § 438.2), (Now § 438.320)

This section of the proposed rule defined “EQR” and “EQRO.” It also defined the terms “quality” and “validation” as they pertain to EQR.

Comment: One commenter concurred with our requirement that EQR be a multipronged approach which recognizes that none of the activities alone can ensure quality in the complex Medicaid population. One commenter supported the definitions as proposed.

Response: We appreciate that the commenters agreed with our approach to EQR and the proposed definitions. We have retained the multipronged approach to EQR as proposed in the proposed rule.

Comment: One commenter asked that the definition of quality include assessments of structure and process as well as measurements of health and functional outcomes. Several commenters recommended that the definition of quality include both clinical and nonclinical measures of consumer satisfaction and define quality in a way that would be meaningful to people with disabilities. One commenter stated that this definition should address the multifaceted needs of people who have chronic and disabling conditions, for whom there is little likelihood of demonstrable improvement. The commenter recommended that we convene focus groups of consumers, including people with disabilities and families of children with disabilities, to identify how quality should be defined from the consumer's perspective and that the definition should not focus solely on health outcomes. One commenter concurred with the definition of quality as proposed.

Response: We agree with the commenter that the proposed definition of quality did not address situations when beneficiaries have conditions where maintenance or improvement of health outcomes is not likely. We have, therefore, revised the definition to mean the degree to which an MCO or PIHP increases the likelihood of desired health outcomes through the provision of health services that are consistent with current professional knowledge. The revision is consistent with the Institute of Medicine's definition of quality. We do not agree with the remaining recommendations by commenters on how to revise the definition of quality because we think that the commenters' concerns are addressed by other provisions of the regulation. Under § 438.358, we identify three activities that must be conducted to provide information for the EQR. These activities also are required in the Medicaid managed care final rule. They include: (1) The review of compliance with structural and operation standards; (2) the validation of performance measures;^[1] and (3) the validation of performance improvement projects. The optional EQR-related activities are activities that some States currently conduct as part of EQR and we believe are also appropriate to an assessment of quality (such as consumer surveys). We are providing States with the flexibility to determine which, if any, of these optional activities will be included in the EQR and what types of performance measures and performance improvement projects to require of their contracting MCOs and PIHPs. We suggest in the performance improvement project protocol that projects be conducted to address both clinical and nonclinical areas that cover the various categories of beneficiaries and services provided. We also note, as stated in the Medicaid managed care final rule, that EQR is a part of the State's quality strategy, and therefore, States are to provide for the input of Medicaid beneficiaries and other stakeholders in this component of the strategy.

Comment: One commenter suggested amending the definition of EQR to read “* * * quality of health care services furnished or contracted for by each MCO * * *”

Response: We agree with this comment and, as stated previously, have revised the final rule to clarify our intent that the EQR provisions apply to all services received by Medicaid beneficiaries regardless of whether those services are provided by the MCO or PIHP directly or through a subcontract (§ 438.320).

Comment: Several commenters stated that the definition of EQR too narrowly limits the scope of EQR because the definition implies that EQR is primarily concerned with analysis and evaluation of data rather than with collection of data. One of the commenters expressed concern that this would limit the EQRO's ability to identify and bring to the State's attention individual quality of care concerns revealed during data abstraction, or to provide provider-specific feedback on performance measures. The commenter recommended that the rule avoid any reference to “aggregate” information in the definition of EQR. One commenter recommended that the definition of EQR include the development of aggregated data. Another commenter stated that external review should not be limited to the review of information. The commenter believes the external review of plans should include an on-site review of provider practices and procedures and that data alone are insufficient to evaluate performance.

Response: We do not agree that the definition of EQR limits the scope of EQR. We define EQR as the analysis and evaluation of aggregated information. That aggregated information, according to this rule, must be obtained from activities that are consistent with protocols, as defined in this rule, to ensure that data to be analyzed are collected using sound methods widely used in the industry. For each activity, as specified in § 438.364, the entity conducting the activity must report on the objectives, technical methods of data collection and analysis, a description of the data obtained, and conclusions drawn from each activity. Therefore, as part of these activities, the entity conducting them will need to identify and assess quality of care concerns revealed by the activities. The EQR analysis will incorporate findings from all activities, including the evaluation of MCO or PIHP structure and operations. The findings of the overall analysis will need to include an assessment of the strengths and weakness with respect to quality, timeliness, and access of care, and make recommendations for MCO or PIHP improvement in the EQR results as required under § 438.364. Further, we note that under the BBA statutory provisions, EQR is a review of a Medicaid MCO under contract to the State. EQR of individual providers or provider practices is not provided for in the BBA. We believe that the appropriate unit of analysis of EQR is the MCO and PIHP, not individual practitioners.

C. State Responsibilities (§ 438.350)

This section of the proposed rule set forth the State's responsibilities related to EQR. We proposed that each State agency that contracts with MCOs, PHPs, or other entities that have comprehensive risk contracts must, except as provided in § 438.362, ensure that (1) An annual EQR is performed for these contracting entities by a qualified EQRO; (2) the EQRO has sufficient information to use in performing the review; (3) the information that the State agency provides to the EQRO is obtained through methods consistent with protocols specified by CMS; and (4) the results of the EQR are made available, upon request, to specified groups and to the general public.

Section 1932(c)(2)(A) of the Act requires that each contract with an MCO “provide for an annual (as appropriate) external independent review, conducted by a qualified independent entity * * *” In this section we interpreted the parenthetical statement (for which there is no explanation in the legislative history) to be a reference to those MCOs that may be exempted from EQR under section 1932(c)(2)(C) of the Act on the basis of “deemed compliance.” We invited comment on other possible interpretations, which are discussed at the end of this section.

Comment: One commenter noted they concurred with this section of the rule.

Response: We appreciate the commenter's support for the provisions in this section of the proposed rule and retain the provision that requires the State to ensure that the EQRO has information obtained from EQR-related activities and that the information provided is obtained through methods consistent with the EQR protocols established under § 438.352 in this final rule.

Comment: Several commenters asked us for a definition, or the criteria that we will use to determine if State-established protocols are consistent with those developed by us. One of the commenters noted that it would be difficult for all States to follow a single set of protocols because State Medicaid programs vary as to structure, capacity, funding, and governing laws. One commenter asked that we also establish criteria for denominators, numerators, and units of measurement for performance measures. Other commenters concurred with the requirement to use protocols that are “consistent with” rather than “identical to” those developed by us to accommodate the rapidly changing field of quality assessment and improvement.

Response: Section 1932(c)(2)(A)(iii) of the Act required the Secretary in coordination with the National Governors Association, to contract with an independent quality review organization to develop protocols to be used in EQR. In planning for the development of the protocols, we had to determine the level of detail to be specified in each of the protocols. Because States have flexibility to choose what aspects of quality to measure and in order to accommodate different methodological approaches to studying quality, we contracted for the development of protocols that specified activities and steps of data collection and analysis that would produce valid and reliable information. These apply regardless of the data collected or the topics that States choose. Protocols will be considered “consistent” with ours to the extent that they affirmatively address each element specified in § 438.352, including the activities and steps for collecting data. We have revised the regulations under § 438.352(c) to clarify that instead of following “detailed procedures,” the EQR-related activities follow “activities and steps” specified for accurate, valid, and reliable data collection.

Comment: One commenter recommended that external review be required every 3 years rather than on an annual basis. The commenter noted that the National Committee for Quality Assurance (NCQA) requires a standard external review every 3 years and believes that this rule and the protocols should not set a standard more stringent than the industry standard.

Response: Section 1932(c)(2)(A)(i) of the Act clearly states that contracts “shall provide for an annual (as appropriate) external independent review.” We discuss later in this preamble why the parenthetical was not intended to modify what is otherwise an explicit requirement that EQR be conducted annually. An annual EQR has been a statutory requirement since 1986 under section 1902(a)(30)(C) of the Act. Pub. L. 106-113 made it clear that the provision was being replaced by 1932(c)(2) of the Act. We further note that the EQR described in this rule is very different from the accreditation review performed by NCQA. However, in the monitoring for compliance with the standards protocol that provides accreditation-like data, we only provide that information from a review of compliance with standards be generated every 3 years. This is consistent with the industry standard.

Comment: One commenter asked for confirmation that § 438.356(a) allows for EQR for a single MCO or PIHP to be performed by more than one EQRO.

Response: We are revising proposed § 438.356(a) to clarify that while we allow a State to contract with different EQROs to conduct EQR and EQR-related activities for a single MCO or PIHP, we believe and continue to require that the final analysis of all the information, as distinguished from the EQR-related activities, be performed by a single EQRO. This provides State flexibility to use different contractors to conduct different activities. Section 438.350 addresses the analysis and evaluation of information derived from mandatory and any optional activities. We believe that a single EQRO should perform this function to ensure that one entity receives all the available information and draws the overall conclusions about a particular MCO or PIHP. To clarify our intent to require that one EQRO perform the overall analysis (that is, conduct EQR) but that multiple EQROs may conduct EQR-related activities, we revised the language from the proposed rule to (1) remove the reference to “other related activities” in the definition of EQR, (2) add the reference to EQR-related activities to the definition of EQRO at § 438.320, and (3) add the reference to EQR-related activities to § 438.370 which provides for the 75 percent enhanced match. We also revised § 438.356(a) to clarify that States may only contract with one entity for EQR but may contract with multiple entities to conduct EQR-related activities.

Comment: One commenter recommended the addition of language allowing States the option to employ alternative quality assessment and improvement methods approved by CMS to substitute for the EQR requirements. The revised language should emphasize the State's responsibility under section 1932(c)(1)(A) of the Act to develop and implement a quality assessment and performance improvement (QAPI) strategy that includes, but is not restricted to, EQR-related activities. If CMS seeks to define minimum specifications for a State's QAPI strategy, those specifications should be set out in a proposed rule and subject to public review and comment.

Response: Our Medicaid managed care final rule outlined the elements of a State quality strategy, of which EQR is one element. States have the flexibility to determine how to ensure the quality strategy elements are designed and implemented. The public had the opportunity to review and comment on the proposed elements in the Medicaid managed care proposed rule published August 20, 2001 in the Federal Register (66 FR 43614). The EQR proposed rule addresses EQR in greater detail than does the managed care final rule, including what activities can be funded under the EQR enhanced matching rate. In this final rule, we describe optional EQR-related activities for which a State can obtain the enhanced Federal match under § 438.370. We believe we have provided States with the flexibility to design their EQR to best meet State needs while at the same time ensuring, through the three mandatory activities, that essential quality activities are conducted.

Comment: Several commenters recommended that we require that States coordinate their EQR with the State's quality strategy established under § 438.200 through § 438.204 of the Medicaid managed care rule and that EQR evaluate compliance with standards for quality, timeliness, and access in § 438.206 through § 438.242 of the Medicaid managed care proposed rule.

Response: We agree with the commenter. The Medicaid managed care final rule provides that an annual EQR be one element of a State's quality strategy. The EQR rule provides that information from a review of compliance with structural standards (including quality, timeliness, and access) be used in the EQR. Because of this we believe that the two rules together will require each State to coordinate its EQR with all other components of its State strategy.

Comment: One commenter agreed with our interpretation of the statutory provision requiring an external review annually “as appropriate” as being a reference to the deemed compliance provision. The commenter also suggested that reasons for not conducting a review be expanded to include (1) when the MCO is new and there are no historical records and (2) when the population of the MCO is too small to conduct a particular study.

Response: We disagree that newly contracting MCOs and PIHPs should not be subject to EQR. New MCOs and PIHPs will be required to meet structural standards, and we believe that information about MCO and PIHP compliance with these standards should be subject to EQR. We understand that the calculation of performance measures and the implementation of performance improvement projects require time to complete and may not be available at the time of the EQR. Therefore, while we acknowledge there are mandatory activities for EQR that may not be possible the first year of an MCO's or PIHP's operations, we do not agree that the MCO or PIHP should be entirely exempt from EQR. We also do not agree that small population size should be a reason to exempt an MCO or PIHP from EQR. Rather, the State, or MCO or PIHP if the State permits, should choose a performance improvement topic for which the entity has a sufficient number of enrollees to conduct a valid study.

Comment: Several commenters believe that the “as appropriate” parenthetical allows CMS the discretion to interpret EQR time frames more broadly and to give States discretion to require EQRs less frequently than annually. One commenter suggested that “as appropriate” modifies the word “annual,” not “review.”

Response: We do not believe that the Congress intended for us or the States to have discretion to provide for reviews less frequently than annually. As discussed above, section 1932(c)(2) of the Act replaces a statutory requirement for annual review that has applied since 1986. There is no indication in the legislative history that the Congress intended to change this. To the contrary, there is a persuasive alternative explanation for the Congress having inserted the parenthetical language. Section 1932(c) of the Act, unlike section 1902(a)(30)(C) of the Act has exemptions from the EQR requirement. Annual reviews for exempt entities are not appropriate.

Comment: One commenter interpreted the parenthetical to allow States to conduct reviews more frequently, not less frequently. If the EQR identified problems, the EQRO could be authorized to conduct follow-up evaluations, as appropriate, to ensure progress toward compliance.

Response: We do not agree with the commenter's interpretation because we believe that if problems are identified in the reports that the EQRO provides the States, the States can follow-up on any corrective action. Because we were not persuaded by any of the comments received for a different or additional interpretation of the parenthetical “as appropriate,” we are retaining in the final rule the interpretation that it refers to “deemed compliance” under section 1932(c)(2)(C) of the Act.

D. External Quality Review Protocols (§ 438.352)

In this section, we proposed that EQR protocols must specify: (1) The data to be gathered, that is, the substantive areas to be covered by the protocol; (2) the sources of the data; (3) detailed procedures to be followed in collecting the data to promote its accuracy, validity, and reliability; (4) the proposed methods for valid analysis and interpretation of the data; and (5) all instructions, guidelines, worksheets and any other documents or tools necessary for implementing the protocol. At the time the proposed rule was published, the protocols were under development. The strategy and timeline for protocol development were undertaken in response to BBA language that directed the Secretary to “contract with an independent quality review organization” to develop the protocols. The contract procurement process and scope of work necessitated that the protocols be completed after publication of the proposed rule. On November 23, 2001, we published a notice in the Federal Register (66 FR 58741) announcing the completion of the protocols and asking for comment on their burden or any other aspect of the protocols. Comments received on the November 23, 2001 Federal Register notice are addressed later in this preamble.

In developing the protocols, we instructed our contractor to draw from existing protocols that have been tested for reliability and validity and that have been used in the public and private sectors to conduct reviews of the quality of MCO and PHP services, consistent with current industry practice. We also expressed a preference for protocols that are in the public domain. The principle reason for not including the protocols in our regulation is because quality measurement is a rapidly changing field. The protocols must be revised regularly to reflect the changing state-of-the-art in quality improvement. Protocols developed in the private sector for validation of performance measures and administration of consumer surveys are usually revised annually. The delays inherent in revising regulations would make it difficult to make frequent changes. In addition, the protocols are detailed and lengthy, as they provide optional worksheets and recording documents in addition to the required activities and steps.

We proposed that all activities that provide information for EQR must be undertaken consistent with the protocols. Use of the CMS protocols or others consistent with ours will ensure that the conduct of the activities is methodologically sound, thereby maintaining a standard of quality for the review. However, by requiring protocols that are “consistent,” rather than “identical,” with those that we specify, we leave the States free to improve their protocols continuously, as the art and science of quality measurement improves.

Comment: One commenter asked that the protocols not pose an undue burden on physicians, clinical, or nonclinical personnel, noting that many physicians contract with more than one MCO and that duplicative information gathering should be avoided.

Response: EQR focuses on the MCO's and PIHP's structure and processes, and their ability to manage access to and provide quality services to Medicaid beneficiaries. The review process is not directed to individual physicians or other clinical or nonclinical personnel. However, it will be necessary for MCOs and PIHPs to request information from providers in order to conduct some of the activities required in this regulation. In recognition of the potential for burden, our request for proposal (RFP) to procure the development of the protocols specified that, “the protocols must be sensitive to the effect the burden to produce or provide additional data and information will have on organizations' ability to carry on their day-to-day operations.” We also specified that the protocols incorporate, as much as feasible, the tools, techniques, and methods to assess and improve health care quality already in place in the private sector. As a result, we believe the protocols impose the minimal additional burden necessary to carry out the statutory requirement.

Comment: In order to allow for parents to choose an MCO for their child on the basis of pediatric care, one commenter stated that the protocols should require that data on pediatric populations be analyzed apart from data on the MCO's adult population. The commenter also suggested that pediatricians and pediatric subspecialists have input into the development of the protocols.

Response: As required by statute, the protocols were developed by an independent quality review organization. In the scope of work for that contract, we required that the organization convene a panel composed of (1) current EQRO contractors; (2) CMS representatives; (3) State Medicaid agency directors, (4) managed care directors and quality system managers; (5) State licensure agencies; (6) advocacy groups; (7) health plans; (8) accrediting agencies; and (9) other experts in the area of quality improvement. A number of these panel members had experience with child health issues. We published a notice in the Federal Register on November 23, 2001 announcing the completion of the protocols and asking for comment on their burden. At the same time, the protocols were also made available on our website. The protocols are a methodologically sound set of generic instructions that will guide the reviewer in assessing quality. These instructions can be used for the entire Medicaid population in the MCO or PIHP or, in some instances, can be used for subpopulations such as children who receive Medicaid services. Some protocols address how MCOs, PIHPs, and States can stratify by specific populations, such as older adults or children with special health care needs. In addition, we note that States currently use many performance measures related to care for children. We, therefore, do not believe it necessary for the protocols to address pediatric populations apart from adult populations.

Comment: One commenter asked that we provide a definition for and examples of performance measures and performance improvement projects. One commenter agreed that we should not include the protocols in the proposed rule, given the dynamic state of quality evaluation and measurement. The commenter asked that we clarify what protocols for “calculating performance measures” means, that is to clarify whether it refers to protocols for the development of measures, the calculation of performance thresholds from reported measures, or some other EQR function.

Response: The definition and explanations of performance measurement and performance improvement projects are discussed in both the Medicaid managed care final rule and, in detail, in the protocols for calculating performance measures, validating performance measures, conducting performance improvement projects and validating performance improvement projects. In general, we refer to performance measurement as the calculation of the rate at which a desired event occurs. Readers are referred to the protocols available at http://www.cms.hhs.gov/​medicaid/​managedcare/​mceqrhmp.asp for further discussion.

Comment: Many commenters believed that the protocols should require MCOs to report on Americans with Disabilities Act (ADA) compliance issues for themselves and their providers to ensure that persons with disabilities have an opportunity to benefit from covered services that is equal to persons without disabilities.

Response: Compliance with the ADA provisions is addressed in the Medicaid managed care final rule and in the EQR protocol entitled Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs)—a protocol for determining compliance with the Medicaid managed care final rule provisions. It is the State's responsibility to ensure that its MCOs and PIHPs comply with Federal laws, including ADA.

Comment: Many commenters recommended that the sample for calculating performance measures, including baseline and follow-up measures for performance improvement projects, should be sufficient to look at specific measures of clinical care; and that the protocols should describe how reviewers will analyze the quality of care when data are missing. The commenters also believed that the protocols should require that MCOs use a common core of widely used, objective performance measures that are issued annually and revised as needed to reflect advances in performance measurement, that these measures and their methods of calculation be publicly available, and that they include measures for persons with special health care needs. The commenters also recommended that MCOs be required to (1) collect specified HEDIS measures; (2) conduct the Consumer Assessment of Health Plan Study (CAHPS) survey; and (3) conduct a focus study annually of specialized services to persons with special health care needs. The EQR should evaluate these measures in making findings on the quality of care. Finally, the commenters asked that instructions be provided on how to adapt the measures to FFS and PCCM settings and for those enrolled less than 12 months.

Response: As stated before, the protocols are a set of methodologically sound generic instructions that will guide a reviewer in assessing quality. The protocols include instructions on proper sampling methodology, assessing missing data, and processes for analyzing data. The protocols do not specify which performance measures are to be used. Performance measures are chosen by the State or MCO or PIHP and will vary over time. The Medicaid managed care final rule gives us the authority to require specific performance measures and levels if we decide to do so in the future. The results of the EQR, however, will be made available to the public upon request and will identify the specific measures collected, the technical methods of data collection and analysis, and the conclusions drawn from the data.

The BBA placed the requirement for EQR on capitated managed care programs, but not on FFS or PCCM settings. Therefore, we do not in this rule provide an explanation of how to adapt these activities to the FFS/PCCM environment. We do, however, encourage States to address the quality of care provided in these service delivery systems. Through a new partnership initiative with State Medicaid and State Children's Health Insurance Programs (SCHIP), we will be discussing how best to apply performance measures to these two delivery systems.

Comment: One commenter asked that we retain the ability of State agencies to continue to improve the protocols as advancement occurs in the art and science of quality measurement. Several commenters stated that because the protocols may quickly become out of date because the field of quality improvement is constantly changing, they should not be promulgated as regulation. These commenters were concerned about CMS developing detailed and lengthy protocols instead of either guidelines for States or streamlined protocols that specify only the basics for ensuring statistically sound, reliable, and valid results. One of these commenters stated that our intent appears to limit State flexibility and suggested that CMS significantly simplify the protocols to ensure feasibility for State agencies. This commenter also asked that CMS obtain State input on the draft protocols.

Several commenters believed that CMS should require that States use the protocols. One commenter felt that the proposed rule allows States to develop their own external review protocols. This commenter asked CMS to mandate the use of the protocols in order to comply with section 1932(c)(2)(A)(iii) of the Act which directs the Secretary to “* * * contract with an independent quality review organization to develop protocols to be used in external reviews conducted * * *” The commenter asserted that mandating the protocols would promote efficiency, lessen burden on the States, and promote the development of standardized data and information about services provided in Medicaid managed care.

Response: This regulation provides States with the option to use the protocols developed by us or protocols that are consistent with our protocols. We believe that by allowing States to use “consistent” protocols, States will be able to improve the protocols over time as the state-of-the-art advances and at the same time ensure that reliable and valid methods are used when conducting EQR-related activities.

The protocol documents include a discussion of the activities and steps necessary to soundly conduct the quality assessment function addressed by each protocol. In addition, each protocol includes guidance on how to implement the essential elements of the protocol as well as optional worksheets and appendices that States may use at their discretion. The activities and steps contained in the protocols are generic, relatively brief, but contain the essential components for a methodologically sound review that the statute envisions. Therefore, we believe that the protocols allow for State flexibility while ensuring the methodologically sound and valid EQR.

Comment: Several commenters noted that it is difficult to determine the full extent of the impact of the protocols on EQR activities until they are published. These commenters stated that they hope the protocols will respect States' individuality and provide flexibility whenever possible to allow for tailoring of EQR activities to local conditions and circumstances. One commenter further stated that there are many clinical guidelines and protocols that are already published, easily available, and in current use (for example, those developed by the Agency for Health Care Policy and Research (AHCPR) now the Agency for Health Care Research and Quality (AHRQ), American Heart Association, etc * * *) that are not mentioned in the proposed rule.

Another commenter stated that the protocols should be subject to full public scrutiny because they carry the full weight of the regulation. The commenter believes the protocols significantly exceed both the intent of the Congress in the BBA and the proper role of this regulation. Specifically, the commenter noted that the statute does not specify the activities that the protocols should address or other details included. The commenter was also concerned that States will find the 75 percent match for EQR activities a strong incentive to outsource this function, which the commenter believes appropriately rests with the government. As a result, this commenter believes that activities now done by the State according to locally developed protocols will be shifted to contract staff to be performed using externally derived standard protocols.

Another commenter asked that current State practices not be totally dismissed and that consideration be given to the quality improvement system for managed care (QISMC) standards and how they can be incorporated into the EQR process.

Response: We published a notice in the Federal Register on November 23, 2001 (64 FR 58741) announcing the completion of the protocols and asking for comment on their burden. At that time, the protocols were also made available on our website. Comments on the protocols and our responses are incorporated in this preamble. We believe the protocols are generic and can be used by all States. They are not clinical protocols like those published by AHCPR (now AHRQ), the American Heart Association, and other organizations. We believe that the protocols are consistent with the intent of the Congress in the BBA. We also note that we have provided States with great flexibility to conduct all EQR-related activities, allowing States to perform EQR-related activities either themselves or through the use of contractors, as long as they are performed consistent with our protocols. While the enhanced Federal financial match for EQR-related activities is not available under the statute if conducted by State personnel, other provisions of Medicaid law provide for enhanced Federal financial match for qualified medical activities when conducted by State staff who qualify as skilled and professional medical personnel.

The protocols are based on existing protocols already in use in the public and private sector. The contractor used QISMC guidelines as well as other public and private sector protocols in developing all the protocols. With respect to the QISMC standards (as opposed to their interpretive guidelines) we note, for Medicaid, that the QISMC standards were superceded by the Medicaid managed care final rule. QISMC standards are no longer current for the Medicaid program. For each protocol developed, specific information can be found in the protocol regarding which public and private sector protocols were reviewed and the extent to which they were incorporated.

Comment: One commenter was concerned that the JCAHO does not have a traditional background in this area and may take a different approach than NCQA.

Response: The BBA specified that the protocols be developed by an “independent quality review organization.” The JCAHO was selected through an open competitive procurement process, which required them to provide evidence of their experience in protocol development. In addition, they developed the EQR protocols using existing protocols widely used in the public and private sector, including protocols used by national accrediting organizations, and national consulting firms which have developed quality measurement tools for us in the past.

Comment: One commenter asked if health plans will have to create an entirely different audit response to the protocols in addition to responding to the existing standards of NCQA and of other State entities.

Response: Because the protocols were based on quality assessment approaches already in use by public and private quality oversight organizations, we believe that the methods MCOs and PIHPs use to respond to existing private and public sector audits will be able to be used to respond to EQR. In addition, the nonduplication provisions under § 438.360 are revised in the final rule to allow States in certain circumstances to exempt both Medicare+Choice (M+C) organizations and MCOs and PIHPs meeting standards of national accrediting organizations approved and recognized by CMS for M+C deeming from compliance with some structural standards.

Comment: One commenter stated that the protocols being developed are, in fact, EQR-related activity protocols and that there does not appear to be any protocol that will guide the analysis and evaluation of the data and information provided by these EQR-related activities. This may cause the analysis and evaluation to vary due to lack of equivalent specifications for these processes. The commenter recommended that the rule more clearly define requirements for EQR and distinguish between EQR and EQR-related activities.

Response: The commenter is correct that we do not provide a protocol for the analysis and evaluation of information provided as a result of the EQR activities in the aggregate. We do not believe that we should develop a protocol for the analysis and evaluation of all EQR information. The information derived from EQR activities will vary enormously. For instance, the variation in the types of services provided and the populations covered under the MCO and PIHP contract will impact the performance measures chosen and performance improvement projects to be conducted. Other activities are optional for States. The approach to analysis depends upon the findings of the individual EQR-related activities and we expect these findings to be as individual as the MCOs and PIHPs being reviewed. Therefore, we do not believe that we can adequately predict all the possible variations of information that will be provided to an EQRO and, therefore, we do not provide for a protocol on how to conduct an analysis and evaluation of this information. We believe it is more appropriate for us to require that the activities that provide information for the analysis and evaluation be done in a methodologically sound manner. We do specify qualifications for EQROs and thereby believe that EQROs will have the skills necessary to perform qualitative and quantitative analysis of EQR-related information and draw proper conclusions. In addition, each EQRO must provide results as specified in § 438.364 that include a technical report specifying the objectives of, methods used, description of data obtained, and conclusions drawn from the EQR.

Comment: Many commenters were concerned that there has been no public review process for the protocols and that the meetings of the expert panel have been closed to the public. The commenters recommended that the public have the opportunity to review and comment on the draft protocols, that the protocols be issued annually, and the public have the opportunity to comment on any changes to the protocols. The commenters also stated that the protocols should be made publicly available on the CMS website. Several commenters asked that we provide an opportunity for interested parties and the public to comment on the protocols. They noted that providing the opportunity for all affected entities to review and provide comment on the protocols before they are finalized will allow for a better quality product and lend credibility to the protocols. One of the commenters further noted that even though CMS convened an expert panel to review the protocols as they were being developed, consumer participation was very limited.

Response: As stated earlier, on November 23, 2001, we published a notice in the Federal Register announcing the completion of the protocols and requesting comment on their burden or on any other aspect of the protocols. Comments on that notice and our responses to those comments are incorporated into this preamble. We will be publishing a notice in the Federal Register every 3 years on the protocols as required by the Paperwork Reduction Act. This notice will provide the opportunity for the public to comment on the burden or any other aspect of the protocols. The protocols are available to the public on the CMS Web site at http://www.cms.hhs.gov/​medicaid/​managedcare/​mceqrhmp.asp.

Comment: One commenter requested that in developing the protocols, JCAHO take into consideration that some factors that affect MCO performance are not within the control of the MCO, such as instability in eligibility status and changes in the characteristics of the enrolled Medicaid population.

Response: We agree that measuring performance on the Medicaid population needs to take into account issues such as changes in eligibility status. The protocol on performance measures recognizes those issues.

Comment: Because of the length of the protocols and the need to change them on an ongoing basis, one commenter requested that we clarify that the protocols be issued as guidelines rather than requirements and that we clarify the flexibility States will have in implementing them.

Response: Section 1932(c)(2)(A)(ii) of the BBA requires that protocols be used in the conduct of EQR activities. We provide States the option to use our protocols or protocols consistent with those we develop.

E. Qualifications of External Quality Review Organizations (§ 438.354)

Section 438.354 of the proposed rule set forth the requirements that an entity would be required to meet in order to qualify as an EQRO under the new BBA external review provisions in section 1932(c)(2) of the Act. The proposed rule did not specify categories of entities that would be qualified to perform EQR under section 1932(c)(2) of the Act. This is a departure from the existing external review requirement in section 1902(a)(30)(C) of the Act (which will no longer be in effect when these final regulations are implemented), under which only certain entities could perform external review. (These entities were: (1) A “quality improvement organization” (QIO) that contracts with Medicare to perform review (QIOs were formerly known as quality control peer review organizations, or “PROs”); (2) an entity that meets the requirements to contract with Medicare as a QIO; and (3) a private accreditation body. Only contracts with the first two categories were eligible for a 75 percent matching rate under the pre-BBA rules.)

Under proposed § 438.354, in order to qualify, entities would be required to meet specified competence and independence standards. We proposed two tests of independence. Under the first proposed test, the EQRO and any subcontractors would have to be independent from the State Medicaid agency and from any MCO or PHP they review. Second, the relationship between the MCO/PHP and the EQRO could not involve any potential conflicts of interest. We specifically requested comments on (1) how better to identify situations that create conflict of interest; (2) the proposal to allow State entities to qualify as EQROs; and (3) our decision in the proposed rule to apply the “independence” requirement to subcontractors as well as contractors.

We also proposed that EQROs be selected by State agencies through an open, competitive procurement process. As noted in the preamble to the proposed rule, CMS would not, under our proposal, approve EQR contracts. However, contracts entered into by the States would be subject to review to ensure that, as a condition for FFP at the 75 percent rate, the State agency followed all applicable procedures and criteria. This proposed procedure is consistent with current practice, which is for State agencies to use competitive procurements to select EQROs that perform review under section 1902(a)(30)(C) of the Act. It is also standard practice for our regional office staff to monitor implementation of Medicaid managed care initiatives. For EQR, regional office staff may review the State's most recent RFP for external review services, the EQR contract, or the EQR reports.

Comment: One commenter asked that a review of the current EQR process under section 1902(a)(30)(C) of the Act be performed by an independent review body to assist the Secretary in deciding whether current contractors are performing adequately.

Response: Section 1932(c)(2)(A)(ii) of the Act clearly instructed us, in consultation with States, to establish a method to identify entities qualified to conduct EQR. We chose to pursue a method that would allow States to have access to the greatest number of entities with the qualifications necessary to perform EQR and EQR-related activities. Therefore, we did not limit ourselves to a review of current contractors permitted to perform review under section 1902(a)(30)(C) of the Act, but attempted to discern all types of contractors that States have found capable of performing EQR-related activities. We believe this will provide States with much needed flexibility to promote greater competition and improvement among potential EQR contractors.

Comment: One commenter supported the provisions in the proposed rule that allowed for a variety of organizations to serve as an EQRO, but cautioned that EQRO criteria should include an unbiased approach to managed care. The commenter expressed concern that an anti-managed care organization could be awarded the contract, and that this would adversely affect the organization's ability to objectively make an assessment of MCO strengths and weaknesses and making recommendations for improvement.

Response: A State may contract with any entity to conduct EQR as long as the entity meets the competency and independence criteria. EQR is an important component of a State's quality strategy, and we trust that States will select entities to conduct EQR that will perform objective reviews.

Comment: Many commenters supported this provision because it provides States with more flexibility to contract with a range of organizations while still obtaining the 75 percent matching rate currently limited to contracts with QIOs, and entities that meet the requirements to contract as QIOs. Several of these commenters specifically supported the competence and independence standards proposed. One commenter agreed that the regulation should require organizational qualifications.

One commenter, however, found the requirements vaguely defined, and recommended that we stipulate additional requirements, such as proper licensure or certification from accrediting organizations for performance of validation of performance measures and surveys. Another commenter expressed concern that the proposed competency criteria would encourage the use of entities that are less qualified than the QIOs with which most States currently contract. The commenter believed that QIOs as nonprofit organizations, were independent, objective, and had access to needed physicians and experience in quality improvement. The commenter recommended that § 438.354(b)(1) be revised to read, “require an organization to have staff with appropriate credentials and demonstrated experience.”

Response: The BBA required us to work in consultation with States to establish a method for the identification of entities qualified to conduct EQR. We believe that had the Congress desired to retain the three categories of entities allowed to perform EQR under section 1902(a)(30)(C) of the Act, it would have done so. Similarly, the Congress could have easily stated that only QIOs should perform EQR. The Congress chose neither of these approaches, but instead asked us to establish a method to identify qualified entities. We believe that the Congress chose to respond to States' frequently stated desires to have a greater range of organizations with which to contract. Therefore, under the auspices of the National Academy for State Health Policy (NASHP), we worked with States, consumer advocates, and other stakeholders to provide us with their recommendations on a methodology to identify qualified entities. Many commenters strongly supported the competency provisions we proposed under § 438.354(b). Therefore, the final rule retains these requirements from the proposed rule. We leave it up to States to determine if they would like to impose additional requirements such as certified vendors. We agree that demonstrated experience should be required of an EQRO, and in response to this comment, we have changed § 438.354 (b)(1) to require staff with demonstrated experience.

We also made some revisions to proposed § 438.354(a) to clarify that these provisions apply to those entities a State contracts with as an “EQRO,” regardless of whether the EQRO performs EQR or specific EQR-related activities.

Comment: One commenter felt that the proposed conflict of interest requirements failed to recognize that since the State contracts with the EQRO, the EQRO would be reluctant to tell the State what it may not want to hear. The commenter recommended having the EQRO funded by an external Federal agency, such as AHRQ (formerly AHCPR), or to require or create financial incentives to have the State report on comparable performance measures for all MCOs licensed in the State.

Response: Section 1932(c)(2) of the Act explicitly requires States that contract with Medicaid MCOs to provide for an EQR of each MCO, and provides for an enhanced Federal match rate for this review. We believe that it is clear that the Congress intended that States share the costs of EQR, and be the contracting party. We do not agree with the commenter's assumption that the State will not want to be informed if an MCO or PIHP is not performing adequately. We believe the provisions in this rule will encourage States to use EQROs to conduct numerous quality activities, both because of the flexibility that the rule provides to States, and because of the availability of the 75 percent enhanced match for these activities without regard to whether the entity performing review is a QIO or meets the requirements to contract as a QIO.

Comment: One commenter requested that EQROs be required to include clinical staff with pediatric training in order to be qualified to review a Medicaid MCO. One commenter recommended that the entity be required to have staff with knowledge of section 504 of the Rehabilitation Act of 1973, and of titles II and III of the ADA, based on the commenter's research suggesting that individuals who have mobility impairments routinely encounter physical barriers to care. The commenter's research also indicated that access to preventive care was significantly lower for individuals who use wheelchairs, and few PHPs know which of their clinicians are accessible to patients with mobility or sensory impairments.

Response: We do not agree that it is necessary to include specific requirements for EQROs to have clinical staff with pediatric training in order to qualify to review an MCO or PIHP. Section 438.354(b)(3) requires that the organization have the clinical skills necessary to carry out the EQR activity, which we believe requires that the EQRO or its subcontractor have the necessary training. We also do not agree with the commenter's suggestion that we specifically require an entity to have staff with knowledge of the Rehabilitation Act or the ADA. While MCOs and PIPHs are required to comply with these laws, there are separate enforcement mechanisms for ensuring compliance with their provisions. We note that it is the responsibility of an EQRO to assess the MCO's or PIHP's ability to provide access to services in a timely manner. If this is accomplished for all enrollees, this would, in effect, constitute compliance with these laws. Through its review of compliance with State-established structural standards, as required in § 438.358(b)(3) of the final rule, the EQRO must ensure that Medicaid beneficiaries, including those who are disabled, do not encounter barriers to care.

Comment: One commenter suggested modifying proposed § 438.354(b)(1)(iii) to read “* * * include quality assessment and improvement technologies and methods.”

Response: We agree with the commenter's suggestion that the word “methods” be used and believe that this term already encompasses technologies that may be employed by the State as a method for assessing and improving quality. Accordingly, in response to this comment, we are revising § 438.354(b)(1)(iii) to use the word “methods.”

Comment: One commenter supported our proposal to allow State agencies to qualify as EQROs in certain situations. Another commenter believed it would also be appropriate for the State HMO licensing organization to be eligible to be an EQRO. Conversely, one commenter felt that EQROs should be independent of most State agencies, particularly Medicaid purchasing or managed care licensing authorities. Another commenter believed that it was extremely important that the definition of independence be explicit for State Medicaid agencies, and that CMS's regional offices should review determinations as to the independence to make sure that true independence is obtained. This was based on concern over what the commenter saw as an inherent conflict of interest permitted under our proposed rule. In the commenter's view, this conflict arises from the fact that State agencies, departments, and universities are ultimately accountable to State legislatures and the Governor who act on purchasing decisions made by the State Medicaid agency, and who appoint members to boards of these entities. One commenter expressed the view that no State agency is truly independent and recommended prohibiting State entities from serving as EQROs.

Response: Section 1932(c)(2) of the Act requires that a State contract with an independent organization in order to get the enhanced 75 percent FFP for EQR. The expert panel composed of State representatives, advocacy organizations, and other stakeholders that was convened under the auspices of the NASHP recommended that we allow State agencies to qualify under certain circumstances as EQROs. Because we agree with this recommendation and believe it to be reasonable with the safeguards on independence we have in place, the final rule retains the independence requirements that permit State Agencies under certain circumstances to qualify as EQROs. We note that we have received only a few comments opposing our proposal to let State entities qualify as EQROs. CMS regional office staff will assess the EQRO contracts to ensure compliance with the provisions of this rule as part of regular monitoring reviews.

Comment: One commenter did not agree with the requirement that a State entity be governed by a board or similar body, the majority of whose members are not government employees, in order to qualify as an EQRO. The commenter believed that State universities should be permitted to be EQROs because they can produce high quality work for significantly less cost than QIOs.

Response: We understand that the requirement will limit the number of State entities that can qualify as EQROs, including some State universities. We took this recommendation from the expert panel convened under the auspices of the NASHP. This panel included State licensure and Medicaid representatives. We are aware that several States have State entities that meet the criteria set forth in the proposed rule. We have received minimal comments opposing this provision. We conclude that this is a feasible arrangement, and think that the provisions related to the governing board are appropriate and necessary in order to fulfill a requirement for meaningful independence. We also believe it represents a reasonable compromise between banning State entities altogether, and allowing any entity to serve as an EQRO. Therefore, the final rule retains the governing board provision.

Comment: One commenter representing a Medicaid program not operating in the continental United States felt that the proposed independence criteria would have the effect of precluding all of its governmental procurement possibilities related to EQR. The commenter recommended that the independence criteria be waived, or that implementation be postponed, due to the financial burden the commenter believed that the rule would impose on it because it would have to contract with EQROs in the continental USA.

Response: The statute requires that the EQRO be an independent entity. Consistent with the interpretation of “independence” under the existing external review requirement in section 1902(a)(30)(C) of the Act, we interpret this to mean independent from both the MCO/PIHP and from the State. Thus, it is not clear how this final rule would create a financial burden by referring a contract with an outside entity, since this is already required. We do not agree that exceptions should be made based on a Medicaid program's ability to contract with an EQRO locally. We recognize that many State agencies, departments, and universities do not meet these criteria. However, as noted above, several States do have State entities that meet the independence criteria. We also note that this regulation provides more flexibility than in the past for a variety of organizations to qualify as EQROs.

Comment: One commenter disagreed with our proposal to apply the independence requirement to subcontracts, suggesting that this would result in States being unable to take advantage of the experience of nationally renowned experts affiliated with academic health centers that have ownership interests in MCOs that serve Medicaid beneficiaries. In contrast, one commenter endorsed applying independence criteria to EQRO subcontractors as balanced and reasonable.

Response: The independence provisions are broad enough to allow for a variety of organizations to qualify as EQROs and a variety of experts to subcontract with EQROs. In formulating the provisions, we sought balance between providing flexibility to States to choose from numerous qualified entities, and ensuring that entities were sufficiently independent from the State and the MCOs and PIHPs. We realize these requirements will limit some contracting opportunities when experts or the organizations for which they work do not meet the independence criteria.

Comment: Many commenters agreed with the expert panel recommendation that the EQRO should not share management or corporate board membership with the MCO it reviews. The commenters also suggested that the individuals employed by the EQRO or subcontracting with the EQRO should be free of any potential conflicts of interest with the MCO that they review.

Response: In the preamble of the proposed rule, we explained that we did not solely rely upon the recommendation that an EQRO should not share management or corporate board membership with the MCO it reviews, because we do not think this criterion is stringent enough to ensure against conflict of interest. Therefore, we incorporated in § 438.354(c)(3)(i), the concepts of “control” in 48 CFR 19.101, which effectively preclude affiliation between the EQRO and the MCO/PIHP under review. Specifically, this means that there can be no control through common management (which includes interlocking management, common facilities, and newly organized concerns) as well as through stock ownership, stock options and convertible debentures, voting trusts, and contractual relationships (which includes joint ventures, that is, procurement and property sale assistance and franchise and license agreements). We retain this provision in our final rule. In order to provide further clarification in § 438.354(c)(3)(i) of the final rule (§ 438.354(c)(3) of the proposed rule), we now specify the different types of control addressed in § 19.101. In determining whether this type of control exists, the details in § 19.101 under each category would apply.

Comment: Several commenters recommended strengthening the requirements for EQRO independence from MCOs by revising § 438.354(c)(3) to read as follows: “A private entity may not (1) have managed care licensing authority, including the authority to certify managed care plans in compliance with standards that serve as the basis for deemed certification with Federal or State regulatory standards; (2) deliver any health care or related services to Medicaid recipients for which it is paid by the Medicaid State agency or by a managed care plan. Related services include enrollment services, grievance resolution, external review of health care coverage decisions, or other similar activities; (3) conduct, on the State's behalf, any other ongoing Medicaid program operations related to oversight of the quality of MCO services; and (4) have financial interest that would prevent it from exercising independent judgement when engaging in EQRO activities.” The commenters also suggested adding a new § 438.354(c)(4) providing that “a private entity must be governed by a board or similar body, the majority of whose members are not MCO employees.” Another commenter did not agree with the provision that prohibits an organization from performing EQR if it also conducts ongoing Medicaid program operations related to quality, arguing it could be less expensive to use a single contractor to perform multiple functions. One of the commenters found the definition of control in 48 CFR 19.101 a useful concept, but felt that it has little relevance to the potential organizational relationships between EQROs and MCOs in the Medicaid program.

Response: The independence criteria set forth in the proposed rule did not address those private organizations that provide health care services to Medicaid beneficiaries or that conduct ongoing Medicaid program operations related to quality. We agree with the commenters that organizations performing these functions have a conflict of interest. Therefore, in response to this comment, we are revising § 438.354(c)(3)(ii) in this final rule to preclude private organizations, as well as State entities, that provide health care services to Medicaid beneficiaries from qualifying as EQROs. We also are revising § 438.354(c)(3)(iii) to preclude private organizations as well as State entities, that conduct ongoing Medicaid managed care operations related to quality from qualifying as EQROs. We narrow the scope of this provision from entities that conduct program operations to entities that conduct managed care related operations in order to allow States to contract with entities that conduct quality activities for the States such as FFS medical and utilization review activities. We agree with the last commenter who agrees that it will be more efficient for States to use a single contractor to perform multiple functions; therefore, we intend to allow entities that conduct limited quality activities such as providing technical assistance to States in the collection of encounter data or who assist the State in other quality improvement areas to qualify as an EQRO. These activities would not be considered ongoing operations conducted on behalf of the State.

We do not permit an entity to qualify as an EQRO if that entity conducts activities that State staff would otherwise conduct in Medicaid managed care program operations related to quality oversight. As an example, a State university or consulting firm that designs and implements or has significant responsibility for the State's Medicaid managed care program operations would not qualify as independent.

We do not agree with the commenter who recommended that the independence provisions should preclude any organization from being an EQRO that has the authority to certify managed care plans in compliance with standards that serve as the basis for deemed certification with Federal or State regulatory standards. These organizations, while they may provide services under contract to a State, follow their own independently set standards and procedures. We believe that States should be permitted to contract with these organizations to consolidate review processes. This is consistent with congressional intent as indicated by the nonduplication and deemed compliance provisions in sections 1932(c)(2)(B) and (C) of the Act.

As stated above, we agree with the commenters' suggestions to revise the independence criteria as it applies to private organizations that deliver health care services to Medicaid beneficiaries or who, on behalf of the State, conduct Medicaid managed care program operations related to quality. However, we do not agree with the commenters' suggestions to add to this provision health care-related services such as enrollment services, grievance resolution, and review of health care coverage decisions. We leave it to the States to determine if health care-related services are Medicaid managed care program operations related to quality, in which case the organizations would be precluded from qualifying as an EQRO. In addition, States have the flexibility to adopt a more strict standard for “independence” if they wish and to deny entities that provide any health care-related services from contracting as an EQRO.

We agree with the commenters' suggestions that the final regulation include a provision to prohibit an EQRO from having a financial interest that would prevent it from exercising independent judgement when engaging in EQRO activities. The types of “control” addressed in 48 CFR 19.101 address financial relationships involving such things as stock options and convertible debentures. To be consistent with other CMS regulations, however, and in order to respond to this comment, we believe the financial relationship between organizations must be addressed in the conflict of interest requirements. Therefore, we revised § 438.354(c)(3)(iv) to address direct and indirect financial relationships. We also have added a definition for financial relationships under § 438.320.

We believe the language in proposed § 438.354(c)(2) addresses the suggestion by one commenter that we add a provision requiring a private entity to be governed by a board or similar body, the majority of whose members are not MCO employees. By referencing 48 CFR 19.101, specifically § 19.101(f)(1), a concern is considered controlling through interlocking management if officers, directors, employees, or principal stockholders serve as a working majority of the board of directors or officers of another concern. As noted above, to provide clarification, the final rule under § 438.354(c)(3)(i) (§ 438.354(c)(3) of the proposed rule) specifies the elements that constitute control of one entity over another as those in 48 CFR 19.101.

Comment: Several commenters expressed support for our independence requirements. One commenter supported our proposal to allow States to contract with entities that possess the necessary skill and expertise to conduct the mandatory and optional EQR activities, but suggested that we query State agencies for specific citations or contract language that they have used to define independence, or for concrete examples of situations that may create conflicts of interest. The commenter also suggested that we consider delineating specific competence standards for each of the mandatory activities. One commenter agreed that it is critical for CMS to establish a set of criteria to which States must adhere when selecting EQROs.

Response: At the expert panel meeting convened under the auspices of the NASHP, we asked the panel for recommendations on how to define conflict of interest. This panel included State representatives as well as representatives from advocacy organizations and other stakeholders. The expert panel recommended that independence be established by requiring the disclosure of any ownership interest of greater than 5 percent of the entity seeking to become an EQRO. As was discussed in the proposed rule, we believe this “disclosure of ownership” requirement is inadequate to ensure independence, first, because is does not preclude an entity from being an EQRO but only requires disclosure of the financial interest, and second, because there may be other types of conflicts such as interlocking management, common facilities, and so forth. Moreover, in the proposed rule, we requested comments on how better to identify situations that create conflict of interest. As noted above, we made some changes based on comments we received.

We do not believe that it is necessary for us to revise the competency requirements to address each EQR activity. The criteria outlined in the proposed rule were intentionally broad to provide States with the flexibility to contract with one or multiple entities that have the skills necessary to conduct the particular activity/activities under contract. For example, if a State wants to have one of its EQROs conduct only encounter data validation, to meet the requirement under § 438.354(b)(3), the EQRO would not need to possess the clinical skills but would need the “nonclinical skills” in its organization (or through a subcontract) to conduct encounter validation.

Comment: A commenter believed that the proposed rule did not make clear who, specifically, would be responsible for designating an entity as an EQRO. The commenter recommended that this responsibility rest in our Office of Clinical Standards and Quality, as it already has oversight responsibility for Medicare's Health Care Quality Program.

Response: Under this rule, States are required to select and thereby designate EQROs through an open, competitive procurement process. CMS will not be designating EQROs, as it currently does in the case of QIOs and entities claiming that they meet the standards to contract as a QIO. When monitoring State Medicaid managed care programs, CMS regional office staff have the opportunity to review RFPs, contracts, and EQR results to ensure compliance with the EQR provisions.

F. State Contract Options (§ 438.356)

This section set forth proposed requirements State agencies would be required to follow, and options that they would have selecting EQROs. We proposed that State agencies may contract with more than one EQRO. The final rule in § 438.356 (a)(1) and (a)(2) reflects clarifications made to the provisions based on comments discussed in an earlier section of the preamble.

We also proposed that each EQRO be permitted to use subcontractors. EQROs that use subcontractors are accountable for, and required to oversee, all EQR activities performed by the subcontractors. In addition, we proposed that each EQRO be required to meet the competency requirements, and each EQRO and EQRO subcontractor be required to meet the independence requirement. We also proposed that State agencies follow an open competitive procurement process that is in accordance with State law and regulation and consistent with 45 CFR part 74, as it applies to State procurement of Medicaid services.

Comment: Several commenters supported the language in § 438.356 as proposed. One commenter specifically agreed that all subcontractors should be required to meet the test of independence, and that the contract must be procured through a competitive bid process.

Response: We appreciate the commenter's support for the provisions, and have retained them in the final rule.

Comment: One commenter believed that a competitive bidding process was the most appropriate way for States to secure efficient cost-effective reviews.

Response: We agree that competitive bidding provides the best means to select a qualified contractor at the best price, and we retain the requirement for competitive procurement of EQROs in the final rule.

Comment: One commenter asked us to clarify whether the State Medicaid agency could contract directly with a State organization without using a competitive procurement process if the State organization otherwise meets the standard of being “independent,” and meets the requirements of a qualified EQR.

Response: The Department of Health and Human Services has regulations governing the extent to which States are required to competitively procure contracts. Those regulations apply to EQRO contract as cited under § 438.356(e).

G. Activities Related to External Quality Review (§ 438.358)

Section 438.358 proposed a requirement that EQR utilize information obtained from specified mandatory activities that must be performed by the State agency, a State agent, or the EQRO. Proposed § 438.358 also identified optional activities that the State agency or its agent may perform, or have the EQRO perform, to produce additional information for use in EQR. The mandatory activities are consistent with the requirements set forth in the Medicaid managed care final rule. The optional activities were not included in that rule. They are, however, activities that States have had their EQR contractors perform in the past.

We proposed that each year, the EQRO must use information obtained from the validation of performance improvement projects performed that year, and the validation of performance measures reported that year, by the MCO. To be consistent with the private sector, however, we proposed that information used by the EQRO from a review of MCO and PHP compliance with State structural and operational standards be from the most recent review performed within the previous 3 years.

Proposed § 438.358 also would allow States to have their EQROs provide technical guidance to groups of MCOs and PHPs to assist them in conducting the mandatory and optional EQR-related activities.

Comment: One commenter requested that States be required to provide technical support to MCOs to ensure that pediatric measures are implemented. The commenter also expressed a concern that the proposed EQR regulations did not separately address children with special health care needs, noting that it was critical that CMS require State Medicaid managed care programs to provide adequate protections and considerations for these children.

Response: States have the flexibility to provide technical support to MCOs and PIHPs on pediatric measures as well as generic measures, preventive care measures, measures for disabled adults, or any other measures. This rule does not require this technical support, however, because we do not believe that it would be necessary in all cases.

With respect to special needs children, this regulation implements the BBA EQR provisions by specifying who is qualified to conduct EQR and what information should be included in such a review. The Medicaid managed care final rule requires States to have quality strategies that must include procedures that assess the quality and appropriateness of services provided to all Medicaid enrollees under MCO and PIHP contracts. This includes children with special health care needs. The EQR will evaluate activities undertaken by MCOs and PIHPs in accordance with the State strategies. States can elect to have their MCOs and PIHPs determine what measures to collect or States can require MCOs and PIHPs to collect specified measures appropriate to the populations served.

Comment: One commenter strongly recommended that these regulations mandate that States require MCOs to develop and administer a provider satisfaction survey. The commenter thought this would allow the MCOs to use the results of the surveys to identify additional approaches to enhance quality of care. It also would allow States to identify MCOs that may be poised to experience a rapid withdrawal of providers, which could place beneficiaries at risk of having difficulty accessing care, or otherwise disrupt their medical home. Another commenter felt that the validation of consumer or provider surveys would be difficult. This commenter asked whether we were proposing that EQROs contact respondents to ask them if the answers that were recorded were the answers given.

Response: This rule does not require that States have their MCOs and PIHPs develop or administer consumer or provider surveys. It does, however, allow States to have their EQRO administer or validate a consumer or provider survey, and receive the 75 percent enhanced match for this activity as long as the EQR survey protocol or a consistent protocol to the one we developed is used. The EQR survey protocol does not require that respondents be contacted to validate survey responses. We agree that this would be costly and burdensome. The survey protocols outlines generic steps that must be followed to ensure reliable and valid methodological approaches to administer and validate surveys.

Comment: One commenter recommended that we require that EQROs measure and report the participation of pediatricians, pediatric medical subspecialists, and pediatric surgical specialists when conducting activities related to the establishment of provider networks.

Response: EQRO reviews for compliance with structural and operational standards will include a review of the delivery network. The review will ensure, consistent with the Medicaid managed care final rule, that MCOs and PIHPs maintain and monitor a network of appropriate providers to furnish services covered under the contract and that they consider the anticipated Medicaid enrollment with particular attention to the needs of enrolled children; the expected utilization of services; and the geographic location of providers and enrollees. When developing and maintaining their provider network, MCOs and PIHPs will also need to consider the characteristics and health care needs of enrollees.

Comment: One commenter believed that while it arguably was reasonable to require external auditing of broad, publicly disclosed quality performance measures, the same mandate should not be imposed on other quality improvement data such as the findings of focused clinical studies. In this commenter's view, these types of data are intended to promote MCO self-assessment and stimulate quality improvement activities, and should not be subject to an external audit.

Response: We do not agree with the commenter that the findings of focused studies or other quality improvement projects should not be subject to an EQR. Our Medicaid managed care final rule requires MCOs and PIHPs that contract with States to provide Medicaid services to conduct performance improvement projects, calculate performance measures, and comply with structural and operational standards. In order to ensure compliance with these requirements, we believe a review of all these activities is essential to determine the quality, timeliness, and access to services provided to Medicaid beneficiaries. However, § 438.364 requires that only the aggregated findings of the EQRO analysis of all information derived from the EQR activities be produced, and it is only this summary information that is to be made available to the public upon request.

Comment: One commenter believed that it was vital to include in EQR a range of activities beyond “focused studies” and medical record review. This commenter felt that the mandatory activities proposed would require the collection and use of data from multiple sources, and that we may want to consider mandating the validation of primary data sources such as encounter data and survey data. Another commenter asked that focused studies be a mandatory activity, and that MCOs be required to show measurable improvement in them. One commenter supported our establishing mandatory activities as well as the optional activities that are eligible for the 75 percent matching rate.

Response: We are aware of the importance of the integrity of the MCO's and PIHP's underlying information systems for the conduct of some EQR activities, and we address this issue in the protocols for review for compliance with structural and operational standards, performance measures, and encounter data. We do not include focused studies as one of the mandatory activities in this regulation because the Medicaid managed care final rule requires that MCOs and PIHPs conduct performance improvement projects. A performance improvement project begins with a focused study to select a clinical or nonclinical topic and measure performance in that area, but takes steps beyond a focused study to implement activities to improve performance. This regulation requires that the State include information regarding the validation of these studies as part of EQR.

Comment: Several commenters were concerned that this rule potentially would permit EQROs to analyze and evaluate data collected by a party not subject to the same conflict of interest requirements as the EQRO. These commenters were concerned that the EQRO would be held accountable for the validity, accuracy, and reliability of the MCOs' projects without necessarily having access to the raw data. One of the commenters suggested that there be continued discussions with the QIO community about the need for raw data files from MCOs in order to evaluate the performance improvement projects and performance measures. The commenter also felt that EQR performance measures should be standardized and consistent to allow comparisons among the States, and among the MCOs operating in more than one State. Another commenter recommended that the final rule require that EQR activities be carried out by the EQRO. If the information provided for the EQR is collected by the State or another agency, the commenter suggested that the EQRO be required to validate the data or information before analyzing it or forming conclusions about quality, timeliness, and access.

Response: In order to receive the enhanced 75 percent Federal match provided for in section 1903(a)(3)(C)(ii) of the Act, we believe most States will use an EQRO to conduct the mandatory EQR-related activities. However, in order to provide flexibility to States to coordinate their quality oversight activities, we permit States or their agents to perform the mandatory EQR activities, and only require that States use an EQRO for the conduct of EQR (as defined under § 438.320) and for the production of the EQR results as specified under § 438.364. If a State chooses not to have an EQRO conduct the mandatory activities, the State still needs to use, or have its contractor use, our protocols or protocols that are consistent with ours when conducting these activities. The State will also need to provide the EQRO with the information specified under § 438.364(a)(1)(i) through paragraph (iv) for each of the EQR-related activities as required in § 438.350(b). We believe this last requirement may not have been clear in our proposed rule, and we have therefore provided a cross-reference to § 438.364(a)(1)(i) through paragraph (iv) in § 438.350(d) in this final rule. This clarification addresses the comments above by identifying the types of information we expect to be provided to an EQRO if the State or a contractor other than the EQRO is conducting the EQR-related activity. We also provide clarifying language in a new § 438.358(a) of this final rule, which sets forth a general rule making clear that a State can conduct, or have another State contractor or the EQRO conduct, the mandatory and optional EQR-related activities that provide information for the EQR function.

We do not agree that the EQRO must revalidate activities already validated by the State or another State contractor that uses our protocols. We believe the use of the protocols will ensure that each of the activities, including an assessment of the underlying data systems, is conducted using reliable and valid methods.

We are not requiring standardized performance measures. In our Medicaid managed care final rule, we require States to require MCOs and PIHPs to use standard measures. The Medicaid managed care final rule also gives CMS the authority to prescribe standard measures in consultation with States and other stakeholders. Currently, States have the flexibility to determine which measures they will require of their MCOs and PIHPs. The CMS protocol for performance measures sets out a standard method to validate performance measures. We have also developed a protocol for calculating performance measures, as this is an optional EQR-related activity.

Comment: One commenter believed that allowing the use of information obtained by the State or its agent for EQR means the information is not truly independent. The commenter further contended that the methods used by the State or its agent do not have to be consistent with the EQR protocols, since the State or its agent is not an EQRO.

Response: Consistent with provisions at § 438.350(b) and (c), whoever conducts the mandatory or optional EQR-related activities must use the protocols or methods consistent with the protocols. We have made this clear in the final rule.

Comment: Several commenters noted that the activities under § 438.358 are currently in some cases conducted by the State, the county, or both. They added that having the EQRO perform this same activity, or even review these activities would be redundant and costly. One of these commenters suggested that we allow these activities to be done directly through the State or county survey process.

Response: EQR-related activities may be conducted by the State or by any State contractor other than the MCO or PIHP as long as the activities are conducted consistent with our protocols. However, if a State chooses to have its EQRO conduct these activities it can obtain the enhanced 75 percent Federal match under section 1903(a)(3)(C)(ii) of the Act.

Comment: One commenter asked that we clarify whether information derived from optional activities performed by other fiscal government agencies could be used by the EQRO.

Response: As long as the other agency uses our protocols or methods consistent with the protocols, the information derived from EQR-related activities performed by other State agencies can be used as part of EQR. The State, however, would not be able to receive the enhanced 75 percent Federal match unless the other government agency qualified as an EQRO, and the contract to conduct the activities was procured consistent with § 438.356(e). We clarify in this final rule that the information obtained from optional EQR-related activities must be from information derived from optional activities conducted within the preceding 12 months.

Comment: Several commenters believed that MCOs should be required to report on standardized performance measures for specific conditions. One of these commenters also recommended that MCOs be required to report on aggregate measures of changes in health status for all people who meet a definition of disability. The commenter further urged that the development of these measures be a priority for both quality assurance and reimbursement purposes.

Response: As stated previously, the Medicaid managed care final rule provides States with the authority to specify what performance measures to require their MCOs and PIHPs to calculate and report. We are allowing this flexibility because State Medicaid managed care programs differ in the services they contract for and the populations served by MCOs and PIHPs. We think it is important that States be able to make comparisons across their contracting MCOs and PIHPs and, where this information is available, we require that it be provided as part of the EQR results as specified in § 438.364(a)(4). However, while the Medicaid managed care final rule provides CMS with the ability to prescribe performance measures in consultation with States and other stakeholders, at this time we are not requiring the collection of comparative data nationwide.

We are also not requiring that States collect health status information from their MCOs and PIHPs. States are free to do this if they choose, and an increasing number of States are assessing the health status of MCO and PIHP enrollees for purposes of risk adjusting payments, or for quality activities. This rule also allows States to have their EQRO administer consumer surveys and obtain an enhanced Federal match of 75 percent. Approximately 30 States currently administer consumer surveys, primarily the CAHPS survey, which collects health status information from the perspective of consumers.

Comment: One commenter felt that the EQR-related activities were not clearly defined, and were limited in scope. The proposed language did not appear to the commenter to require the State to provide actual data to the EQRO, only information on the validation of the data. The commenter was concerned that the State could report to the EQRO that the data are valid, without actually providing the data itself.

Response: We do not agree with the commenter that the EQR-related activities are limited in scope. The activities reflect those that States have used existing EQR contractors to conduct in the past. These activities are more fully explained in the protocols that we reference in this final rule. On November 23, 2001, we published a notice in the Federal Register announcing the completion of these protocols noting their availability on our website and asking for comment on the extent to which they impose a burden, as well as any other issues the commenters wished to raise. Our protocols clearly define EQR activities, and the steps needed to conduct these activities in a valid and reliable manner. As noted in the preamble of our proposed rule, the full content of the protocols themselves was not included in the proposed rule, and is not included in this final rule because the protocols are more detailed than appropriate for Federal regulations, will need to be revised as the state-of-the-art of quality improvement changes, and States may use other protocols as long as they are consistent with those we developed. The need for the EQRO to have raw data will depend on the activities a State chooses to have its EQRO perform. For the actual conduct of EQR as defined in § 438.320, as well as the mandatory activities, access to raw data will not be needed. If the EQRO conducts all of the mandatory activities, it will be responsible for validating the methodological approach used by the MCO and PIHP for the conduct of performance improvement projects, and the calculation of performance measures. Regardless of who conducts the EQR-related activities, the CMS protocols, or a method consistent with the CMS protocols, must be used, and the information derived from the activity, as specified in § 438.364(a)(1)(i) through paragraph (iv), must then be provided to the EQRO.

Comment: One commenter did not support our decision to make performance improvement projects a mandatory activity, while focused studies are an optional activity. The commenter expressed concern that performance measures tend to focus on things that are easy to fix, and do not always provide a reliable picture of quality across a broad range of concerns.

Response: As the state-of-the-art of quality assessment and improvement has changed, we have found it more suitable to implement performance improvement projects than focused studies. Focused studies aim to assess the quality of care provided at a point in time, whereas performance improvement projects, in addition to assessing a focused area of care at a point in time, aim to initiate an intervention to improve care over time. In our proposed rule, we discussed the limitations of solely using focused studies, without information from other quality activities, to assess the care provided to all enrollees of a State Medicaid managed care program. It is for these reasons that improvement projects are mandatory while focused studies are optional. We note, however, that States may employ focused studies and use an EQRO to conduct this activity, thus accessing the enhanced 75 percent Federal match under section 1903(a)(3)(C)(ii) of the Act.

In this rule, we provide for a multipronged approach to quality improvement that uses information from three sources: (1) Determination of compliance with standards, (2) validation of performance improvement projects, and (3) validation of performance measures. We believe that this approach will provide for a reliable assessment of the quality, timeliness, and access to care provided to Medicaid beneficiaries by an MCO/PIHP.

Comment: One commenter interpreted the proposed rule to prohibit States and EQROs from conducting focused studies, and to instead require States to perform comprehensive reviews of all areas of the MCO contracts every year. This commenter recommended that we reconsider the scope of annual review, suggesting that a 1 year cycle does not allow sufficient time to procure an EQR contract, conduct and complete EQR activities, and report results on the EQR as specified in this rule. The commenter also recommended that we allow for a multiyear rotational approach to quality measurement and improvement (for example, rotate specified performance measures, focused clinical topic reviews). One commenter similarly believed that 1 year was too short a period of time in which to conduct the activities under § 438.358 (a)(1)(i) and (ii) of the proposed rule. This commenter suggested that this time period instead be left up to the State agency. Another commenter recommended that we require only that the information used by the EQRO for validation of performance improvement projects be from the most recent review performed within the previous 3 years, rather than requiring a yearly review.

Response: Section 1932(c)(2) of the Act requires an annual external review. In the final rule, we require that there be three sources of information used in this review. First, for performance improvement projects, this final rule requires that there be performance improvement projects underway during the previous 12 months. We understand that an MCO or PIHP may have multiple projects underway at a given time, and these projects may be at various stages of implementation. In response to this comment, we have revised the language under proposed § 438.358(a)(1)(i) (now § 438.358(b)(1)) to clarify that performance improvement projects need to be underway during the preceding 12 months, instead of having been completely performed during the preceding 12 months. Consistent with private sector practices, we therefore would allow States to use a multiyear rotational approach when conducting performance improvement projects and calculating performance measures. Second, for performance measures, the rule requires that one or more measures be reported annually. Finally, as was indicated in our proposed rule, EQR also needs to employ information from a review of structural and operational standards, conducted within the previous 3-year period.

Comment: Many commenters suggested that the list of mandatory activities include an examination of reasons for disenrollment and termination.

Response: Under § 438.358(b)(3) of this final rule, we require a review of MCO and PIHP compliance with State standards, in accordance with the Medicaid managed care final rule. This includes standards for enrollment and disenrollment. The Medicaid managed care final rule includes standards for disenrollments requested by the beneficiary, as well as those requested by the MCO or PIHP. In addition, the Medicaid managed care final rule requires MCO and PIHP compliance with State standards for health information systems. As part of the health system provisions, we require that the State ensure that the MCO or PIHP information system provides information including, but not limited to, utilization rates, grievances, and numbers of appeals and disenrollments. We believe these provisions adequately address the commenter's concern, and that no additional requirements are necessary.

Comment: One commenter noted that there was no cross-reference in the proposed EQR rule to the requirements in the then proposed Medicaid managed care rule that required MCOs to measure performance and conduct performance projects, and to comply with State-mandated standards. The commenter suggested that we make this cross-reference to the applicable sections in the Medicaid managed care rule.

Response: We have in this final rule added cross-references to the appropriate citations in the Medicaid managed care final rule.

Comment: One commenter recommended that we establish a core set of State standards for MCOs and evaluate these during the EQR process. The commenter was concerned that allowing States to determine the measures to be collected would provide little or no comparable plan or State level data.

Response: We do not agree that this rule should specify standardized performance measures for States or their contracting MCOs and PIHPs. The Medicaid managed care final rule specifies that States, through their contracts, must require their MCOs and PIHPs to calculate performance measures or submit data to the State that enables the State to measure MCO's or PIHP's performance. Many States currently require that standard performance measures be collected across MCOs. In addition, we believe that States will require that specified measures be calculated over time to enable the State to evaluate MCO and PIHP performance. In § 438.364(a)(4), we require that the EQR results include comparative information, as determined appropriate by the State. Furthermore, § 438.10(i)(2)(ii) of the Medicaid managed care final rule requires, for those States that provide for mandatory managed care under section 1932(a)(1)(A) of the Act, that the State provide comparative information annually. This must include, to the extent available, quality and performance indicators as required under § 438.10(i)(3)(iv). In addition, the Medicaid managed care final rule provides that CMS may, in collaboration with States and other stakeholders, prescribe standard performance measures.

Comment: One commenter asked us to clarify how proposed § 438.358(a)(1) fulfills the statutory requirement of EQR, and specifically how this information relates to a review of “the quality outcomes and timeliness of, and access to, the items and services for which the managed care organization is responsible under the contract.”

Response: In order to make an assessment about the quality, timeliness, and access to services provided by MCOs and PIHPs, there must be information from which an assessment can be made. Section 1932(c)(A)(iii) of the Act required us, in coordination with the NGA, to contract with an independent quality review organization to develop protocols to be used in EQR. In order to develop protocols, we first needed to define EQR, as it was not defined under section 1902(a)(30)(C) of the Act. We also needed to determine what activities we consider necessary or appropriate to provide information for a quality review. The EQR activities in § 438.358(b) and (c) are activities that (1) the expert panel convened under the auspices of the NASHP recommended be included as part of EQR; (2) a survey of States by the Department of Health and Human Services' Office of Inspector General identified as quality review activities used by States; and (3) a survey of States by NASHP confirmed as activities most frequently used by States for EQR. The EQRO must develop a report, based on the information provided, as specified in § 438.364, that includes a detailed assessment of each MCO's and PIHP's strength and weaknesses with respect to the quality, timeliness, and access to health care services furnished to Medicaid beneficiaries.

Comment: A commenter noted that the rule does not clearly identify which entities are qualified and competent to undertake the validation of performance measures and performance improvement projects. In the commenter's view, as drafted, the rule could be interpreted as allowing entities other than EQROs, including the State or the MCO itself, to undertake these tasks. The commenter recommended that we clarify what types of entities can engage in validation activities and at a minimum require those entities to be competent and independent.

Response: The State, an EQRO, or other State contractor can undertake any of the EQR-related activities. However, it is only when an EQRO, that meets the competency and independence criteria, conducts any of these activities that a State can obtain the enhanced 75 percent Federal match under section 1903(a)(3)(C)(ii) of the Act. Regardless of who conducts the activity, the CMS protocols (or other protocols consistent with ours) must be used to gather information for the mandatory and optional activities used in EQR. We did not intend to allow the MCO or PIHP itself to be able to conduct any EQR-related activities and in response to this comment we have revised § 438.358 so that it is clear that “the agent” must be an entity other than an MCO or PIHP.

Comment: One commenter recommended that we modify the regulation to grant State agencies the discretion to adapt these requirements to more appropriately address the circumstances of small or new MCOs and PHPs. The commenter suggested that enrollment in some MCOs and PHPs may be too small for an EQRO to validate the data for performance improvement projects or performance measures. Similarly, for an MCO that is not yet operational or which has only been operating for a short amount of time, there may be insufficient experience to use to evaluate for compliance with standards.

Response: We do not agree that we should modify the regulation to allow States to adapt the requirements to address small or new MCOs and PIHPs. If enrollment in an MCO or PIHP is small, the entire applicable population, as opposed to a sample, can be used when conducting performance improvement projects, calculating performance measures, or validating these activities. Regarding compliance with State standards, all MCOs and PIHPs that contract with a State to provide Medicaid services must be in compliance with the contracting requirements in the Medicaid managed care final rule. Regardless of when the EQR is conducted, MCOs and PIHPs should have procedures in place to be compliant with these provisions. Therefore, an assessment of compliance with these standards must be conducted and the findings provided to the EQRO to make its assessment regarding quality, timeliness, and access to services provided by the MCO or PIHP to Medicaid beneficiaries.

Comment: One commenter felt that State Medicaid agency staff should conduct the review of MCO compliance with structural and operational standards, as the review requires extensive knowledge of the State Medicaid program, its regulations, and the MCO contract. This commenter believed that this requirement was duplicative of current practice and unnecessarily burdensome, and did not provide States needed flexibility to choose which activities it wants to have its EQRO conduct. The commenter suggested deleting this provision. Another commenter urged that the review of compliance with standards be an optional instead of mandatory activity. The commenter noted that States conduct this activity through various means, and that mandating this be done through EQR would mean an increase in Federal and State funding for the EQR contract. One commenter believed that the proposed requirement for review of structural and operational standards went beyond the statute's reference to “quality outcomes, and timeliness of, and access to items and services for which the organization is responsible under contract.” This commenter recommended that we reevaluate the extent of this review to ensure that it is consistent with the intent of the statute. The commenter further noted that this review was so broad that it would encompass most of the areas currently reviewed by States under their general contract responsibilities.

Response: States are not required to contract with an EQRO to conduct a review of the MCO's or PIHP's compliance with State structural and operational standards. A State can conduct this activity using the CMS protocols or protocols consistent with ours and provide the results of the review to the EQRO. The regular 50 percent administrative FFP match would be available to the State for this activity if it is not conducted by the EQRO. The EQRO will use this information in conjunction with information derived from the other two mandatory activities and any optional EQR-related activities conducted to determine quality of, timeliness of, and access to the quality of care provided by the MCO or PIHP. This final rule provides States with the flexibility to determine which activities it wants to have its EQRO conduct. Although we prescribe mandatory activities, which are consistent with the requirements set forth in the Medicaid managed care final rule, the State does not have to have its EQRO conduct these activities. A State is only required to have an EQRO conduct the analysis and evaluation of the information derived from the activities to determine if an MCO or PIHP is providing access to quality services. We do not believe that the scope of the mandatory activities goes beyond the statutory provisions under section 1932(c) of the Act which require States to have a quality assessment and improvement strategy which includes access standards, and measures to assess care, including grievance procedures and marketing and information standards. Furthermore, the statute requires that States implement monitoring strategies that address the quality and appropriateness of care. We, therefore, retain the review of MCO and PIHP compliance with State standards as a mandatory activity in our final rule.

Comment: One commenter believed that the intent and usefulness of the proposed language in § 438.358 requiring the EQR to “use information” obtained from the mandatory and optional EQR-related activities was unclear. The commenter recommended changing the language to read “The State or the EQRO shall/must conduct” the EQR-related activities.

Response: Sections 1932(c)(2)(A)(ii) and (iii) of the Act required us to (1) in consultation with States, develop a method to identify qualified entities for the conduct of EQR, and (2) in coordination with the NGA, develop protocols to be used in EQR. In order for us to determine who was qualified to conduct EQR and for us to develop protocols to be used in an EQR we first needed to define EQR. Based on the advice of an expert panel convened under the auspices of the NASHP, the proposed rule, and this final rule, define EQR as the analysis and evaluation by an EQRO of aggregated information. Based on this definition, the expert panel confirmed the types of activities that would produce information as it relates to the quality, timeliness of, and access to care provided to our beneficiaries. These are the mandatory and optional activities found in this section of our rule. To provide consistency with the definition of EQR, and because we do not require that States contract with an EQRO to conduct these activities, we retain the language that an EQR must use information derived from the EQR-related activities in the final rule.

Comment: Many commenters did not agree with our proposal to require that information be used from a review of structural standards every 3 years, and cited the statutory language requiring “an annual * * *” review. Many commenters recommended that all activities be done annually, citing reasons such as the changing status of provider networks, and pressures to control utilization. One commenter claimed that we did not adequately explain our rationale for permitting the use of data and information that may be up to 3 years old. The commenter argued that given the volatility of both the managed care market place and State Medicaid programs, the problems identified in Medicaid managed care systems throughout the country, and the fact that the majority of beneficiaries are children, allowing the use of 3-year-old data was inadequate. The commenter suggested that an evaluation of quality, timeliness, and access to services must be timely to allow for effective interventions to correct the problems.

Response: Reviews of MCO and PIHP compliance with structural and operational standards are very time consuming and costly. To be consistent with private industry standards, we proposed that information from the review of MCO and PIHP compliance with standards be from the most recent review conducted within the previous 3 years. Both NCQA and JCAHO perform their accreditation reviews once every 3 years. As stated earlier, our rule takes a multipronged approach to quality assessment and improvement. This is one reason why we require the EQR to use information from a minimum of the three mandatory activities to render a decision regarding the quality and timeliness of and beneficiary access to health care services. We believe that this comprehensive approach addresses the commenters' concerns, and that annual reviews for compliance with structural standards is not justified.

H. Nonduplication of Mandatory Activities (§ 438.360)

Proposed § 438.360 provided State agencies, under certain circumstances, the option not to require a review of MCO or PHP compliance with certain structural and operational standards specified in proposed § 438.358(a)(2) if the MCO or PHP is a certified M+C organization with a current Medicare contract, and has been evaluated and approved by us, our contractor, or certain approved accrediting organizations as a part of accreditation for compliance with these standards. The December 1, 1999 proposed rule also provided that a State agency under certain circumstances may similarly avoid duplicate reviews of all mandatory activities (listed in paragraphs (a)(1) and (a)(2) of proposed § 438.358) for any MCO or PHP that serves only individuals who are eligible for both Medicare and Medicaid. Under the December 1, 1999 proposed rule, if the State agency exercises this option, each MCO and PHP must make available to the State agency all reports, findings, and other results of the Medicare quality review or the accreditation survey that is to substitute for the Medicaid review.

Comment: Several commenters supported provisions designed to avoid duplication in the EQR process.

Response: We retain the nonduplication provisions in the final rule while providing clarifying language on their applicability, as discussed in responses to comments below, in order to better explain our intent.

Comment: Several commenters asked that the provisions in this section not be restricted to Medicaid MCOs that have M+C contracts. The commenters believe that the BBA does not restrict the nonduplication provision to these organizations.

Response: We agree with the commenters that the BBA does not require that an M+C contract be in place in order for the nonduplication provisions to apply. In response to these comments, we have changed the final rule to allow States, under certain circumstances, to elect not to review structural and operational standards of an MCO or PIHP that has been accredited by a national accrediting organization approved by CMS under the procedures in 42 CFR 422.158 as applying standards at least as stringent as Medicare, where the standards are comparable to those imposed by the State under § 438.204(g). The EQRO must review the reports, findings, and other results of the accreditation review to use in the EQR.

Comment: Several commenters recommended that we amend our regulations to permit accreditation programs that address only a portion of the § 438.358(a)(2) requirements. One commenter wanted us to retain the provision that allows an EQRO to use a review conducted by a private accrediting organization, or as part of an external review conducted under the Medicare program. Another commenter suggested that we revise § 438.360(b) to allow a State to exempt an MCO from a review of the mandatory activities, as opposed to exempting the MCO from the mandatory activities.

Response: We agree with the commenters that a State should be permitted to use only certain portions of a Medicare or accreditation review in place of a portion of a Medicaid review. As stated above, the final rule provides States with the option of using a Medicare or (if approved by CMS under § 422.158) private accreditation review to serve as the Medicaid compliance review of any or all of the standards required to meet provisions under § 438.204(g) as long as the MCO or PIHP meets the requirements of § 438.360(b) or (c). Because we received numerous comments on the applicability of this provision, we have revised the language in this section to more clearly explain our intent to apply it to MCOs and PIHPs that have been reviewed by an accrediting organization approved under § 422.158. We also clarified the regulations text to better identify the activities and standards to which this section applies, and what information needs to be provided to States and us to comply with this provision.

Comment: One commenter did not agree with provisions in § 438.360(b)(3) or (c)(3) requiring that a State receive a copy of all findings pertaining to the most recent accreditation review. The commenter contended that standard-specific information is adequate and that all review materials such as noted deficiencies, corrective action plans, and summaries of unmet accreditation requirements are excessive and unnecessary. The commenter suggested that we require MCOs to provide the State with applicable reports, findings, and results. Many commenters agreed that we should require that States receive and review information from the Medicare review or accreditation review.

Response: We agree that requiring all reports, findings, and other results of the Medicare review or accreditation review could be excessive. We have revised the language § 438.360(b)(3) and (c)(3) to reflect that the reports, findings, and results provided can be limited to those applicable to the standards for which the Medicare or accreditation review or quality activities will substitute for the Medicaid review activities.

Comment: One commenter asked us to clarify whether the nonduplication provision exempts the MCO from a review for compliance with standards, such as enrollee rights, maintaining a grievance system, or using practice guidelines. One commenter recommended that we allow deeming of credentialing and recredentialing requirements if the MCO is NCQA certified.

Response: We provide that the State may permit the findings from other allowable reviews to substitute for a duplicate review of the structure and operations of the MCO or PIHP. Under this provision, an MCO or PIHP is not exempted from a review of standards under § 438.204(g). Rather, States are permitted the option of using Medicare reviews or accreditation findings, including a review of credentialing and recredentialing procedures, instead of conducting a separate (and potentially duplicative) review, as long as the provisions under § 438.360 are met. This would apply to information on compliance with standards such as the requirements set forth in proposed § 438.358(a)(2)(i) through (a)(2)(xiii) cited by the commenter.

Comment: Many commenters agreed that external reviews need to validate performance measures specific to the Medicaid population in the case of Medicaid contracts. In contrast, one commenter recommended that an MCO fully accredited by a private accrediting organization should also be exempt from calculating performance measures (for example, HEDIS). The commenter believed that this would eliminate the need for new-capacity building or criteria to ensure consistency.

Response: We do not agree that an accredited MCO or PIHP should be exempt from a validation of performance measures calculated under § 438.358(a)(1) unless it provides services to dual eligibles only. As stated in our December 1, 1999 proposed rule, we believe the types of data collected, measures calculated, and studies conducted, on the Medicare population would differ from those for the Medicaid population unless the MCO or PIHP served only dually eligible Medicare and Medicaid beneficiaries. We believe this argument is also valid when applied to the commercial population. We, therefore, retain the language as written in the December 1, 1999 proposed rule. We note that if the accrediting organization, acting as the EQRO of the State, validates the performance measures required of the MCO or PIHP by the State, the State can obtain the 75 percent match under section 1903(a)(3)(C)(ii) of the Act for having the accrediting organization conduct that activity.

Comment: One commenter recommended that we revise the regulation to give State agencies discretion to determine what EQR activities are duplicative.

Response: We do not agree that States should have discretion to determine what EQR activities are duplicative. Except in the case of an MCO or PIHP that provides services to dual eligibles only, we limit the nonduplication provisions to the structure and operational standards reviewed under § 438.358(b)(3).

Comment: Several commenters noted that accrediting organizations differ in how they characterize the status conferred when MCOs meet their accreditation standards. For example, these commenters pointed out that not all accrediting organizations use the term “full accreditation.” One commenter recommended that we clarify proposed § 438.360(b)(2)(ii) to avoid confusion regarding what accreditation level must be attained to meet the requirements of the paragraph. Another commenter asked us to clarify “fully accredited” and recommended that we negotiate with accreditors seeking to be recognized under this section to determine what type of accreditation would meet the intent of this section.

Response: We understand that accrediting organizations use different terms to describe the extent to which MCOs or PIHPs meet their standards. However, in this provision of the regulation, we are not requiring that the MCO or PIHP achieve a certain level of accreditation. Rather, we are allowing States to use information gathered in the private accreditation process that is shared with the State to assess compliance. To make this more clear, in response to this comment, we have removed the term “fully accredited” from the regulations text. We also have revised the language of this section in order to make our intent more clear. We now specify that accrediting organizations that have been approved by us for M+C deeming under § 422.158 meet the requirements of this provision.

Comment: Several commenters did not agree with permitting States to avoid mandatory activities by relying upon information gathered from a Medicare or private accreditation review in order to assess MCO compliance with structural and operational standards. Some of these commenters specifically strongly opposed the exemption from mandatory activities when an MCO has a Medicare contract. They believed that activities such as review for the availability of services, establishment of provider networks, enrollee information, confidentially, and use of practice guidelines all have Medicaid and pediatric components that would not be examined under a Medicare review. If an exemption is allowed, the commenters suggested that additional activities be required to ensure compliance in problem-prone or sensitive areas that reviews by Medicare or private accrediting organizations may not adequately address. One of the commenters recommended that if an MCO is being considered for the exemption, that there must be substantial overlap between the Medicare and Medicaid products in (1) geographic service area, (2) network composition and management, (3) quality management structures and processes, and (4) levels of accreditation. Many commenters suggested that unless our quality review or accreditation has established the quality of the Medicaid provider network and administrative structures, these activities should not be exempted under nonduplication.

Response: The Congress clearly intended that we provide States the option to avoid duplicating review activities conducted for Medicare or by accrediting organizations. We limit the applicability of this provision to the mandatory activity designed to help States assess structural and operational standards for all MCOs and PIHPs other than those serving only dual eligibles. For the latter, under § 438.360, we also permit States to use this option with respect to the validation of performance measures or the validation of performance improvement projects. We believe proposed § 438.360 generally places sufficient parameters on States that choose to exercise this option.

We retain the provision that permits States to use this option to assess compliance with standards. We note that § 438.207 of the Medicaid managed care final rule requires that MCOs and PIHPs submit documentation to the State of compliance with requirements in the Medicaid managed care final rule that requires MCOs and PIHPs to maintain a network of providers that is sufficient in number, mix, and geographic distribution to meet the needs of the enrollees in the MCO or PIHP. In addition, § 438.207 requires that any time there has been a significant change in MCO or PIHP operations that would affect adequate capacity, additional documentation must be submitted. We believe this information adequately complements any review of availability of services that would be conducted by Medicare or an accrediting organization that provides information for the EQR.

We are concerned, however, that the wording of proposed § 438.360 has caused some confusion about the intent of this provision. Specifically, our words “A State may exempt an MCO from mandatory activities * * *” may be interpreted by some as exempting an MCO or PIHP from oversight, rather than an exemption from State Medicaid reviews that duplicate Medicare and private accreditation reviews. To clarify this, we have removed the word “exempt” from this provision in the final rule (noting also that the Congress did not use this word in the corresponding statutory provision) and replaced it with language reflecting the fact that these provisions do not exempt MCOs from review for compliance with structural and operational standards, but instead permit States to use information generated through Medicare or private accreditation review to assess compliance with these standards, in lieu of engaging in their own otherwise “mandatory” review activity.

In addition, in response to the commenters' concerns about permitting States to substitute Medicare or private accreditation review for direct State review, we are adding a new paragraph (4) to § 438.360(b) and (c) requiring that States identify in their qualities strategies those standards and activities they plan to monitor through the use of Medicare or private accreditation review data, and explain why direct State review would “be duplicative.” This will help ensure that this approach is only taken when State review would truly be needlessly duplicative of review already performed.

Comment: One commenter was concerned that proposed § 438.360 appeared to allow the nonduplication exemptions to last indefinitely, and believed that it was not unusual for plan performance to vary significantly from year to year due to organizational changes. The commenter recommended that States be required to develop mechanisms to periodically re-evaluate MCO compliance with standards during the course of a 3-year period, and to re-institute a direct Medicaid agency review if accreditation, Medicare, or State oversight indicate potential quality problems.

One of the commenters cited recent OIG studies that identified significant issues with accrediting bodies, and did not think that States should relinquish their direct MCO oversight responsibilities to the accreditation industry.

Response: Neither the statutory nor conference committee language discussed any time limit on a State using Medicare or accreditation review data in its assessment of an MCO or PIHP in lieu of a direct Medicaid review. We believe it appropriate to allow States to make the determination as to whether this remains appropriate. We note that the new paragraph (4) that we have added to § 438.360(b) and (c) requires that States explain and justify their use of this approach, and believe that it is appropriate to permit the approach to be used for so long as this justification remains valid. Therefore, we do not specify a time limit in the final rule.

With respect to the commenter's recommendation for periodic re-evaluation every 3 years, § 438.360 requires that information obtained from a Medicare review or a review by an accrediting organization be provided to the State, which must then provide the information to the EQRO for use in the EQR. Because this information must be obtained from a review of compliance with standards conducted within the past 3 years, this requirement should address the changes in plan performance that the commenter is concerned about. Moreover, the Medicaid managed care final rule requires that States have a quality strategy that has procedures for assessing the quality and appropriateness of care provided to Medicaid beneficiaries, and that States must regularly monitor and evaluate MCO and PIHP compliance with operational standards.

As noted in earlier responses, we believe the Congress clearly intended States to have the option of avoiding duplicate reviews of MCOs that have been accredited by a national accrediting organization, and we accordingly allow for this in the final rule.

Comment: One commenter recommended that we clarify that States may only eliminate elements of the EQR process, whether mandatory or optional, if components of the M+C evaluation process or private accreditation review are the same as or similar to those of the Medicaid review process. Several commenters felt that this provision should address two concepts: whether the standard or requirement is duplicative, and whether the methodology of the review is duplicative. One commenter asked that we clarify what we mean when we say, under § 438.360(b)(2), that the “* * * methodologies must be * * * established by the State, not CMS.” The commenter noted that it is the State, not CMS, that establishes the standards for the mandatory activity under § 438.358(a)(2) and therefore it is not clear what benchmark we intend to use to determine comparability.

Response: This section of the regulation applies only to mandatory activities as specified in § 438.358(b). Because the optional activities are not required, we do not address optional activities in the nonduplication provisions. As stated earlier, we have clarified the regulations text to better explain that Medicare or accreditation standards must be comparable to those established by the State. We have removed the reference to standards and review procedures needing to be as stringent as those established by CMS because we agree with the commenter, that it is the State, not CMS, that will establish standards to comply with § 438.204 of the Medicaid managed care final rule. As for review methodology, the statute required that we establish protocols to be used in EQR. The protocols we developed include generic activities and steps to be followed to ensure that the EQR activities are conducted in a reliable and valid manner.

Comment: One commenter asked that, because implementation of proposed § 438.360(b)(2)(ii) would depend upon our approval and recognition of private accrediting organizations under § 422.158 as having standards and review procedures as stringent as those established by Medicare, we move forward to make these later determinations so this provision can be implemented in a timely fashion when these regulations become final.

Response: We have already received and approved applications for M+C deeming from several accrediting organizations: (1) NCQA, (2) JCAHO, and (3) the Accreditation Association for Ambulatory Health Care (AAAHC).

Comment: One commenter was confused about the distinction between proposed § 438.360 and proposed § 438.362, and felt they were redundant. The commenter also objected to our provisions applying to dual eligibles, specifically the State's option of permitting information obtained from performance improvement projects and performance measures specific to dual eligibles to substitute for Medicaid specific information.

Response: We do not agree that § 438.360 and § 438.362, which permit States to exempt an MCO or PIHP from EQR in its entirety, are redundant. However, we agree that proposed § 438.360 was potentially confusing in its use of the word “exempt.” We have revised the language in § 438.360 to clarify that § 438.360 allows States to use the findings of Medicare or accreditation reviews in place of a Medicaid review in order to avoid duplication, but does not exempt MCOs or PIHPs from EQR, as does § 438.362 where it applies. We think that there is a clear distinction between § 438.360 under which analysis and evaluation of information must still be conducted, and § 438.362 under which the MCO or PIHP is exempted from the EQR function. We disagree with the commenter concerning the appropriateness of the dual eligible provision. In the case of dual eligibles, Medicare review necessarily is targeted to the population involved. We therefore believe that Medicaid review could be particularly duplicative in this case.

Comment: One commenter requested that if accreditation is to be used as the basis for exemption, regulations require that the MCO be specifically accredited with respect its Medicaid line of business, and that information from this Medicaid enrollee review be provided to the State.

Response: We do not agree that we should limit the applicability of the nonduplication provisions in § 438.360 to MCOs or PIHPs accredited specifically for their Medicaid product. Most accrediting organizations do not conduct separate reviews for an MCO's or PIHP's Medicaid product. With respect to the commenter's second point, we do require that the findings of the accreditation be provided to the State and then, in turn, to the EQRO to be used as part of the EQR.

Comment: One commenter urged that we allow for the use of review findings of related “focus studies” of groups that Medicaid serves (for example, the elderly or disabled) which are conducted by other types of certified Medicare organizations.

Response: As long as a focused study is conducted using a methodology consistent with our protocols, and the study population is composed of Medicaid beneficiaries, a State can have its EQRO use the review findings. In addition, if the organization that conducts the focused study is the State's EQRO, the State can obtain the 75 percent enhanced match for its review of these findings.

Comment: One commenter believed that the activities under proposed § 438.358(a)(2) are not the same regardless of the populations served, and specifically that there is a difference when serving individuals with disabilities. To address this concern, the commenter felt that the EQRO must be knowledgeable and sensitive to people with disabilities in order to effectively assess an MCO's compliance with standards.

Response: As specified in § 438.354, an EQRO must meet certain competency requirements, including having staff with knowledge of Medicaid beneficiaries. In addition, our Medicaid managed care final rule requires, under the State's quality strategy, that the State have procedures in place for assessing the quality and appropriateness of care and services furnished to enrollees with special health care needs. This includes individuals with disabilities.

Comment: One commenter recommended that audits conducted by the State licensing organization be coordinated with the EQRO, and that the audit of components conducted by the State licensing organization be “deemed” to have been performed by the contracted EQRO.

Response: States can use their State licensing organization to assess MCO or PIHP compliance with State standards, or perform any of the mandatory or optional EQR-related activities identified in § 438.358. If a State wants to use this information for the EQR, the review must, at a minimum, use our protocols or protocols that are consistent with ours. Thus, there would be no reason to “deem” these reviews to have been performed by the EQRO, other than to claim the 75 percent match that would apply if the EQRO performed these functions. As noted above, however, if a State uses entities other than EQROs to perform activities, the 75 percent match rate under section 1903(a)(3)(C)(ii) of the Act would not be available. We hope and anticipate that States will coordinate the EQR and EQR-related activities with other State quality activities currently in place.

Comment: Many commenters believed that direct Medicaid agency external reviews should always be performed with respect to grievance systems because these commenters believe that the Medicaid fair hearings process is unique.

Response: The EQRO is not responsible for reviewing the State's fair hearing process. It must review information about the MCO or PIHP internal grievance system. In order for a State to use a Medicare or accreditation compliance determination to substitute for a Medicaid review of the MCO's or PIHP's grievance system, the State will need to address in its quality strategy the basis for considering the Medicare or accrediting organization's standard comparable to the State's grievance processes standard that needs to comply with the provisions of subpart F of the Medicaid managed care final rule.

Comment: One commenter expressed concern that we excluded Medicare beneficiaries who are eligible for Medicaid as a result of spenddown requirements from the definition of dually eligible persons.

Response: We have not excluded from the definition of dually eligible those Medicare beneficiaries who are eligible for Medicaid as a result of spenddown requirements. We consider any person who is receiving both Medicare and Medicaid benefits as a “dually eligible” person.

Comment: One commenter believed that the meaning of MCO in proposed § 438.360, and § 438.362 was not clear. The commenter noted that corporate entities may be wholly owned subsidiaries of other corporate entities, and may hold multiple licenses. The commenter also noted that in some cases a plan may have a large Medicaid product and a very small Medicare product, calling into question the assumption that adequate management of the Medicare enrollees is an appropriate proxy for their Medicaid enrollees. The commenter recommended a more complete definition of MCO, as it relates to the MCO's Medicare and Medicaid product lines being incorporated into the rule.

Response: The definition of MCO as used in this regulation is defined in § 438.2 of the Medicaid managed care final rule. According to this definition, an MCO is the entity that holds the Medicaid comprehensive risk contract. We believe that this definition addresses the commenter's concern, as the Medicare review provisions will only apply if the same entity that holds the Medicaid contract holds the Medicare contract.

Comment: One commenter recommended that we make clear that a State may undertake optional EQR activities, even if it has exempted an MCO from a portion of or all of the mandatory activities.

Response: A State may conduct the optional EQR activities when it uses Medicare or accreditation review findings for the mandatory activities. As long as the State uses the protocols developed by us or protocols consistent with ours, the information derived from the optional activities can be used in the EQR.

Comment: One commenter believed that when an MCO is accredited by a private accrediting body, the States should be strongly encouraged not to duplicate the review performed by the private accrediting body.

Response: The final rule provides States the option to use the findings of an accrediting body instead of conducting its own review of MCO or PIHP compliance with certain standards, if the MCO or PIHP has been accredited by a national accrediting organization recognized by us. We believe that States should have the discretion to make this decision, and individuals who believe that this option should be adopted should encourage States to do so.

I. Exemption From External Quality Review (§ 438.362)

Proposed § 438.362 provided an option for a State agency to exempt an MCO or PHP from the EQR requirements in section 1932(c)(2)(A) of the Act if: (1) The MCO or PHP has a current Medicare contract under part C of title XVIII or under section 1876 of the Act; and (2) for at least 2 years, the MCO or PHP has satisfied EQR requirements under section 1932(c)(2)(A) of the Act with respect to its Medicaid contract. In addition, we proposed that the Medicaid and Medicare contracts be required to cover all or part of the same geographic area. We also proposed that the State agency require each exempted MCO and PHP to annually provide the State with copies of all Medicare reviews performed by us, by our agent or any private accrediting organization, with respect to the quality, timeliness, and access to its services.

Comment: Many commenters opposed this exemption of certain MCOs from EQR. One of the commenters felt that this provision completely abrogates the responsibility of the States and CMS to monitor the quality of Medicaid managed care systems for children. One commenter agreed with this provision, as long as it was an option for States.

Response: In the BBA, the Congress expressly provided States with the option of exempting from EQR those MCOs that provide Medicare services and also have had experience serving the Medicaid population. This provision, however, does not exempt States from monitoring MCOs and PIHPs for compliance with the mandatory activities listed in § 438.358. These activities, required of MCOs and PIHPs under our Medicaid managed care final rule, are essential to ensure the quality of services provided to Medicaid beneficiaries by MCOs and PIHPs. For example, the BBA requires that States have a quality strategy in place when contracting with MCOs and PIHPs. States will still need to ensure MCO and PIHP compliance with the BBA provisions and our regional offices will continue to monitor States for compliance regardless of whether or not an EQR is conducted.

Comment: One commenter asked how this provision would impact a Medicaid plan that gave up its M+C product. Specifically, the commenter asked if there would be an immediate requirement for an EQRO review.

Response: Under § 438.362(a)(1), the MCO and PIHP must have a current Medicare contract. Therefore, as EQR is an annual requirement, the year following the termination of the M+C plan, the State is required to contract with an EQRO to, at a minimum, review and analyze information from the validation of performance improvement projects conducted by the MCO or PIHP and performance measures calculated by the MCO or PIHP that year. The State will also need to ensure MCO or PIHP compliance with structural and operational standards. If the MCO or PIHP had been reviewed by Medicare or an accrediting organization within the previous 3 years, that information could be used in the EQR. If this were the year that the MCO or PIHP was to be reviewed for structural and operation standards, the State or its contractor, or the EQRO would have to conduct a review.

Comment: Several commenters asked us to clarify who we considered appropriate to determine whether an MCO or PIHP performed acceptably in previously conducted EQRs, as this was not a requirement under the section 1902(a)(30)(C) of the Act EQR requirements. Some of the commenters stated that it would not be appropriate for the State to make the determination, as the independent nature of the EQR might be compromised. Many commenters asked us to clarify what we consider to be acceptable performance and recommended that an MCO or PHP be required to perform acceptably on quality, timeliness, and access in order for a State to allow for the exemption.

Response: Whether an MCO or PIHP has performed acceptably is determined by the State based on the results of the EQR, which must include a detailed assessment of each MCO's and PIHP's strengths and weaknesses with respect to quality, timeliness, and access to health care services provided to Medicaid beneficiaries. If a State elects to exempt an MCO or PIHP from an EQR it must, as specified in § 438.362(a)(3), ensure that an MCO or PIHP not only have had a Medicaid contract for 2 years but that the MCO or PIHP has also been subject to an EQR as specified in this rule. This effectively means that no MCO or PIHP could be exempted under § 438.362 until EQR under this final rule is in effect for at least 2 years. As long as the provisions under this section are met, the State will determine the length of time for which it will exempt an MCO or PIHP from EQR. The State will be able to use information obtained from the Medicare or accreditation reviews, as the submission of Medicare review findings is required under § 438.362(b).

Comment: One commenter was concerned that similar geographic coverage areas do not necessarily ensure similar administration, networks, benefits, and quality improvement projects for the different beneficiaries who are served by the Medicare and Medicaid programs. Another commenter agreed with the requirement that the two contracts cover the same geographic area, but was concerned that practice patterns tend to vary geographically for given clinical topics and specific types of treatment. The commenter suggested we change the geographic requirement to require similar or identical service areas instead of overlapping areas. Two commenters supported the requirement that the two contracts cover all or part of the same geographic area, but suggested that we include additional requirements that the two contracts must (1) include the same provider networks and (2) offer the same or similar benefit and services to consumers. The commenters believe this is important because M+C plans serve markedly different populations, provide different benefit packages, and often offer different provider networks than Medicaid plans. One commenter asked us to clarify whether the Medicaid and Medicare services areas have to be identical for MCOs and PHPs to qualify for exemption.

Response: Under § 438.362(a)(2), we require that the Medicare and Medicaid contracts cover all or part of the same geographic area in order for a State to exempt the MCO or PIHP from EQR. We required an overlap of service areas in this provision because we believe this will increase the likelihood that the findings from the Medicare review will serve as a proxy indicator of the care delivered to the MCO's or PIHP's Medicaid beneficiaries. We have made some clarifying language changes to the regulations text in the final rule to more clearly state our intent that the contracts must cover all or part of the same geographic area within the State that is allowing the MCO or PIHP exemption from EQR. However, we think that requiring identical service areas or the same or similar benefit packages is too restrictive, and may effectively exclude the use of an exemption intended by the Congress.

Comment: Several commenters asked that we not restrict the exemption provision to M+C organizations, but also allow it to apply to MCOs and PHPs that have undergone or achieved “excellent” status by a private accreditation review.

Response: In the BBA, the Congress applied the total exemption in section 1932(c)(2)(C) of the Act only to M+C organizations. Consequently, we have not applied this provision to commercial MCOs and PIHPs. However, we address nonduplication provisions related to EQR activities as they apply to private accreditation under § 438.360.

Comment: Several commenters concurred with the requirement that an MCO or PHP must demonstrate acceptable performance determined by the EQR for the 2-year period before exemption. One of these commenters, however, was concerned that the regulation appears to allow exempt status to last indefinitely, and noted that it is not unusual for plan performance to vary significantly from year to year due to organizational changes. Several commenters recommended that States be required to develop mechanisms to periodically re-evaluate an MCO's exempt status, and to re-institute EQR if accreditation, Medicare, or State oversight indicate potential quality problems. One commenter opposed our proposal to require that the MCO have complied with EQR requirements for 2 prior years. This commenter believed that this interpretation was unduly restrictive, and inappropriately limited the discretion given to State agencies to exempt MCOs based on the State agencies' experience with the MCOs or PHPs.

Response: We believe that the language in this rule properly reflects congressional intent to allow States the option to exempt a Medicare MCO from EQR. Once an entity is exempted, and continues to meet the criteria for exemption, we believe that the Congress intended that the Medicare quality review requirements serve as a proxy for the Medicaid EQR requirements. Because the State will have access under § 438.362(b) to data from these reviews, any problems that develop should be recognized through this process. We thus do not believe it would be appropriate to require States affirmatively to re-evaluate an MCO's or PIHP's EQR-exempt status.

With respect to our requirement that 2 years of success in Medicaid EQR be required, as noted in the preamble to the proposed rule, we considered several interpretations of the statutory provision that requires at least 2 years of Medicaid contracting in order for this exemption to apply. We concluded that the Congress' intent in requiring 2 years of Medicaid contracting experience was to ensure that the MCO had sufficient quality measures in place to meet Medicaid EQR standards before it could be exempted from Medicaid review. Since these EQR standards are new, this necessarily would require that an MCO have a Medicaid contract for 2 years under these EQR requirements before the exemption in § 438.362 would apply. This ensures that all MCOs and PIHPs have been subject to Medicaid EQR at some point, and have been found to be compliant with Medicaid standards in this review.

We emphasize again, however, that the EQR requirements, from which MCOs and PIHPs can be exempted under § 438.362 are only one part of the Quality Strategy provided for in the BBA. Other BBA provisions require States contracting with MCOs to ensure the quality and appropriateness of care and services furnished to Medicaid beneficiaries. We believe that if States find MCOs or PIHPs not to be providing appropriate quality care, they would exercise their option to require an EQR.

Comment: Many commenters agreed that MCOs should be required to submit copies of reviews performed by Medicare or an accrediting organization. One commenter did see the benefit in receiving Medicare review reports. One of the commenters cautioned that accreditation reviews are generally performed less frequently than annually.

Response: We only require that information from the Medicare or accreditation review be provided annually. We are not requiring that Medicare or accreditation reviews be conducted annually. If no new information is obtained in a specific year, it is not necessary for the MCO or PIHP to provide the State information provided the previous year. If a State chooses to exempt the MCO or PIHP, this does not relieve the State from ensuring that access to timely and quality services is being provided. Findings from a Medicare or accreditation review will provide the State a useful source of information to determine access to quality services for Medicaid beneficiaries. To better explain the types of information we are requiring be provided if a State chooses this option, and to address situations in which an entity is accredited by a private accrediting body approved by CMS under § 422.158, we have added clarifying language that makes a distinction between when a Medicare review is conducted by us or our contractor and when an accreditation review based on deemed compliance by such an approved entity. The findings of an accreditation review of an MCO or PIHP must be from a review of the Medicare line of business as this provision only applies to an M+C organization.

Comment: Many commenters recommended that MCOs that have established distinct provider networks for Medicaid and Medicare beneficiaries not be exempt from EQRs.

Response: As explained in an earlier response, we attempted to address differences inherent in Medicare and Medicaid contracts by requiring the contracts to have some geographic overlap. We do not believe, however, it is necessary or appropriate to require that Medicare and Medicaid beneficiaries of the MCO or PIHP use the same providers. We believe that an MCO or PIHP that demonstrates satisfactory compliance in M+C external review has demonstrated that it has appropriate quality safeguards in place, and that these would extend to all providers, whether seen by Medicare, Medicaid, or commercial enrollees.

We note that in providing for this exemption in section 1932(c)(2)(C) of the Act, the Congress did not require that Medicare and Medicaid enrollees use the same providers. It did require, however, that the entity have 2 years of Medicaid contracting experience. Under our interpretation of this requirement, discussed in a previous comment response, an MCO or PIHP would be required to demonstrate satisfactory results from 2 years of Medicaid EQR under part 438 before it would be eligible for the exemption under § 438.362. Thus, even if different providers are used by Medicaid enrollees than Medicare enrollees, the MCO or PIHP would have demonstrated for 2 years that the Medicaid providers performed satisfactorily in EQR before being exempted from this review. Having already demonstrated that its Medicaid providers met quality standards, the fact that it continues to satisfy quality standards in future years under Medicare external review is an indication that the entity is continuing its level of commitment to quality.

Comment: Many commenters recommended that the regulations specify that in the case of mergers and acquisitions, MCOs be treated as new contractors in the Medicaid program, and be subject to an EQR.

Response: We do not agree with the commenter that the regulations should specify that all MCOs and PIHPs that have been acquired or merged with another MCO or PIHP be treated as new contractors. There are a variety of scenarios that occur when a merger or acquisition occurs as indicated by the complex rules that govern how private accrediting organizations address these situations. In addition, States have their own laws and regulations governing mergers and acquisitions. We, therefore, believe the States are in the best position to determine quality improvement requirements for newly formed entities and this regulation provides States the option to allow for the exemption as long as all the provisions in this section are met.

Comment: One commenter asked that we revise § 438.362(b)(1) to specify that the State agency must require each exempted MCO to provide it annually with copies of Medicaid reviews performed by State agents or any private accrediting organization with respect to the quality, timeliness, and access to services instead of Medicare review findings.

Response: We are not revising § 438.362(b)(1) to require Medicaid review findings be submitted to the State because if a State or its agent conducted a review, there would be no need to require the MCO or PIHP to submit the review findings, as the State would already have this information. There is a need, however, for the MCO or PHP to submit Medicare review findings if a State chooses to exempt an MCO or PIHP from EQR, which is why this requirement is included in § 438.362(b). The exemption provision does not relieve a State from the responsibility for ensuring the adequacy of care provided by an MCO or PIHP, and the data from Medicare quality reviews are a source of information that will be necessary for States to use to determine the appropriateness of exempting an MCO or PIHP from an EQR the following year.

Comment: One commenter recommended allowing States the flexibility to decide if their Medicaid services can properly be evaluated by a Medicare review.

Response: States have the flexibility to determine if Medicaid services can be appropriately evaluated by a Medicare review. This provision provides States with the option to exempt an MCO or PIHP from EQR. It does not require the exemption.

J. External Quality Review Results (§ 438.364)

In § 438.364, we proposed a requirement that the product of EQR be a detailed technical report, containing (1) a detailed assessment of each MCO's and PHP's strengths and weaknesses with respect to quality of the health care services furnished to Medicaid enrollees, (2) recommendations for improving the quality of the services furnished by each MCO and PHP, (3) comparative information about all MCOs and PHPs as determined appropriate by the State agency, and (4) an assessment of the degree to which each MCO and PHP addressed effectively the recommendations for quality improvement, as made by the EQRO during the previous year's EQR. Proposed § 438.364 also specified that the State must provide the results of the EQR to members of the general public upon request, and that the information released may not disclose the identity of any patient.

Comment: One commenter suggested that, because of the differing nature of adult and child health care needs, all data produced during the course of an EQR should be available by age groups so that parents may choose an MCO on the basis of the provision of quality pediatric care.

Response: This rule requires information from a variety of activities to be provided to an EQRO and included in the analysis and evaluation of the care provided by MCOs and PIHPs. Not all of the EQR activities provide detailed information that can be broken out by age groups or other categories. For example, a review for compliance with structural and operational standards would not yield beneficiary specific information. However, encounter data could potentially provide that information. In addition, the populations served by MCOs and PIHPs are likely to vary along multiple dimensions, including age, income, diagnosis, and ethnic group. Because of the variability in the populations served by particular MCOs and PIHPs, we have provided States flexibility to determine the content of the results made available and the manner in which it is presented. To the extent that this information identifies quality issues pertaining to a specific population, the State may include that information in the results it makes available. However, we are not in the final rule requiring that EQR results be available by age groups, as this may not always be possible or appropriate for a given MCO or PIHP or for given data.

Comment: One commenter contended that not all quality improvement studies monitor quality, timeliness, and access. The commenter accordingly suggested that neither the State nor the EQRO should be required to summarize the strengths and weaknesses of the MCO or PIHP for each of these elements. The commenter also believed that if multiple studies are conducted, project time lines are not likely to coincide. In addition, the commenter recommended that proposed § 438.364(a)(5) be revised to require “An assessment of the degree to which each MCO has addressed effectively the recommendations for quality improvement as made by the EQRO during the previous measurement of the measure or of a similar measure, as appropriate to the study performed.”

Response: The commenter suggesting that the State or EQR should not be required to summarize strengths and weaknesses of an MCO or PIHP for “each of the elements” of quality, timeliness, and access implies that the results of the EQR process need not address all three of these areas. Because section 1932(c)(2)(A) of the Act requires that an annual EQR include all three of these elements, it is essential that strengths and weakness identified by the EQR process with regard to each are described in the results. Because there appears to be confusion on this point, we have revised § 438.364(a)(1) to specifically reference “timeliness and access.”

The commenter's suggestion that § 438.364(a)(5) be revised to permit the use of a “previous measurement of a measure,” as opposed to the previous year's EQR recommendations (as the baseline against which improvements in MCO or PIHP performance are assessed) is inconsistent with the clear direction of section 1932(c)(2) of the Act that EQR be an annual review. Further, the Medicaid managed care final rule requires performance measurement and improvement projects be underway on an annual basis. Consequently, we retain but modify the language of the proposed rule requiring the EQR to contain as assessment, as opposed to a “detailed” assessment of the degree to which each MCO and PIHP has addressed effectively the recommendations for quality improvement, as made by the EQRO during the previous year's EQR.

Comment: One commenter believed that the reference to “strengths and weaknesses” in proposed § 438.364(a)(2) implies a subjectivity that the commenter found inappropriate in carrying out the EQRO's responsibilities. The commenter recommended that the EQRO be required to report objectively on the performance of each MCO based on the measures selected. This commenter also questioned whether having an EQRO make recommendations for improving care and assessing the degree to which an MCO has met the previous year's recommendations are appropriate elements of the reports, because this is currently—and appropriately in the commenter's view—the province of the State (that is, identifying deficiencies in contract performance and holding MCOs accountable for correcting these deficiencies). The commenter requested that we exclude from the EQR reports, an EQRO's recommendations for improving care and assessing the degree to which the previous year's recommendations were met. If we retain these provisions, the commenter asked that § 438.364(a)(3) be revised to (1) allow the MCO the opportunity to submit a corrective action plan, which, if accepted would be adopted by the EQRO as its recommendation or (2) at a minimum, have the opportunity to comment on the EQRO's proposed recommendations. The commenter also suggested that § 438.364(a)(5) be revised so that the recommendations made by the EQRO are reviewed and approved by the State before finalizing the recommendations.

Response: We do not agree with the commenter that the report of EQR results should not address MCO and PIHP strengths and weaknesses. While we agree that the EQRO should consider the information produced by various EQR-related activities in an objective manner, the results of the analysis and evaluation of information will likely identify differences in the performance of MCOs and PIHPs with respect to issues under study. We believe that it is reasonable to expect the EQRO to be able to identify MCOs and PIHPs that had higher or lower scores on the State's standardized performance measures, and MCOs and PIHPs that had stronger evidence of compliance with certain standards. It is also reasonable for interested parties to expect this information to be publicly available. We note that this is common practice in the private sector where private accrediting organizations release comparative information on health plans.

We agree with the commenter that the State is the entity responsible for holding MCOs and PIHPs accountable for contract performance. The EQR is a source of information States can use to determine the adequacy of MCO and PIHP contractual performance regarding quality, timeliness, and access to services. The State may choose to require MCOs and PIHPs to submit corrective action plans based on the EQR results. In addition, as the State is the entity that holds the contract with the EQRO, the State may specify that it have the opportunity to review, comment, or approve the recommendations. The EQR results will be provided to us upon request, and will most often be requested and used by our regional office staff when conducting managed care program monitoring reviews. As a result, we retain the language included in the proposed rule.

Comment: One commenter concurred with proposed § 438.364, and specifically supported the requirement that EQR results (including assessments of MCO strengths and weaknesses and recommendations for improvement) be documented in sufficient detail and made publicly available. The commenter felt this was vital in order to allow interested parties to evaluate the conclusions of the EQR. Another commenter concurred with proposed § 438.364, and noted that the report required therein could be made available on the internet, to all interested parties, thus reducing the burden of report distribution.

Response: We agree with the commenters. Because the proposed language at § 438.364(b) could be interpreted to require the release of information in hard copy format only, in response to this comment we have modified the regulations text to indicate that the State must provide the information specified in paragraph (a) of this section, upon request, through print and electronic media, to interested parties.

Comment: One commenter noted that State staff currently perform the activities in paragraph (a)(2) of proposed § 438.364, and that requiring an EQRO to do this would increase the cost of the EQRO contract. The commenter also believed that the EQRO should not be making recommendations on improving the health care services furnished by each MCO, as specified under paragraphs (a)(3) and (a)(5) of proposed § 438.364. The commenter felt that the MCO should be responsible for designing interventions for improving its members' quality of care, and the EQR process should evaluate the effectiveness of these MCO interventions. Another commenter recommended these sections be deleted, contending that the Act does not require an external entity to perform any of the activities listed under paragraphs (a)(2) through (a)(5).

Response: As stated earlier, we agree that the State is ultimately responsible for rendering decisions about MCO and PIHP performance, and that EQR results represent one source of information States can use to determine MCO and PIHP performance. However, the Congress, in the BBA, stated that the EQRO is to perform a review of “the quality outcomes and timeliness of, and access to the items and services for which the organization is responsible.” The Congress further required that the results of the reviews be made available to multiple parties. We believe that a review requires the EQRO to make judgements regarding the MCOs' and PIHPs' performance in these areas and that the judgements can reasonably be expected to point to the MCOs' and PIHPs' strengths and weaknesses, recommendations about the quality, timeliness, and access to services provided by MCOs and PIHPs, and for how to make improvements. In order to enable the EQR process to be as effective and useful as possible, we retain these provisions in the final rule.

Comment: One commenter recommended that the regulation specify that the EQR results be made available in alternative formats for persons with sensory impairments, when requested.

Response: This comment appropriately suggests accommodations for persons with disabilities. At the end of § 438.364(b), in response to this comment we have added a sentence requiring States to make the EQR results available in alternative formats for persons with sensory impairments when requested.

Comment: Several commenters believed that while it may make sense to mandate disclosure of valid, reliable, and objective performance, and satisfaction measures, States should not be required to disclose the results of other health plan operations, such as contractual compliance, and quality improvement studies. In the view of these commenters, EQR activities should promote a frank assessment of performance in order to provide MCOs and PIHPs the knowledge necessary to perform better in the future. The commenters suggested that if the results of quality improvement studies were made public, MCOs would not treat the process as an unfettered opportunity to assess their own performance. Instead, the commenters believed they would tend to conduct studies in a way that is likely to generate favorable outcomes and, thereby, meaningful quality improvement efforts. One of these commenters also noted that if the primary audience for this information was Medicaid enrollees, we needed to consider whether such a detailed technical report would be relevant to our beneficiaries' needs.

Response: As we indicated in the preamble to the proposed rule, we proposed to require only that summary information made generally available is sufficient to enable interested parties to evaluate the conclusions of the EQR. The State is not expected to provide more detailed underlying data to beneficiaries or the general public. However, to clarify the level of detail to be provided in the EQR results, in response to this comment, we are revising § 438.364(a)(1)(iii) to require only that a description of data be provided in the technical report, as opposed to requiring that the actual data obtained be provided. Our intention was never to require that raw data be provided. In addition, as noted above, we are providing clarifying language in § 438.364(a)(1) to make clear that the technical report conclusions address timeliness and access to care as well as quality of care.

We note that section 1932(c)(2)(A)(iv) of the Act specifies that EQR results be made available to providers, enrollees, and potential enrollees. In the proposed rule, we broadened this requirement to specify that the results be made available to the general public. To ensure that adequate information is available for beneficiaries, as well as providers, beneficiary advocates, and other stakeholder, we believe that some detail in the report is warranted. In addition to making the EQR results available, States have the flexibility to repackage these results in order to address specific audiences more appropriately.

Comment: Many commenters agreed with our effort to ensure public access to EQR results. The commenters also recommended that the findings of private accreditation reviews be made available to the public when they substitute for all or part of the EQR. They stated that this is consistent with the President's Advisory Commission of Consumer Protection and Quality in the Health Care Industry recommendation that when a private accreditation is used, there must be full disclosure of the standards, survey protocols, and the detailed information from the surveys.

Response: Section 438.364 identifies the results of the EQR process that must be made available and to whom it must be made available. When an EQRO is using private accreditation or Medicare review results under the nonduplication option under § 438.360, the EQR results, in accordance with § 438.364(a)(1), must still include the information required under paragraphs (a)(1)(i) through (a)(1)(iv) of this section. We believe that when a State chooses to use the results of a Medicare or private accreditation review to replace a Medicaid review, that there must be information on the data obtained from the Medicare or accreditation review and conclusions drawn from the data consistent with § 438.364(a)(1)(iii) and (a)(1)(iv).

Comment: One commenter asked us to clarify whether the regulation envisions that the full technical report be available to the public, or whether only certain information about the technical report will be made available. The commenter recommended that we establish guidelines for preparation of a summary report that must be developed from the technical report. The commenter believes that a summary report will be more useful to the public and will avoid the potential for the release of proprietary information that might appear in the reports.

Response: As we stated in the preamble of the proposed rule, we are only requiring that States make available summary-level information that is “sufficient to enable interested parties to evaluate the conclusions of the EQR.” The State is not expected to provide more detailed underlying data or proprietary information to beneficiaries or the general public. As we noted earlier, to provide clarification on the level of detail to be provided in the EQR results, we are revising § 438.364 (a)(1)(iii) to require that a description of data be provided in the technical report as opposed to requiring that the data obtained be provided.

K. Federal Financial Participation (FFP) (§ 438.370)

Proposed § 438.370 provided that FFP would be available (1) at the 75 percent rate for EQR, the conduct of EQR activities, and the production of EQR results, by EQROs and their subcontractors, and (2) at the 50 percent rate for EQR-related activities performed by entities not qualifying as EQROS. The 50 percent rate applies even if the activities are of the same type as those that would be matched at the 75 percent rate if performed by an EQRO.

Comment: Several commenters asked us to clarify whether a State must contract with an EQRO in order to fulfill its EQR obligations under these regulations, and specifically whether it would fail to fulfill its obligation under the law if it contracts with an entity not qualified to be an EQRO.

Response: To fulfill its obligations under this regulation, a State must contract with an EQRO to conduct an analysis and evaluation of the aggregated information produced from, at a minimum, the mandatory EQR-related activities and produce the EQR results as required under § 438.364. In response to this comment, we have made clarifying changes to § 438.370 to better explain for what activities and functions States can obtain a 75 percent, or 50 percent match. That is, States can obtain the 75 percent enhanced match for EQR (the analysis and evaluation of information produced from EQR-related activities), EQR-related activities, and the production of EQR results as long as these functions and activities are conducted by an EQRO. States can obtain the 50 percent match for EQR-related activities conducted by entities not qualified as EQROs. However, States must contract with an EQRO that meets the requirements of § 438.354 to perform the EQR function of analyzing and evaluating the aggregate information from EQR-related activities. If a State did not so contract, it would be out of compliance with the requirement in section 1932(c)(2) of the Act for EQR.

Comment: One commenter asked whether the enhanced FFP is available for the optional activities a State may include in an EQR. Another commenter supported the enhanced FFP rates provided for in the Act.

Response: The enhanced FFP is available for the optional EQR activities as long as they are conducted by an EQRO that meets the requirements of § 438.354 using the appropriate CMS protocol or a consistent protocol.

Comment: One commenter requested clarification as to whether the upper payment limit (UPL) can be adjusted to take into account administrative expenses and if not, whether States will be able to request waivers of the UPL to reflect these additional expenses.

Response: The Medicaid managed care final regulation replaced the UPL requirements at § 447.361 with new rate setting rules (§ 438.6) by incorporating and expanding requirements for actuarial soundness. These new requirements recognize administrative costs and allow for States to adjust capitation rates to reflect MCO and PIHP administrative costs.

Comment: One commenter recommended that we revise § 438.370 to require States to appropriate a portion of the enhanced FFP to cover each MCO's administrative cost associated with meeting this EQR requirement.

Response: We believe that the statute does not permit States to use the enhanced funds to pay for MCO and PIHP administrative costs associated with EQR. The 75 percent enhanced match is only available for costs incurred by States for contracting with an EQRO. However, as noted above, with the elimination of the UPL, States now reflect administrative costs in capitation payments to MCOs and PIHPs.

Comment: One commenter asked us to clarify whether validation activities are reimbursable at the 75 percent enhanced FFP rate for EQR activities.

Response: The following validation activities are reimbursable at the 75 percent enhanced match as long as they are conducted by an EQRO that meets the requirements of § 438.354 and the EQRO uses protocols developed by us, or protocols consistent with our protocols: validation of performance measures, validation of performance improvement projects, validation of consumer or provider surveys, and validation of encounter data.

L. Miscellaneous Comments on the Preamble of the December 1, 1999 Proposed Rule

We noted in the preamble to the proposed rule that we followed two principles in its development: first, to provide flexibility to State agencies; and second, to reflect well-accepted advances in the methods of quality measurement and improvement.

The proposed rule also acknowledged that in a separate rule published in 1998, we had proposed to eliminate the requirements in § 434.53 that States have a system of periodic medical audits.

The proposed rule included a proposed effective date of 60 days following publication with provisions that must be implemented through contracts with EQROs to be effective with contracts entered into or revised on or after 60 days, but no longer than 12 months from the effective date. We received the following comments relating to the above issues.

Comment: Several commenters expressed support for the approach taken in the proposed rule in providing flexibility for States, and asked us to retain mechanisms States already have in place for EQR. Several commenters, however, found that the proposed rule did not afford States the flexibility and discretion afforded by the BBA. One commenter argued that States that demonstrate that their quality improvement processes meet or exceed the goals of these regulations should be permitted to continue with current arrangements. The commenter further contended that section 1932(c)(1)(B) of the Act, which requires that the Secretary's standards not preempt any State standards that are more stringent than those in the proposed rule, supports their position.

Response: Section 1932(c)(1)(B) of the Act refers to the quality assessment and improvement strategy that States are required to develop and implement. The components of this strategy were set forth in the Medicaid managed care final rule published on June 14, 2002. The EQR requirement is one component of this overall State strategy. We agree that the statute allows States to exceed the requirements of the quality assessment and improvement strategy as outlined in the Medicaid managed care final rule. However, the BBA also required the Secretary to undertake the activities set forth in this rule; that is, establish a method for identifying qualified entities to conduct EQR, develop protocols to be used for EQR, and otherwise implement the EQR provisions of the BBA. States will continue to have the flexibility to exceed the requirements included in this rule and conduct optional EQRO-related activities.

Comment: Several commenters asked us to explain how QISMC, the final Medicaid rules, and the EQR compose a cohesive vision and how States should integrate the proposed rule into other quality assessment and performance improvement activities. One of the commenters believed that the proposed rule appeared to set a standard for an overall evaluation rather than a specific external review study. Since QISMC sets overall standards, the commenter believed that a nonduplicative connection to QISMC was important. The second commenter asked us to clarify how the EQR regulations will fit in with current and pending State requirements.

Response: This final rule, as did the proposed rule, provides for an overall evaluation by an EQRO of the MCO's or PIHP's ability to provide timely and quality services to Medicaid beneficiaries as required by section 1932(c)(2) of the Act. The mandatory EQR activities are based on standards and activities that States must have in place under subpart D of the Medicaid managed care final rule.

Key elements of the QISMC document were incorporated into the Medicaid managed care final rule, as appropriate. However, in other instances the QISMC standards, which we previously offered to States as guidelines and not requirements, were not appropriate as requirements in the regulations text. Further, the QISMC standards in a number of ways have become outdated. For example, the QISMC document does not sufficiently address individuals with special health care needs. Individuals looking for a cohesive vision of a quality improvement system for Medicaid managed care should look to three documents: (1) The Medicaid managed care final rule, (2) this EQR final rule, and (3) the EQR protocols developed in response to the BBA statutory requirement. The QISMC document has been superseded by these three documents for the purposes of Medicaid. Each of these documents is accompanied by text describing how they should be integrated into State quality improvement systems.

Comment: One commenter contended that the proposed rule significantly reduced State flexibility in defining the content and cycle of EQR, exacerbated what the commenter considered a double standard for quality oversight between Medicaid FFS and Medicaid managed care, and placed new requirements on States not previously required of managed care programs. The commenter was concerned that this rule would create another reason to discourage MCOs and potentially PIHPs (especially those that provide behavioral health services) from participating in Medicaid resulting in fewer managed care options for Medicaid agencies and beneficiaries.

Response: We do not agree that this regulation significantly reduces State flexibility. EQR is not a new requirement on States. EQR has been a requirement for States contracting with MCOs since section 1902(a)(30)(C) of the Act was enacted in OBRA 1986. The BBA introduced new requirements for EQR and provided parameters we are obligated to follow in developing this regulation. The new requirement in section 1932(c)(2)(A)(iii) of the Act that protocols be developed which must be followed by States necessarily limits State flexibility to some extent. However, we believe that we have provided appropriate flexibility in implementing this statutory requirement. To do this, in collaboration with an expert panel that included State participants, we defined what activities we considered to be essential for an EQR. The statute also requires that EQR be conducted annually. While flexibility as the nature of review under EQR may have been limited somewhat by the requirement in section 1932(c)(2)(a)(iii) of the Act that protocols be followed, the new rule provides States with substantial new flexibility by allowing an expansion of the types of entities with which States can contract to conduct EQR activities, and extends the 75 percent match rate to these types of entities. In addition, this final rule allows a State to conduct EQR-related activities itself or through other State contractors. Thus, we do not believe that this rule will discourage managed care contracting.

Comment: One commenter expressed concern that the rule will limit a State's ability to maintain and improve distinct State quality initiatives due to more extensive Federal quality improvement initiatives. Specifically, the commenter believes the rule would require States to either externalize or duplicate ongoing State quality improvement activities.

Response: We do not believe that these EQR requirements will result in a duplication of any ongoing State quality improvement activities. A State may conduct any of the EQR-related activities internally or through other State contractors. The State will need to conduct the activities using our protocols or protocols consistent with ours if the information is to be used as part of the EQR. Therefore, at a minimum, our protocols or protocols consistent with ours must be used for the mandatory activities. As stated earlier, the protocols are generic instructions to ensure that the activities are conducted in a methodologically sound manner. If a State chooses to conduct EQR activities internally or have a State contractor other than the EQRO conduct the activities, the State expenses will be matched at 50 percent. States must contract with an EQRO for only one function, that is for the analysis and evaluation of the aggregated information provided from the EQR activities and the development of the EQR results. States can also continue to conduct other quality initiatives outside of the scope of EQR and claim the 50 percent administrative match.

Comment: One commenter contended that the proposed rule exceeded our statutory authority. Specifically, the commenter argued that with this rule, we effectively assumed control of a State's quality assessment and performance improvement strategy by specifying (1) the details of QI activities through detailed protocols developed without input from individual States, and (2) which activities can be performed by a State government entity, and which must be delegated to the EQRO. The commenter recommended that the proposed rule be withdrawn and redrafted to: (1) Allow for public review and comment of the protocols, and (2) permit States to carry out their statutory responsibilities as reflected in section 1932 (c)(1)(A) of the Act. The commenter also doubted that uniformity of EQR results could be accomplished in light of State programs that demand custom-tailored management and oversight models.

Response: We do not agree that we have exceeded our statutory authority in developing this regulation. The statute clearly required us to develop protocols to be used in the external review. We developed the protocols, as mandated, through an independent quality review organization with the guidance of an expert panel that included State representation, as required by the statute. A Federal Register notice announcing the completion of the protocols was published on November 23, 2001 (66 FR 58741). In that notice, we asked for comment on the extent to which burdens were imposed by the protocols, or on any other aspect of the protocols. Comments received from that solicitation, and our responses, are included in the preamble to this final rule.

We also believe we have provided significant flexibility to States as to which activities must be performed by an EQRO, as the only activity that must be conducted by the EQRO is the analysis and evaluation of the aggregated information produced from the EQR activities, and production of the results of that review as defined in § 438.364. The State can conduct the mandatory EQR-related activities, or have another State contractor conduct these activities, as long as the State uses our protocols or protocols consistent with ours.

Comment: One commenter believed that the EQR activities in the proposed rule were duplicative of the scope of work required in Independent External Evaluations of waivers under section 1915(b) of the Act, and recommended that the proposed rule be withdrawn until we develop a unified, coordinated approach to waiver oversight.

Response: The EQR activities in this rule are not duplicative of activities conducted as a part of independent assessments under section 1915(b) of the Act. The independent assessment requirement is a review of a State's mandatory managed care program under the authority of section 1915(b) of the Act. It reviews how adequately a State ensures access to quality services in the mandatory managed care waiver program, and the costs of the waiver program. The unit of analysis of the independent assessment under section 1915(b) of the Act is the State's managed care program as a whole, not individual MCOs or PIHPs. In contrast, the EQR review is a review of individual MCOs and PIHPs. The EQR requirement applies to all MCOs and PIHPs regardless of whether the program is voluntary or mandatory or whether it is authorized under a waiver. Further, EQR is conducted annually, whereas the review under section 1915(b) of the Act is conducted for the first 2-year period of the waiver, and the first renewal period (assuming the review results are acceptable). In addition, the independent assessment that we require in the case of a waiver under section 1915(b) of the Act applies to PCCM programs as well as programs with capitated arrangements. The EQR requirement does not apply to PCCM programs.

Comments: One commenter supported the proposed elimination of the requirement in § 434.53 for a system of periodic medical audits.

Response: While we note that this comment does not directly pertain to this proposed rule, we agree with the commenter. We believe that the system of periodic medical audits under § 434.53 is an out-dated approach to quality assessment and improvement which would be duplicative of EQR activities. (In this sense, the matter is relevant to this final rule.) Consequently, the Medicaid managed care final rule published on June 14, 2002 eliminated this requirement, as well as other regulations in subpart E of part 434.

Comment: Several commenters thought the proposed time period for bringing contracts into compliance with the new EQR requirements did not provide sufficient time for States. One commenter suggested that the new EQR rules apply to contracts entered into or revised on or after 90 days, but no longer than 18 months from the effective date. One commenter believed that States needed more than a year to implement this rule. One commenter recommended implementation of the redrafted rule on January 1 to be consistent with NCQA and other planning cycles and allow up to 180 days before implementation.

Response: To be consistent with the Medicaid managed care final rule, we have retained the effective date of this rule to be 60 days following its publication. However, we have revised the time frame for provisions to be implemented through contracts with MCOs, PIHPs, and EQROs so that they must be effective with contracts entered into or revised on or after 60 days following the publication date. States have up until no longer than 12 months from the effective date to bring contacts into compliance with the final rule provisions.

M. Collection of Information Requirements: December 1, 1999 Proposed Rule

In the December 1, 1999 proposed rule, we asked for comment on the following provisions that contain information collection requirements: nonduplication of mandatory activities (§ 438.360), exemption from external quality review (§ 438.362), and external quality review results (§ 438.364).

A. General Comments

Comment: One commenter contended that the burden to the MCO of working with the EQRO is not included.

Response: As part of the MCO and PIHP contracts with States, MCOs and PIHPs are required to work with States on a routine basis. This includes working with State contractors. We do not believe that working with EQROs adds burden for MCOs and PIHPs but continue to believe that it is part of the normal course of business for MCOs and PIHPs with Medicaid contracts. Further, a requirement for EQR is not new. It has been in place since the late 1980's under section 1902(a)(30)(C) of the Act.

Comment: One commenter felt that while the financial impact of this rule may be difficult to quantify, the proposed regulations would significantly increase the time and administrative burden on States, EQROs, MCOs, and PHPs well beyond the hourly estimates in the preamble.

Response: We do not agree that the regulation will significantly increase the time and administrative burden of States, EQROs, MCOs, and PIHPs beyond what we estimated in the proposed rule. Through our data and information collection, we know that the EQR-related activities referenced in this rule are those that are already typically required by States. Similarly, MCOs have previously been complying with EQR requirements subsequent to the enactment of section 1902(a)(30)(C) of the Act in 1986.

Section 438.360 Nonduplication of Mandatory Activities

Comment: Several commenters argued that the estimate of the total burden for the State for the proposed nonduplication provisions was too low, and asked how the estimate of 4 hours was determined. One commenter asked what data the MCO would need to provide to the State under proposed § 438.360(b)(2) and (c)(2).

Response: We estimated that it would take State staff approximately 4 hours to collect, copy, and disseminate the reports, findings, and other results of Medicare reviews or information obtained from the accreditation reviews and sent to the State. Because we received several comments indicating that this estimate was low, but commenters did not provide us with what they believe the estimate to be, we have increased the burden hours by 100 percent, to 8 hours. In accordance with § 438.360(b)(3) of the final rule, the MCO or PIHP needs to provide to the State any reports, findings, or results from an accreditation review or our review for Medicare for the standards in § 438.204(g) that are being substituted in place of a Medicaid review. In addition, if the MCO or PIHP provides services to dually eligible individuals and the State allows the MCO or PIHP to provide information from a Medicare review of performance measures and performance improvement projects for the EQR in place of separate Medicaid measures and projects, under § 438.360(c)(3), the MCO or PIHP will need to provide the results of Medicare review activities to the State.

Section 438.362 Exemption From External Quality Review

We did not receive any comments on the information collection burdens associated with complying with this provision.

Section 438.364 External Quality Review Results

Comment: One commenter noted that the preamble of the proposed rule addresses the burden of disseminating information, but not of creating the content listed. The commenter believed that the burden for creating the information required to comply with § 438.364(a)(2) would be significant, and would serve no purpose other than to comply with the rule. The commenter recommended deleting § 438.364(a)(2). Several commenters argued that the effort to compile and aggregate the data, analyze, and formulate the review reports will take a significant number of hours above the estimated number.

Response: The proposed rule did not address the burden of conducting EQR activities, because we had not completed the protocols at the time the proposed rule was published. A request for comment on the information collection requirement burden of the protocols was solicited in our November 23, 2001 Federal Register notice. We did, however, address in the proposed rule the burden associated with creating the EQR results report. We estimated that it would take 160 hours for an EQRO to prepare and submit the EQR results. Since we received several comments stating that it would take more time than the 160 hours we proposed, but commenters did not provide us with time estimates, we are increasing the burden hours by 25 percent.

We do not agree that the burden of § 438.364(a)(2) is significant, or that it serves no useful purpose. We believe that an assessment of the strengths and weaknesses of MCOs and PIHP performance as it relates to the quality, timeliness, and access to health care services was the intent of the statutory provision that requires the results of EQR be made available to beneficiaries and providers. We retain these EQR results provisions in the final rule.

N. Impact Statement

To comply with Executive order 12866 and the Regulatory Flexibility Act we examined the impact of the December 1, 1999 proposed rule. We determined that the net impact of the proposed rule would be below the $100 million annual threshold, and that a regulatory impact analysis was, therefore, not required.

Comment: Several commenters believed that the proposed rule would result in greater costs and burden to States and MCOs than we estimated in the impact statement. The commenters stated that we did not estimate the increased costs to States and MCOs for external review for compliance with standards. The commenter also felt that we did not consider the negative impact of external auditing on other MCO activities, or new and ongoing infrastructure and labor, needed to comply with these provisions. One commenter contended that these activities would require MCOs and their providers to devote significant staff time to collect, organize, and prepare for review of large quantities of quality assurance data. Another commenter felt that due to the independence requirements, the net results would be that fewer entities would qualify to conduct EQR.

Response: We do not agree with these comments. The only activity that must be conducted by an EQRO is the analysis and evaluation of the information obtained from the EQR activities. If a State chooses to, it can conduct any of the EQR-related activities and receive the 50 percent administrative match as long as the activities are conducted using our protocols or protocols consistent with those we developed. In addition, many States are already conducting or having State contractors conduct many of the EQR activities. As we stated in our proposed rule, most States are already obtaining a 75 percent matching rate for many of these activities and we, therefore, believe there will not be a significant increase in Medicaid expenditures, and that no new significant infrastructure will be needed. We do not believe that this requirement will cause MCOs to devote significantly more time to collect, organize, and prepare for EQR than is already required by States to ensure compliance with their contracts with MCOs and PIHPs.

Because this will be a new requirement on PIHPs, we acknowledged in the proposed rule that there may be additional cost to the Federal government, since States currently conducting these activities receive a 50 percent administrative match, but under this rule they can now obtain the enhanced 75 percent FFP. We do not believe these costs are significant. Based on an analysis of 2001 Quality Improvement Organization funding on the CMS-64, we estimate a cost of $5,800,000.

Comment: One commenter, while supportive of holding MCOs accountable by measuring quality of care, noted that there is no such requirement for the Medicaid FFS program, and that these costs are, therefore, not reflected in the rate-setting methodology for managed care plans. This commenter also noted that undertaking these reviews has a significant cost implication for both the MCOs and the State.

Response: The statutory quality assessment provisions implemented in this final rule do not apply to the Medicaid FFS program. Moreover, there is no statutory or legislative history to indicate that the Congress intended that these provisions should apply to Medicaid FFS. The Collection of Information Requirements and Impact Statement address what we believe to be the cost implications of this requirement as it pertains to Medicaid capitated programs. We note that in the Medicaid managed care final rule, a new methodology was adopted for setting capitation rates. This methodology permits States to reflect MCO and PIHP administrative costs (including costs of complying with quality assessment requirements that do not apply under FFS Medicaid) in capitation rates.

Comment: One commenter believed that requiring an independent organization to conduct a review of an MCO's structural and operational standards would add an additional administrative expense to the program.

Response: States currently review MCOs and PIHPs for compliance with State standards. If conducted by the State, this expense is reimbursed at a 50 percent administrative match. However, some States currently define this activity as part of EQR, and thus receive the 75 percent enhanced Federal match. Under the provisions of this rule, if a State chooses to contract with an EQRO to conduct a review of MCO and PIHP compliance with State standards, a State can obtain a 75 percent enhanced match rate. While this may increase Federal expenditures, we do not believe that the increase will be significant, as some States already have their EQROs conduct this activity. Thus, we do not believe this affects our conclusions regarding the need for a regulatory impact analysis.

Comment: One commenter believed that the proposed reporting requirement would increase costs.

Response: States currently have their EQROs develop reports. We believe that this will not add significantly to the current costs incurred by the Medicaid program.

Comment: One commenter believed that our proposed decision to extend EQR requirements to PHPs would increase costs to States, and that we have not fully analyzed this financial impact.

Response: We stated in our proposed rule that applying this provision to PHPs might result in additional costs. Although States are currently conducting a variety of quality activities with their PIHPs and receiving a 50 percent administrative match for their costs, they now may obtain the enhanced 75 percent FFP match for these activities. Again, while this will result in some additional Federal costs, State costs will decline. We do not believe these costs are significant. As stated in a previous response, based on an analysis of 2001 Quality Improvement Organization funding from the CMS-64, we estimate a cost of $5,800,000.

Comment: One commenter was concerned about the cost of responding to additional EQR requirements, and the potential for duplication and administrative burden to comply with QISMC, the Medicaid rules, and EQR rules.

Response: We do not foresee that there will be any duplication of effort between complying with the BBA provisions, including the EQR provisions, and QISMC. As we stated previously, QISMC has been superseded by the Medicaid managed care final rules that incorporate key elements of the QISMC document.

III. Collection of Information Requirements: November 23, 2001 Federal Register Notice: Discussion of Public Comments

Many of the comments we received in response to the November 23, 2001 Federal Register notice were issues pertaining to the December 1, 1999 proposed rule, as opposed to collection of information requirements or other issues concerning the protocols. Most of those issues were addressed in the previous section that responded to comments received on the December 1, 1999 proposed rule. This section addresses comments related to the burden estimates and any other aspect of the collection of information. We believe that burden estimates apply to the following sections of the regulation: EQR protocols (§ 438.352), Nonduplication of mandatory activities (§ 438.360), Exemption from EQR (§ 438.362), and EQR results (§ 438.364). We first address general comments.

A. General Comments

Comment: Several commenters did not agree with the methodology we used to estimate costs associated with implementing EQR. One commenter believes the methodology is flawed and our projected costs may be significantly lower than actual costs because our sample was too small and the range of estimates is too large for cost averaging. The commenter is also concerned that the methodology does not account for indirect costs such as rent, transportation, and medical record photocopies. The commenter recommended that indirect costs that account for geographic variation should be added to accurately predict the cost of using the protocols. One commenter stated that our approach did not include a determination of whether the function performed by the sampled EQROs approximated the functions that would need to be conducted in accordance with the protocols. The commenter further noted that because we estimated a range of hours for conducting EQR-related activities, we have not provided a representative assessment of the burden to perform the EQR activities. The commenter recommended we develop a more accurate projection of hours and costs associated with performing these activities consistent with the protocols.

Response: While the actual number of EQROs we interviewed was relatively small, as stated in our November 23, 2001 Federal Register notice, these EQROs had reviewed 16 managed care programs in 8 States (Arizona, California, the District of Columbia, Maryland, New Mexico, Nevada, Tennessee, and West Virginia). Each of these States contract with a different number of MCOs to provide Medicaid services, ranging from States contracting with a few MCOs to States with several dozen MCOs. So, even though the number of EQROs we interviewed was small, we believe we chose EQROs that represented a broad range of experience in terms of the number of MCOs they review, as well as representing an adequate geographic mix.

We also recognize that using a broad range of hours given by the interviewed EQROs to estimate the average number of hours it will take to conduct each activity may overestimate or underestimate the actual costs. However, by showing the ranges of costs we averaged, we show the variability across States that are inherent when conducting quality review activities. As stated above, we believe the interviewed EQROs represent an adequate number of MCOs reviewed. In addition, even though we did not specifically ask each EQRO about the methodology that they used to conduct the EQR activities, the protocols represent generic activities and steps that are followed in both the public and private sector. We, therefore, believe that the activities for which we collected cost information were conducted using a methodology consistent with our protocols. Moreover, we have no reason to believe that the interviewed EQROs' estimates provided did not include indirect costs for conducting EQR activities. Because the commenters did not suggest a specific methodology or what other data should be used in such a methodology, we retain the methodology used in the November 23, 2001 Federal Register notice. We have updated the estimates based on more current data on the number of MCOs and PIHPs contracting with State Medicaid agencies to provide services to Medicaid beneficiaries.

Comment: One commenter objected to our not including the time necessary for MCOs to collect and submit the information necessary to perform the functions identified under § 438.358, activities related to EQR. The commenter recommended that we interview health plans to determine the estimates for this activity and include them in our analysis.

Response: We agree with the commenter and include burden estimates in this final rule to address the time and costs associated with MCO and PIHP submission of information necessary for the validation of performance measures, validation of performance improvement projects, and a review for compliance with structural and operational standards. The protocols for all three of these activities require that documentation be provided by the MCO or PIHP. We do not anticipate, however, that new documentation will need to be developed. For example, the documentation review activity that occurs when a review for compliance with standards is conducted includes a review of reports, policies, and surveys that already exist. We believe that it will take each MCO or PIHP approximately 4 weeks of one full-time equivalent employee to prepare the information to be submitted for the three mandatory activities and we have added this estimate under § 438.352, the EQR protocols.

Comment: Two commenters believe the protocols will result in significant burdens in the areas of data collection, duplication of management oversight, and financial costs to the State and its contracting MCOs. One commenter estimated the new costs associated with the three mandatory activities and the overall EQR will be an additional $250,000 per MCO. Another commenter believes the cost per MCO would be approximately $424,000 for the three mandatory activities. The commenters noted there will be additional indirect cost incurred by the State to administer and oversee the EQRO contracts, and by the MCOs associated with the annual preparation for the three mandatory activities.

Response: We do not agree that the protocols will cause significant financial costs to MCOs and States, cause significant burdens in the areas of data collection, or duplicate other oversight activities. Many States already require their contracting MCOs and PIHPs to conduct performance improvement projects, calculate performance measures, and comply with State standards. The three mandatory activities that ensure compliance with these requirements are also already conducted by many States. However, States may not be contracting with their EQRO for the conduct of all these activities. As stated earlier in this preamble, the State can conduct these activities itself or contract with an EQRO or other entity for the conduct of the EQR-related activities. If the State contracts with an EQRO, it will receive the enhanced 75 percent FFP. If States are not currently contracting with their EQROs for these activities and decide to contract with their EQRO for EQR-related activities under this authority, it will decrease their costs related to quality activities, as opposed to increasing their costs.

We believe that the EQR mandatory activities can easily be incorporated into existing State quality assessment systems and will not duplicate existing oversight activities. The conduct of EQR and the conduct of EQR-related activities is required as part of the quality strategy under § 438.204 of the Medicaid managed care final rule and MCO quality assessment and performance improvement program requirements under § 438.240 of the Medicaid managed care final rule. Furthermore, we believe that there will not be additional costs incurred by the State to administer and oversee the EQRO contracts since this is already an existing requirement on States and MCOs under OBRA 1986. Because the commenters did not provide us with an alternative methodology to use or evidence to support their statement, we retain the approach taken in the November 23, 2001 Federal Register notice on the information collection requirements and in the impact statement in the December 1, 1999 proposed rule.

Comment: One commenter disagreed with our assumption that the implementation of EQR would not have an increased cost to the Federal government. The commenter did not agree that the costs incurred with current EQR activities are representative of costs that would be incurred under the new requirement. The commenter argued that States currently contract with EQROs for a more limited scope of activities.

Response: Our December 1, 1999 proposed rule acknowledged that there is likely to be an increase in Federal expenditures but that we did not anticipate this to be a significant increase. We agree with the commenter that the scope of work may be different under the BBA EQR requirements than it was under the OBRA 1986 requirements. However, we do not believe that the cost difference will be significant and it is likely that there could be a decrease. By expanding the pool of organizations available to conduct EQR, State agencies may be able to negotiate savings. We also hope that additional savings will be realized through opportunities afforded by this rule to coordinate EQR activities with other quality and oversight activities.

As stated in our December 1, 1999 proposed rule, we expect some increase in expenditures since we are applying the EQR requirement to PIHPs. We do not expect this to be a significant increase in expenditures because States already conduct quality review activities on PIHPs and receive a 50 percent FFP. Now States will be able to qualify for the enhanced 75 percent FFP.

Section 438.352 EQR Protocols—General Comments

Comment: One commenter believes the scope of the protocols could result in excessive burdens and they should be revised.

Response: For several reasons, we do not agree that the scope of the protocols will result in excessive burdens. First, all protocols are based on procedures already in use in the private sector. These protocols, therefore, are consistent with common industry practice in widespread use today. Second, many States and MCOs and PIHPs are already conducting these activities, using methods consistent with or more intensive than the activities and steps found in these protocols. For example, many State agencies are using the CAHPS surveys. The protocols for administering these surveys are consistent with our survey protocol, but much more prescriptive. Similarly, many States are also requiring validation of performance measures or encounter data using approaches consistent with these protocols. Third, the States have the option to use the protocols we developed or protocols consistent with ours. The protocols also include sample worksheets that can be used or modified at the State's discretion. Fourth, we note that States are only required to use three of the nine protocols that we have developed; the other six protocols are developed for optional activities that States can choose to undertake or not, at their discretion. For these reasons, we believe the protocols will not be excessively burdensome, and we retain the scope of the protocols as introduced through the November 23, 2001 Federal Register notice.

Comment: One commenter recommended that there be a better explanation of the use and purpose of the protocols.

Response: Section 1932 (c)(2)(iii) of the Act required us, in coordination with NGA, to contract with an independent quality review organization to develop protocols to be used as part of EQR. The purpose of the protocols is to provide EQROs with a set of generic instructions that ensure that EQR activities are conducted using sound methodological principles. To provide ongoing explanation about the use of the protocols, we have created a Web site at http://www.cms.hhs.gov/​medicaid/​managedcare/​mceqrhmp.asp that presents the protocols and an explanation of their intended use.

Comment: One commenter recommended that we not base the protocols on Federal or industry guidelines and standards, but that we incorporate these standards by reference.

Response: We disagree with the commenter. We purposefully directed our contractor to develop the protocols following protocols and quality review activities currently used in the managed care and quality oversight industries. We believe it is important to take advantage of the knowledge and experience that exists in the Medicare program and the private sector. Consistency with these approaches will also minimize the burden of complying with the protocols.

Comment: One commenter believes that the activities in this protocol will result in the State agency becoming the accrediting agency for Medicaid managed care, increasing the scope of prescribing and monitoring necessary by the State.

Response: We disagree with the commenter. The purpose of the three mandatory EQR-related activities is to ensure that MCOs and PIHPs are in compliance with §§ 438.204(g) and 438.240 of the Medicaid managed care final rule. However, many States currently conduct these activities. States that do not currently monitor for compliance with quality standards, monitor MCO and PIHP quality improvement projects or require the calculation of performance measures will need to initiate these activities. We believe that monitoring for these activities is consistent with the intent of the BBA EQR statutory provision to ensure that MCOs and PIHPs are providing access to timely and quality services.

Comment: One commenter believes the protocols are very clear in describing what information needs to be collected.

Response: We agree with the commenter and retain the activities and steps in the protocols introduced through the November 23, 2001 Federal Register notice.

Comment: One commenter believes that the protocols lack an evidenced-based approach to quality improvement. Another commenter believes that measuring MCO performance should be oriented to empirical performance outcomes and applied against quantifiable baselines and benchmarks rather than determining compliance through document reviews and interviews.

Response: We disagree with the first commenter. As we explained above, these protocols were developed consistent with protocols and quality review activities currently used in the managed care and quality oversight industries. Further, the protocols addressing performance improvement projects explicitly incorporate provisions addressing the use of clinical and nonclinical evidence in the selection of quality indicators. We agree with the second commenter that MCO and PIHP performance should be oriented towards performance outcomes that are measured against baselines and benchmarks. This is one reason why the information obtained from the validation of performance measures and the validation of performance projects is to be included as part of the EQR function. We also believe however, that a review of the MCO's and PIHP's compliance with State standards is essential for determining whether access to quality and timely services is provided. We believe this information used in conjunction with the information obtained from the validation of performance measures and performance improvement projects provides for both a qualitative and quantitative approach to assessing MCO and PIHP performance.

Comment: One commenter recommended that specific clinical areas (for example, early and periodic screening, diagnosis, and treatment (EPSDT) reporting) be addressed in multiple protocols.

Response: We believe that a variety of both clinical and nonclinical areas of care need to be assessed by the State and MCO or PIHP over time. However, we do not specify in regulation or in our protocols what those specific clinical and nonclinical areas should be because we believe that States should have the discretion to identify priority topics based on their knowledge of the public health priorities in the State, the health care needs of their beneficiaries, and based on discussions with beneficiaries and other stakeholders in the State. If we do decide that it is necessary to identify national priority topics, § 438.240(a)(2) of the Medicaid managed care final rule provides us with the authority to do so in consultation with States and other stakeholders.

Comment: One commenter asked that the protocols reflect our review criteria for children with special needs.

Response: When States require children with special health care needs to enroll in a capitated Medicaid managed care program, they must follow the review criteria provided in the January 19, 2001 State Medicaid Directors' letter. The Medicaid managed care final rule includes standards States must comply with when contracting with MCOs and PIHPs that enroll Medicaid beneficiaries, including children with special health care needs. These standards address the principles on which the review criteria are based. This protocol does not put forth any new standards, but identifies methods to determine compliance with current standards.

Comment: One commenter suggested that the protocols require the validation of performance measures submitted by MCOs, unless the measures were validated by a reliable entity using comparable standards.

Response: If performance measures are validated by an entity using an approach consistent with our protocol, only the information obtained from that review needs to be provided to the EQRO to be used as part of the EQR function. The review activity itself need not be duplicated. In addition, if the entity qualifies as an EQRO, the State can capture the enhanced 75 percent Federal match.

Comment: One commenter recommended that assessments of quality should include multiple sources of information including audits, certifications of sufficient networks and systems, and other submissions the MCO has provided to the State outside of the review process.

Response: We agree with the commenter that information from multiple sources should be included as part of the EQR. We believe we have accomplished this through the multipronged approach we have provided for in this final rule. The EQR will include information from the validation of performance improvement projects, the validation of performance measures, and a review for compliance with standards that may include plan network adequacy information, service authorization procedures, and other documentation that attests to the structural and operational components of the MCO or PIHP.

B. Protocol for Determining Compliance With Structural and Operational Standards

1. General Comments

Comment: The commenter believes that because we used a combination of private sector protocols in the development of the protocol for compliance with structural and operational standards, our protocol is likely to be more burdensome than that of any one private sector protocol.

Response: We reviewed a number of private sector protocols in the development of the protocol for compliance with structural and operational standards. We identified those elements common to all and used those as a basis for the protocol. Our protocol is not an additive combination of private sector protocols. Conversely, it is a synthesis or a streamlining of common elements found in multiple private sector protocols. Consequently, we do not believe our protocol is more burdensome than any one private sector protocol.

Comment: One commenter argued that CMS, for Medicare, is changing its onsite review process so this will be less frequent and more targeted. Medicare is also streamlining its review guide and will be reviewing less documentation and including more self-auditing by MCOs. The commenter recommended that we adopt a similar approach.

Response: The process for how this protocol will be used is set forth in this final rule, which contains provision for less frequent monitoring, and under certain circumstances, for the nonduplication of activities conducted under the Medicare program reviews or independent accreditation surveys. Through these regulatory provisions, we believe we have adopted a streamlined approach to quality review, similar to that used by Medicare.

Comment: One commenter is concerned that this protocol requires intensive onsite reviews to determine compliance with the structural and operational standards required in the Medicaid managed care final rule. The commenter believes that to meet the goals of EQR, it is not necessary to include all the areas identified in the monitoring protocol and that States should not be required to use this approach. One commenter believes that the guidance on the onsite review process is prescriptive and it is unlikely that the EQRO will need or use this detailed level of guidance. In general, the commenter believes the protocol is overly detailed and should be simplified to examine major structural and process requirements.

Response: The degree to which the protocol relies upon onsite reviews is consistent with the degree to which onsite review is used by private accrediting bodies. Therefore, we do not believe the onsite review specified in our protocol is too intensive. In the private sector, when an accrediting body has a standard, they monitor for compliance with it through a combination of interview activities and document review. We have followed this private sector approach and intend that all Federal requirements be monitored for compliance. Because the protocol contains only “potential” interview questions and documents for “potential” review, States, in using the protocol, will be able to target the reviews as they determine appropriate. We believe the protocol provides an appropriate amount of detail needed to reflect the scope and depth of the quality review activities to be conducted. We note in the protocol that, although the EQR activities must be consistent with the protocol, they need not be identical, thus providing the option for the States to prescribe a less detailed level of activity to the EQRO.

Comment: One commenter recommended that documents be obtained in advance and that multiple fact-finding efforts occur over time before conducting the onsite reviews. This allows State staff to be better prepared and is less disruptive for MCO staff.

Response: The EQR protocols are designed for use by EQROs which in many circumstances are not likely to be staffed by State personnel. However, State staff conducting compliance reviews may also use the protocols at their discretion. The protocols specify that documents may be obtained in advance, and reviewers, though not directed to do so, are not precluded from performing these activities over time.

Comment: One commenter recommended that the protocol include the review of previous monitoring reports and that the MCO's efforts and progress in correcting past problems be noted.

Response: We agree with the commenter. Therefore, in the final protocol, we have added that, before the onsite visit, reports on previous reviews and subsequent MCO and PIHP corrective actions be reviewed to identify areas on which the EQRO might need to focus the current monitoring activities.

Comment: One commenter recommended that the protocol include a mechanism for the State to prepare and submit oversight findings to the MCO and approaches to follow-up to ensure that corrective action has occurred. The commenter also recommended that every onsite review end with an exit interview to focus the MCO's attention on those areas the State is concerned about and intends to address in the findings and recommendations report.

Response: We agree with the commenter that evaluation results need to be reported to the MCO or PIHP. This reporting is common practice upon completion of a performance evaluation and a number of strategies are available for this reporting. We describe four possible alternatives for reporting in the protocol, but States are not precluded from selecting other alternatives that might include exit interviews with the MCO or PIHP at the conclusion of the onsite review.

Comment: One commenter recommended simplifying the compliance scoring system and placing greater emphasis on objective indicators of organizational performance such as performance improvement projects and survey results.

Response: We agree that other sources of information may provide information pertaining to MCO/PIHP compliance with the regulatory provisions, and we list some of these sources in the protocol under Activity 5, “Collecting Accessory Information.” In defining regulatory compliance, we have indicated that the State Medicaid agency will need to identify the level of compliance it requires and what rating or scoring system is to be used. In the protocol, we offer examples of common approaches, but because there is no evidence that one scoring system is better than all others, we allow States the discretion to select the scoring system to be used.

Comment: One commenter believes that of the four alternatives listed in the protocol for reporting evaluation results to the State Medicaid agency, neither the first nor the fourth alternative is acceptable. The commenter claims the first alternative makes information vital to the review; that is, the reviewers' analysis, unavailable to the State, while the fourth alternative represents a complete delegation of the State's monitoring responsibility to the EQRO.

Response: We do not agree with the commenter. In the first alternative, analysis is guaranteed based upon the definition of EQR in this final rule. According to that definition, EQR requires “the analysis and evaluation of aggregated information.” In the fourth alternative, reporting is accomplished based on pre-established State thresholds and guidelines, and therefore does not represent a complete delegation of the State's monitoring responsibility to the EQRO. The four alternatives listed in the protocol are possible scoring strategies; we state in the protocol that other options are available for use by States.

Comment: One commenter recommended that States require EQROs to use a standard written reporting tool.

Response: We agree with the commenter and have included a sample document and reporting tool (Appendix C, Attachment C of the final protocol) for this purpose. However, we allow States to modify this sample tool or develop another standard reporting tool, at their discretion.

Comment: One commenter noted that many questions are broad and not well written so the nature of the response being sought is unclear. The commenter recommended that the entire section for interviews should be reviewed in the context of whether the EQR rule is being exceeded by the data required during the interviews. Several commenters recommended that the interview section be dramatically shortened by eliminating duplicate questions and by deleting questions whose answers cannot be evaluated against the State's MCO contract specifications or a specific provision in the rule.

Response: We do not agree that we should more narrowly construct or abbreviate the interview questions. We have included a range of potential interview questions related to the subject matter of the regulatory provisions for reviewer use in prompting discussion. We expect, in practice, the reviewers will customize the interviews as necessary to clarify issues and confirm document findings. In the protocol, we compiled questions related to the regulatory provisions for each group of interview participants; for example, MCO or PIHP leadership, enrollee services staff. While this format creates some redundancy among the interview groups, we believe it facilitates the interviews by enabling each interview group's questions to stand alone. We also note that it is common practice in private accreditation reviews to ask the same or similar questions of different MCO or PIHP staff and also to review documents to support information obtained from interviews to determine if the information obtained from multiple sources converges and reaffirms the EQROs conclusions.

Comment: One commenter believes the protocols are bureaucratic and administratively burdensome and that there is a lack of evidence of the success of this type of process-oriented oversight. The commenter further stated that the level of detail is excessive to ensure conformance with MCO contracts and the BBA rule, and that the purpose is not for an accreditation.

Response: The protocols are based upon the common elements found in compliance protocols used by private sector accrediting bodies and the Medicare program. Consequently, we do not believe they are overly bureaucratic, administratively burdensome, or without a sound evidentiary basis. We also have followed the private sector approach in specifying that all standards, in this case the Federal requirements, be monitored for compliance. We believe the protocol provides an appropriate amount of detail needed to reflect the scope and depth of the quality review activities to be conducted. We note again that the specific interview questions are suggestions only, and we expect the questions to be customized for each review.

Comment: One commenter claimed that some informational items the EQRO is to collect from the State Medicaid agency do not exist as contract provisions and may not exist as other standard documents. This will create additional paperwork. The commenter recommended that the EQRO should only verify that the State's managed care contracts require compliance with applicable State and Federal laws.

Response: We do not agree with the commenter. The background information that the EQRO will need to collect from the State under this protocol includes written documentation of those standards, requirements, or decisions pertaining to MCOs and PIHPs that the State established to comply with the regulatory requirements that implement the BBA provisions governing standards for contracts with MCOs and PIHPs. This information is needed to assess MCO or PIHP compliance with those regulatory provisions for which the State is required to establish certain standards.

Comment: One commenter claimed that the number and types of documents the EQRO is to obtain from the MCO are too extensive and that many of the Code of Federal Regulations citations used to justify the collection of documentation are incorrect and do not relate to the topic. The commenter recommended that the protocol be reviewed for incorrect citations and references and that corrections be made.

Response: We do not agree with the commenter. We believe the documents listed are those needed to evaluate MCO or PIHP compliance with the Medicaid regulatory provisions. The regulatory provisions cited indicate where information obtained from the documents can be applied in the review process. For example, although § 438.214 pertains to credentialing and recredentialing, this provision is applicable to oversight of delegated activities, if the MCO or PIHP delegates credentialing to another entity.

Comment: One commenter recommended that Appendix B to this protocol have a cross-reference table that summarizes each interview question with the respective oversight organization documentation listed.

Response: We believe the format for the protocol itself is generally comparable to the recommended cross-reference table for Appendix B (Attachment B of the final protocol). The protocol includes a table cross-walking the review documentation with the related regulatory provisions. The subsequent interview sections then aggregate the interview questions by regulatory provision for each interview group.

Comment: One commenter was concerned that we do not include information available from consumers as a source of information to be used in this protocol. Several commenters believe this protocol does not go far enough to examine actual practices of MCOs' or beneficiaries' experience with care; rather, it focuses on policies and procedures. One commenter recommended the protocol include interviews with State Medicaid personnel and providers, and input from consumers, consumer advocates, and people with special health care needs.

Response: We agree that providers, consumers, and others mentioned may offer further information about MCO or PIHP performance; however, interviewing these groups requires additional time and substantial resources. Therefore, in this protocol, we have made provider and contractor interviews optional. However, we have further promulgated a separate protocol for the use of provider and consumer surveys as a source of information that can be used for EQR at the option of the State. We believe that mandating additional surveys as a part of this protocol would be burdensome and unnecessary.

Comment: One commenter believes the MCOs can prepare in advance for the review. The commenter recommended reviewers should interview providers and beneficiaries not preselected by the MCOs to ensure compliance with established policies.

Response: We agree with the commenter's concern regarding preselection. For the reasons previously noted, however, provider interviews are an optional part of this protocol. Consumer and provider surveys are also specified as a separate, optional EQR-related activity for securing input from beneficiaries and providers.

Comment: One commenter recommended that among document review and interviews, we include in our approach extensive file review.

Response: We are unsure what files the commenter is proposing for review. The approach used in the protocol is the same approach used by the private sector accrediting bodies and in the Medicare program. If the commenter is referring to medical record review, these are included and discussed in the protocols for validating and conducting performance improvement projects and validating and calculating performance measures.

Comment: One commenter suggested that because a core component of quality programs is responsibility for the program at the highest level of the organization, we include a discussion of committee structure and committee oversight in the overview section.

Response: We assume the commenter is referring to the MCO or PIHP's quality assurance committee and oversight. The protocol addresses compliance with the standards required in the Medicaid managed care final rule. Because committee structure and committee oversight as a core component of quality programs is not included as a standard in the Medicaid managed care final rule, it would not be appropriate to require it in the protocol.

Comment: One commenter believes that the pertinent issue in team development (p. 6 of the protocol) is the identification of the specific functions to be reviewed and the assignment of appropriate personnel to the task, not the size of the team.

Response: We agree that an important consideration in the development of the review team is the determination of the types of personnel appropriate for the review as related to the functions to be reviewed. Therefore, we have specified the desirability of reviewers possessing knowledge of Medicaid and managed care, and experience and familiarity with the regulatory provisions, the evaluation process, and performance expectations.

Comment: One commenter recommended that we include in the list of documents on page 18, committee minutes, vendor oversight committee, and committee structure of the quality program.

Response: The list of documents on page 18 refers to the documents used for determining compliance with specific regulatory provisions. Because the commenter has not stated what regulatory provisions these documents would be used to address, we are unclear as to how to propose their use and have not included them in the document list.

2. Provider/Contractor Services

Comment: One commenter recommended that the review of credentialing files by the EQRO be deleted because the criteria for auditing the files are inadequate. The commenter recommended that the element be simplified to call for the EQRO to review MCO credentialing policies and procedures for conformance with State contract requirements.

Response: We disagree with the commenter. We believe that a review of policies and procedures alone, when the opportunity exists to review documents providing direct evidence of compliance or noncompliance with the policies and procedures, is a more effective review mechanism. This is consistent with the approach used by private sector accrediting bodies and in the Medicare program.

3. Staff Planning/Education/Development

Comment: One commenter suggested that the requirement for the MCO to produce staff handbooks and information about staff training and orientation be dropped for lack of specificity or rewritten to make clear what criteria the auditors are to use in reviewing the required materials.

Response: We indicate on the list of documents the regulatory provisions to which each document applies. In this instance, staff handbooks and information about staff training and orientation pertain to the requirement that staff be educated about the enrollee's right to receive adequate information; for example, information on disenrollment rights and hearing and appeals. We have specified interview questions for MCO/PIHP leadership, provider and contract services staff, and enrollee services staff concerning how appropriate staff are informed regarding the enrollee right to information. We believe this provides sufficient clarity with respect to the criteria reviewers are looking for and we retain the references to the staff handbook, staff training, and orientation.

Comment: One commenter suggested that the interview questions include probes to determine how staff are trained to comply with Federal and State laws, and how staff advise enrollees of their rights. The commenter recommended further that interview questions address the content, frequency, and thoroughness of the training to confirm no major area of law is overlooked.

Response: We have specified staff handbooks, and orientation and training curriculum, in the list of documents to be reviewed and included interview questions to confirm MCO/PIHP compliance with the regulatory requirements pertaining to enrollee rights and compliance with Federal and State laws. However, if issues arise during the document review concerning the adequacy of the staff's training regarding these provisions, reviewers are directed to explore them during the interviews. We believe this direction affords the reviewers the flexibility necessary to appropriately tailor the review activity. Further, we do not believe it is possible, given the diversity among States and MCO/PIHPs and the scope of the review itself, to include in the list of potential interview questions probes to explore all applicable State laws.

4. Consumer Protections

Comment: One commenter recommended that the protocol include the monitoring of the Medicaid managed care final rule provisions related to consumer protections. The commenter specified for inclusion provisions addressing: the free choice of providers for family planning services (§ 431.51); prohibition on provider discrimination (§ 438.12); availability of out-of-network providers in rural areas (§ 438.52(b)); disenrollment rights as a result of grievance procedures, and related notice and appeal rights (§ 438.56(d) and (f)); enrollee rights regarding treatment, second opinions, and medical record access and correction (§ 438.100); marketing activities (§§ 438.104, 438.700(b)); liability for payment beyond what is legally allowable (§ 438.106); program integrity requirements (§ 438.608); imposition of sanctions (§ 438.700); and multiple charges and denial of services for inability to pay cost sharing (§ 447.53).

Response: We have listed in the protocol documents for review to determine compliance with regulatory provisions related to prohibition on provider discrimination; disenrollment rights as a result of grievance procedures, and related notice and appeal rights (§ 438.56(d)); and enrollee rights regarding treatment, second opinions, and medical record access and correction. We further agree with the commenter and have amended the protocol to include review of the MCO/PIHP's relevant policies and procedures to assess compliance with the regulatory requirements pertaining to the free choice of providers for family planning services; liability for payment beyond what is legally allowable; and multiple charges and denial of services for inability to pay cost sharing. However, the provisions concerning availability of out-of-network providers in rural areas; marketing activities (§ 438.700(b)); program integrity requirements (§ 438.608); and imposition of sanctions (§ 438.700) are responsibilities of the State and not the MCO/PIHP and, therefore, we have not included them as a focus of this protocol. The regulatory requirements in § 438.104, while they pertain to MCO/PIHP marketing activities, are contract requirements that do not directly provide information on quality and are more particular to a State responsibility. Because the protocol is designed to determine MCO/PIHP compliance, we believe it would not be appropriate to monitor these latter activities through the protocol.

5. Enrollee Services

Comment: One commenter believes a State can contract with the MCO to provide information to potential enrollees, and recommends the protocol monitor the MCO's compliance with these informational requirements.

Response: In the August 20, 2001 Medicaid managed care proposed rule, we stated that “it would be unreasonable to require every MCO/PIHP to provide the relevant information to all potential enrollees.” We believe the MCO/PIHP should not be contracted by the State to undertake this responsibility, and explained in the proposed rule that “the State agency is the more appropriate entity to do” the potential enrollee informing. This requirement was, therefore, not included in our Medicaid managed care final rule and we are not changing the protocol to monitor the MCO's/PIHP's compliance with providing information to potential enrollees.

Comment: One commenter recommended the protocol include a standard reflecting the regulatory requirement for the provision to enrollees of information on services not provided due to moral or religious objections.

Response: We agree with the commenter. The protocol identifies the section of the regulation that requires enrollees to be provided with information about services that are not provided by the MCO or PIHP because of moral or religious objections. It also identifies relevant documents to be reviewed to determine compliance (see pages 22 and 77 of the protocol). These documents include Medicaid enrollee service policies and procedures, statement of enrollee rights, and marketing materials.

Comment: One commenter believes the protocol should include guidance on how to measure the adequacy of the MCO's activities to inform enrollees. The commenter recommends the protocol include additional guidance on the fourth grade reading-level standard for materials, and confirmation that written materials are at an understandable grade level and in alternative forms to accommodate individuals with sight impairments.

Response: We note that we have provided guidance on this issue in the August 2001 proposed Medicaid managed care rule. In the preamble to the August 2001 proposed rule, we indicated that materials should be understandable to enrollees at a fourth to fifth grade reading level, or at another level established by the State agency that adequately reflects the potential population to be enrolled. Materials should use an easily readable typeface, frequent headings, and should provide short, simple explanations of key concepts. Technical or legal language should be avoided whenever possible. We proposed further that enrollment notices as well as informational and instructional materials relating to enrollment take into account the specific needs of enrollees and potential enrollees, including furnishing information in alternative formats for the visually impaired and for individuals with limited reading proficiency. Also, in 1999, we developed and distributed to the State Medicaid agencies and made available to others a guide entitled, “Writing and Designing Print Materials for Beneficiaries: A Guide for State Medicaid Agencies.” The guide was produced to assist States and MCOs/PIHPs in the creation of materials appropriate for their Medicaid populations. We believe the guidance that we have provided in the August 2001 proposed rule and through this guide is appropriate and reflects the current state-of-the-art. Because there is no state-of-the-art standard to apply in measuring the adequacy of the MCO's/PIHP's efforts to inform enrollees, we decline to do so in this protocol.

Comment: One commenter recommended that we monitor the States' definition of what constitutes a “significant change” in certain MCO structural and operational features to ensure the State's definition of “significant change” is reasonable and fair to enrollees, and that we provide guidance on what parameters a State can use in setting the definitional standards.

Response: The protocol addresses the extent to which an MCO/PIHP, as opposed to the State, complies with the requirements in the Medicaid managed care final rule. Section 438.10(f)(4) of the Medicaid managed care final rule specifies that the definition of “significant change” is the State's responsibility. It, therefore, would not be appropriate to include in the protocol the monitoring of the State's definition. Monitoring of States occurs through separate activities conducted by our regional offices. Further, as we stated previously, the protocol is not intended as a mechanism to impose additional quality standards on MCOs/PIHPs or States. Therefore, we do not believe it appropriate to provide guidance in the protocol on what parameters a State can use in setting the definitional standards.

Comment: One commenter noted that the interview questions are good initial probes, but suggested the protocol include additional guidance to more fully probe the MCO's dissemination of enrollee information, and require interviews of providers and enrollees regarding the quality of the informational materials.

Response: We specify in the protocol that reviewers should tailor the interviews as necessary to clarify and confirm document findings. We believe this direction affords the reviewers sufficient flexibility to more fully probe areas as appropriate. Further, we do not believe it is possible, given the diversity among States and MCOs/PIHPs and the scope of the review itself, to include in the list of potential interview questions probes to explore every possible problem or issue that might arise. Provider interviews are time and resource intensive, but because they offer an opportunity to secure additional information regarding MCO/PIHP performance, we have included them as an optional activity if informational needs warrant them and resources permit. We provide for the consideration of enrollee input by including the review of the results of Medicaid beneficiary surveys as accessory information under Activity 5.

Comment: One commenter believes the protocol does not adequately address linguistic issues. The commenter recommended that the review confirm that MCOs collect required language information on enrollees and recognize non-English speakers in all transactions. The commenter suggested further that the protocol include the review of documentation regarding professional translations of written materials, and interviews to assess the quality of the written translations and the MCO's oral interpretation practices and resources.

Response: We believe the protocol does adequately address linguistic issues. In Appendix B (page 79, Attachment B of the final protocol), among the materials to be obtained from the State, we include information on the language(s) that the State Medicaid agency has determined are prevalent in the MCO's/PIHP's geographic service area. On page 85, we direct the reviewer to look at marketing, enrollment and other informational and instructional materials relating to enrollment, enrollee handbooks, new enrollee materials, statements of enrollee rights, and other written materials routinely prepared for Medicaid enrollees and potential enrollees to determine whether these materials are available in the language(s) that have been identified as prevalent within the MCO/PIHP's particular service area. Further, the Medicaid managed care final rule at § 438.204(b)(2) requires States to identify the primary language spoken by each Medicaid enrollee and provide this information to the MCO/PIHP at the time of enrollment. Finally, we believe requiring EQRO re-review of translated materials is more burdensome than appropriate and therefore have not included it in the protocol.

6. Enrollee-Provider Communication

Comment: One commenter objected to the implication that by contract MCOs may place limits on providers' communication with enrollees about reproductive health services. The commenter recommended that the protocol include document review and interview questions to address whether reproductive health services are provided and whether restrictions are placed on provider communication. The commenter suggested further that for MCOs that exclude any reproductive health services the State monitor enrollee access to the full scope of services. The commenter noted a potential correlation between restricted access to reproductive health care services and poor outcomes in other women's health areas, and recommended the State monitor related health outcomes and comparison of rates to those of MCOs without restrictions.

Response: Appendix B of the protocol (Attachment B of the final protocol) specifies documents for review and interview questions to address whether the MCO/PIHP has any moral or religious objection to providing, reimbursing for, or providing coverage of, a counseling or referral service for a particular Medicaid service or services. This would include reproductive health services. For counseling and referral services the MCO/PIHP does not cover because of moral or religious objections, the Medicaid managed care final rule at § 438.10(f)(6)(xii) specifies that it is the State's responsibility to provide enrollees with information on where and how to obtain the service(s). The protocol is designed to address MCO/PIHP compliance with the BBA regulatory standards. Consequently, State monitoring of enrollee access to the full scope of services and State monitoring of health outcomes in other women's health areas for enrollees with restricted access to reproductive health care services, and comparison of these rates to those of MCO/PIHPs without restrictions is beyond the scope of the protocol.

7. Emergency Services

Comment: One commenter suggested that the interview questions concerning inappropriate use of emergency rooms emphasize a comparison of their inappropriate use with access to routine and urgent care.

Response: We agree with the commenter and have therefore expanded the relevant interview questions in Appendix B of the protocol (Attachment B of the final protocol) under § 438.210 that addresses coverage and authorization of services to inquire about the potential relationship between inappropriate emergency room use and enrollee access to routine and urgent care.

8. Delivery Network

Comment: One commenter recommended that the protocol, in reviewing the MCO's/PIHP's network of appropriate providers, consider specifically the providers needed to meet the needs of pregnant women, children and individuals with special needs, particularly those targeted for enrollment.

Response: In the Medicaid managed care final rule at § 438.206, we require the MCO/PIHP to establish a network of appropriate providers that considers the “expected utilization of services, considering Medicaid enrollee characteristics and health care needs.” We intend and expect that MCOs and PIHPs that serve pregnant women and individuals with special health care needs will consider their characteristics and needs. However, we do not explicitly identify them in this protocol because they are not explicitly mentioned in the regulation in this provision and because not all MCOs and PIHPs may serve pregnant women and individuals with special health care needs.

9. Access

Comment: One commenter suggested that the review address transportation services to network providers and out-of-network providers for enrollees without access within established time and distance standards, and for enrollees with disabilities and special needs.

Response: The regulations do not contain standards for the provision of transportation services to network or out-of-network providers, or for enrollees with disabilities and special needs. In addition, transportation is a service that may or may not be included under the MCO/PIHP contract. Therefore, in the protocol's document review and interview questions, we include only those transportation issues addressed in the regulation.

Comment: One commenter recommended that the monitoring of access to out-of-network providers include a review of the procedures for determining when in-plan access is unavailable and out-of-network services are appropriate; obtaining access to out-of-network services; and for providing in-plan services for enrollees denied out-of-network access.

Response: The protocol specifies a review of the MCO's/PIHP's administrative policies and procedures pertaining to the use of out-of-network providers. Although we reference documents by generic name or title, we explain that what is important is the presence or absence of evidence to determine compliance with the specified regulatory provision. We anticipate reviewers will use the relevant documents to determine compliance with all aspects of the regulatory provision regarding out-of-network access including those identified by the commenter.

Comment: One commenter suggested that the document review include policies, procedures, and criteria for determining that second opinions are rendered by qualified providers.

Response: We agree with the commenter. The protocol specifies a review of the MCO's/PIHP's administrative policies and procedures for providing enrollees with a second opinion from a qualified health care professional. As previously indicated, although the documents are referred to by generic name or title, we explain that what is important is the presence or absence of evidence to determine compliance with the regulatory provision. We anticipate reviewers will use the relevant documents to determine compliance with all aspects of the regulatory provision requiring that second opinions are rendered by qualified providers.

Comment: One commenter recommended that the document review related to direct access to women's health services be expanded to include materials produced by the State to inform MCOs and by MCOs to inform providers. The commenter suggested further that the review include policies and procedures for implementing direct access to these services.

Response: Within the review of enrollee rights, the protocol specifies a review of staff and provider orientation, education, and training curricula and materials, and other provider and staff communication tools for evidence that staff and providers consider, among the enrollees' rights, direct access to women's health services. We also specify the review of the results of MCO/PIHP monitoring of complaints and grievances, enrollee survey or other MCO/PIHP sources of enrollee information to detect violations of enrollee rights, including the provision of direct access to women's health services. However, we do not include in the protocol a review of materials produced by the State because the protocol is a review of MCOs or PIHPs, not State Medicaid agencies. Review of State compliance with Federal requirements is carried out by our regional office staff through a separate process.

Comment: One commenter recommended that the reviewer monitor the time it takes for enrollees to obtain appointments with network providers.

Response: We agree with the commenter. Our protocol directs the reviewers to obtain the State Medicaid agency's standards for timely access and to review documents showing how the MCO/PIHP ensures compliance and continuously monitors its network providers for compliance with the timely access standards. The protocol lists some acceptable mechanisms the MCO/PIHP may use for monitoring compliance.

Comment: One commenter suggested that inappropriate use of emergency rooms be evaluated according to the “reasonable lay person” standard. The commenter also recommended that the monitoring of emergency room use consider access to nonemergent care and follow-up outreach and education for enrollees using emergency rooms for nonemergency care.

Response: The protocol monitors MCO/PIHP application of the prudent layperson standard in the regulation at § 438.114. As we indicated in our response to a previous comment on emergency room use, we have added an interview question to inquire about the potential relationship between inappropriate emergency room use and enrollee access to routine and urgent care. However, MCO/PIHP follow-up outreach and education for enrollees using emergency rooms for nonemergency care is not a regulatory requirement, and it would be inappropriate to include it in the protocol.

Comment: One commenter suggested expanding the protocol's activities to include the review of training curricula and materials on cultural and linguistic competency, including the scope and depth of the training, its frequency, and extent of staff attendance; the procedures for the translation and testing of enrollee informational materials; and arrangements with community-based organizations representing relevant ethnic groups.

Response: We disagree with the commenter. Our protocol addresses the extent to which an MCO/PIHP complies with the regulatory provisions that implement the Medicaid managed care sections of the BBA. The Medicaid managed care final rule, at § 438.206(c)(2), requires that MCOs/PIHPs participate in the State's efforts to promote the culturally competent delivery of services. Therefore, the protocol specifies a review of documents for evidence of the MCO's/PIHP's participation in the relevant State efforts. The inclusion of additional requirements not required by regulation within the protocol would be inappropriate.

10. Coordination & Continuity of Care

Comment: One commenter recommended that the review of coordination and continuity of care include interview questions regarding the provision of any specialty care services currently not provided in-network, and MCO efforts to make these services available in-network. The commenter also suggested that the interview questions be expanded to inquire what proportion of Medicaid enrollees with special health care needs have a person or entity formally designated as primarily responsible for coordinating their health care services.

Response: We agree, in part, with the commenter. Consequently, we have added an interview question for the organization leaders to inquire about the provision of any specialty care services currently not provided in-network. We have not added questions about MCO or PIHP efforts to make these services available in-network because it is not clear whether or not it is always necessary that all specialty services be provided by in-network providers. We have added additional potential interview questions for enrollee services staff to determine what proportion of Medicaid enrollees with special health care needs have a person or entity formally designated as primarily responsible for coordinating their health care services.

Comment: One commenter believes the protocol should differentiate between gatekeeping activities that are involved with utilization control and care coordination and case management functions that are related to supporting service access and coordination. The commenter believes further that reviewers should consider the MCOs' scope of responsibility for EPSDT case management, and how these services are provided or referrals are made.

Response: We agree with the commenter that a State may want to differentiate between care coordination models. In so doing, a State may decide to explicitly address care coordination for EPSDT care management. We specify in the protocol that MCOs/PIHPs may establish different coordination mechanisms, and in monitoring for compliance with the requirements for care coordination, direct the reviewers to obtain the State's requirements for MCO/PIHP care coordination programs.

Comment: One commenter recommended that the interview protocol address how and who conducts the MCOs' health screens; how the MCO assesses enrollee needs and determines if the provider is qualified to perform the assessment; how enrollees access case management services; how an enrollee's need for a treatment plan is determined; and how the providers are informed of the process. The commenter also suggested additional interview questions to address the number of treatment plans developed by categories of individuals, the number of denied requests for treatment plans and the reason for denial, and the number of treatment plans denied.

Response: The protocol includes interviewer questions for the case managers and care coordinators and for the enrollee services staff regarding the implementation of health screens, the conduct of health assessments for Medicaid enrollees, processes for care coordination, and procedures to determine how an enrollee's need for a treatment plan is determined. The protocol's interview questions for the provider/contractor services staff probe how providers are made aware of and are involved in procedures for assessments, treatment planning, and care coordination. We agree with the commenter regarding the need to explore the MCO's/PIHP's treatment planning. We have revised the protocol to include a series of questions for the case managers and care coordinators concerning the number of treatment plans developed, the number of denied requests for treatment plans and the reason for denial, and the number of treatment plans denied. However, our revision will not include a review of the treatment plans by categories of individuals. We do not require specific categories and, therefore, have no standard against which to measure the MCO's/PIHP's performance.

11. Prior Authorization

Comment: One commenter believes the protocol should include a review of prior authorization procedures and policies and a determination of their reasonableness, reflection of good medical practice, and timely application. The commenter suggested reviewers monitor the number of and reasons for delayed expedited requests, and the health consequences associated with prior authorization delays and denials of expedited authorizations. The commenter further believes the MCOs' informal communications with providers should be monitored, including the handling of provider telephone inquiries, resulting changes to the course of treatment, and provision of enrollee notice and appeal rights.

Response: We agree with the commenter regarding the need to determine compliance with the requirement for timely prior authorization decisions, and therefore have included in the protocol document review and interview questions to determine compliance. However, the regulations include no standards for the reasonableness of the policies and procedures or for their reflection of good medical practice; these issues are therefore beyond the scope of the protocol that is designed to assess compliance with the Medicaid managed care regulatory requirements.

We also agree with the commenter's suggestion to review the number and reasons for delayed expedited requests. We have revised the document review for service authorizations to include the review of tracking logs or other authorization record-keeping documents to address number and reasons for delayed expedited requests.

We do not agree with the suggestion to monitor health consequences associated with prior authorization delays and denials of expedited authorizations. We believe that determinations on whether health consequences were due to authorization delays or denials, or to the normal progression of the enrollees' health condition would be subjective. Further, States are required to maintain records of grievances and appeals and review this information as part of the State quality strategy. If enrollees' health outcomes are adversely affected by the MCO's/PIHP's handling of service authorization requests, this should become evident to the State through this grievance and appeals review. Therefore, we have not added this review activity to the protocol. We are also not requiring the EQR to review informal communication with providers. Informal communications by their nature do not routinely involve written documentation, and we believe it would be burdensome to require reviewers to monitor verbal exchanges.

Comment: One commenter recommended that the interview questions address the MCO's process and criteria for extensions of the standard 14 days for regular prior authorization decisions.

Response: We disagree with the commenter: timeframes for standard prior authorization decisions are established by the State. The protocol addresses compliance with the standard requirements in the Medicaid managed care final rule. Because extensions to State-established timeframes for standard authorization decisions is not included in the regulations addressing enrollee services, it would be inappropriate to include it in the protocol.

12. Enrollment & Disenrollment

Comment: One commenter believes that the protocol should provide guidance to reviewers concerning when it is appropriate for enrollees to use the MCO's grievance process before the State makes a determination on the enrollee's disenrollment request.

Response: The Medicaid managed care regulation does not specify the circumstances under which it is appropriate for enrollees to use the MCO's/PIHP's grievance process before the State makes a determination on the enrollee's disenrollment request. The protocol is designed to address MCO/PIHP compliance with the regulatory provisions and is not intended as a vehicle for either specifying additional requirements or providing guidance.

Comment: One commenter recommended the protocol include comparisons of MCO disenrollment rates and default or automatic enrollment rates because high rates can signify quality or access problems in the former instance and information deficits in the latter.

Response: While we agree with the commenter that disenrollment rates and default or automatic enrollment rates may be correlated, we do not agree that a comparison of rates alone will suffice. Instead, we have revised the protocol to specify that the document review include the MCO/PIHP disenrollment rates, and that the review of the disenrollment sample determine if a relationship exists between the enrollees requesting disenrollment and enrollees enrolled in the MCO/PIHP automatically or by default.

13. Grievance System

Comment: One commenter suggested that the protocol include review of policies and interview questions to ensure the MCO does not deter enrollees from requesting fair hearings. The commenter recommended further that the reviewer consider the number of grievances and fair hearings versus the population served, and determine whether grievances are held in suspense at certain levels of the review process or enrollees are deterred from filing or pursuing grievance or fair hearing requests. The commenter also suggested the reviewer convene focus groups concerning how the grievance system is working.

Response: We believe the protocol, in the portion addressing review of documents related to enrollee grievances, appeals and State fair hearings, addresses the MCO/PIHP compliance with the regulatory provisions, and in so doing, ensures that the MCO/PIHP does not deter enrollees from requesting fair hearings or pursuing grievance or fair hearing requests. The protocol specifies a review of logs, registries, or other MCO/PIHP documentation of appeals, grievances, and requests for State fair hearings made by Medicaid enrollees. Further, States are required to maintain records of grievances and appeals and review this information as part of the State quality strategy. If grievances are held in suspense, this should become evident to the State through this grievance and appeals review. We believe that focus groups, like provider and consumer interviews, are time and resource intensive. Therefore, we include consideration of other accessory information, such as beneficiary surveys that may offer information on how the grievance system is working but do not require in this protocol that the reviewer convene focus groups.

Comment: One commenter believes that notice of action requirements (for denial, reduction or termination of services) apply to all types of plans and asked that this be clearly stated in the protocol. The commenter further suggested the protocol include interview questions to probe the actions that trigger notices required by due process of the law, and a review of the MCO's notices to determine that the notices comply with the legal requirements for adequate notice of hearing rights, assure enrollees the care they receive will not be affected because a grievance has been filed, are in languages prevalent in the service area, and clearly specify the action the MCO is taking.

Response: The protocol is designed to specifically determine MCO and PIHP compliance with provisions in the Medicaid managed care final rule, regardless of whether or not the provisions apply to other types of managed care plans. We have, therefore, addressed these two entities in assessing compliance with the requirements concerning notice of action. We believe a document review is more effective for this issue than interview questions as an approach to compliance determination. Furthermore, the protocol includes the review of a sample of MCO/PIHP notices to determine the extent to which notices include the legal requirements for adequate notice of hearing rights and specify the action the MCO/PIHP is taking. We agree with the commenter and have expanded this review to determine that notices include assurances that enrollees will not be treated differentially, and are in languages prevalent in the service area. We believe that by reviewing a sample of beneficiaries that have been denied services and the reasons for denials, reviewers will identify those actions that trigger notices required by due process of the law.

Comment: One commenter believes the protocol fails to ascertain the extent to which enrollees have realistic access to the grievance process. The commenter recommended that the protocol include interview questions concerning the process and frequency by which enrollees are informed of the grievance procedures. The commenter also suggested reviewers monitor the timeliness of grievance processing, interview enrollees regarding the free exercise of their rights, and review the MCO's procedures for supplying translation and interpretation services during the grievance process.

Response: As we noted in the prior response, we believe a document review is more effective than interview questions in determining compliance with these provisions. The protocol includes the review of the MCO/PIHP's administrative procedures and policies as well as a sample of MCO/PIHP notices. We agree with the commenter that reviewers should monitor the timeliness of grievance processing and review the MCO's/PIHP's procedures for supplying translation and interpretation services during the grievance process. Therefore, we have specified that in reviewing the sample of notices, the reviewer should determine the timeliness of grievance processing, and have included a review of the MCO's/PIHP's procedures for supplying translation and interpretation services during the grievance process. However, since enrollee interviews are time and resource intensive and beneficiary survey results are specified for consideration as accessory information, we have not included this activity.

Comment: One commenter recommended reviewers interview enrollees to determine how they are informed of the right to request continuation of benefits pending resolution of an appeal or fair hearing, and whether continuing benefits were received when requested. The commenter also suggested that the reviewers compare the MCO's policies with the enrollees' experiences.

Response: As noted previously, enrollee interviews are time and resource intensive and are therefore not a review activity included in the protocol. Instead, reviewers are directed to review the results of beneficiary surveys as accessory information. The protocol also specifies a review of the MCO/PIHP administrative policies and procedures, and the review of a sample of notices, to determine the extent to which enrollees are informed of their right to request continuation of benefits pending resolution of an appeal or fair hearing. The findings from the document reviews can then be compared to the survey results as suggested by the commenter.

Comment: One commenter disagreed with the protocol not permitting the combination of case manager and care coordinator interviews with other interviews. The commenter further recommended the protocol include interview questions for case managers and care coordinators on the enrollees' process for accessing case management services to ensure consistency with MCO policies, the procedures for interfacing with carved-out or other services not covered by the MCO, and the ease of accessing specialist care.

Response: The protocol specifies that the case manager's and care coordinator's interviews may be combined with the Medical Director interview or the Utilization Management interview. This option is consistent with the process used by private accrediting bodies and in the Medicare program reviews. The protocol specifies potential interview questions for case managers and care coordinators to confirm MCO/PIHP compliance with the regulatory requirements pertaining to enrollee rights, service access, and coordination and continuity of care. However, if issues arise during the document review concerning the process for accessing case management services, for interfacing with carved-out or other services not covered by the MCO, or the ease of accessing specialist care, reviewers are directed to explore them during the interviews. We believe this direction affords the reviewers the flexibility necessary to appropriately tailor the review activity to the structure, operations, and circumstances identified for each MCO/PIHP. Further, we do not believe it is possible, given the diversity among States and MCOs/PIHPs and the scope of the review itself, to include in the list of potential interview questions probes to explore every possible problem or issue that might arise.

Comment: One commenter believes that in collecting accessory information it is important to consider non-Medicaid enrollee survey results and compare these to the Medicaid results to ensure all enrollees are receiving the same level of care.

Response: We believe there are numerous analyses of EQR-related activities that can be undertaken. Specifically, the results of compliance monitoring, encounter data, and performance measurements can all be compared, contrasted, analyzed, and correlated. We do not believe the Federal government can or should specify a single set of analyses that will yield the most useful information for all States and MCOs/PIHPs. We believe that States will choose their EQROs on the basis of their demonstrated competence in quality review and analysis, and we defer to the State's decisions about the lines of inquiry EQROs should pursue regarding all EQR-related data, including surveys of Medicaid enrollees and possible comparisons to Medicare enrollees, commercial enrollees, and SCHIP enrollees.

C. Protocols for Calculating or Validating Performance Measures

Comment: One commenter asked that clarification be provided regarding the collection and validation of performance measures. The commenter is concerned that there is no description of essential EQRO activities to ensure that the performance measures being used by the State are scientifically sound, meaningful, valid, and reproducible. The commenter does not believe that the collection methodology outlined in the protocols will ensure valid and reliable measures. The commenter recommended that we take steps to ensure that EQROs use only evidence-based performance measures.

Response: We disagree with the commenter. The protocols outline a methodology to be used in the validation or calculation of performance measures to ensure that valid and reliable measures are calculated or to determine the extent to which valid and reliable measures have been calculated by the MCO/PIHP. The protocols were designed to be consistent with approaches used by NCQA and Medicare QIOs but to also describe how to validate or calculate measures such as those found in HEDIS as well as those developed by States or other groups or organizations. We advocate the calculation of measures that have been tested and accepted in the private and public sectors but provide States with the flexibility to develop measures or use measures developed by others that meet their program needs.

In addition to specifying essential activities to be conducted as part of performance measure validation or calculation, we have provided an Appendix to this protocol that provides guidance on how to assess an MCO's or PIHP's underlying information system (IS) to ensure that valid and reliable data are used in the calculation of the performance measures. The IS assessment may be conducted as part of this protocol by the EQRO validating or calculating the performance measures, or the EQRO may review an assessment conducted by another party.

Comment: One commenter believes that States have already invested substantial resources in establishing systems to carry out performance measurement activities and that it is not clear how these established systems can be adapted easily to meet the requirements of the protocols.

Response: Because the essential components of the protocols are accepted practice in both the public and private sector, we expect that States will not have to significantly adapt their approaches to performance measurement. The performance measures protocols are to be used for validating measures calculated by the MCO or PIHP as required by the Medicaid managed care final rule or for calculating additional measures as directed by the State. State approaches to performance measurement might vary but we expect States to require the essential components of the protocol for performance measurement activities—review of MCO/PIHP data management processes, evaluation of compliance with specifications for performance measures, and verification of performance measurement.

Comment: One commenter believes this protocol is outdated and suggested we reference current industry tools. Another commenter argued that the performance measure validation process is heavily biased toward proprietary systems entities developed in the business of accreditation. The commenter believes this bias limits flexibility in the process and promotes a narrow view of performance measurement and jeopardizes State's ability to be innovative in performance measurement.

Response: One reason we did not include the protocols in a regulation was because we recognize that the protocols will need to be updated as the state-of-the-art in quality assessment and improvement changes. However, we believe that the activities listed in the protocol are still those in current use in the industry. Further, to be in compliance with the EQR rule, States only need to ensure that our protocols or those consistent with ours are used.

In addition, we do not agree that the protocol is biased toward proprietary systems. We used three sources to develop the performance measures protocols (that is, NCQA's HEDIS validation protocol, IPRO documents, and documents from the MEDSTAT group). We identified activities common to these tools and incorporated those activities to ensure valid and reliable methods are used when calculating or validating performance measures. Only one of these tools was developed by an organization that is in the business of accreditation, and we do not agree that the performance measures protocol limits State flexibility in the performance measures development process. We provide States with the flexibility to use established measures or to develop their own measures. We recommend, however, when States choose to develop or use measures not widely used in the private and public sector, that these measures should be evidenced-based and tested.

Comment: Several commenters believe the process described for validating performance measures is bureaucratic and administratively burdensome. The commenters state that they do not understand the value of interviewing MCO staff and believe annual onsite review is not necessary and is burdensome.

Response: The process in the protocols for validating performance measures is consistent with the process used in the private sector and the Medicare program. We drew from established tools in the development of these protocols. The protocol includes interviewing MCO and PIHP staff in addition to reviewing MCO/PIHP documentation of how performance measures are produced. The purpose of interviewing staff is not to obtain information that can otherwise be obtained from documentation. It is to supplement and confirm information as needed. In the protocol, interviews of MCO/PIHP personnel are identified as an effective mechanism to understanding an MCO's/PIHP's IS and its application to performance measurement. While much information can be obtained by reviewing an MCOs/PIHPs internal documents describing its IS, we believe that interviews with MCO/PIHP staff can be a helpful adjunct to the review of IS documents in understanding the issues the MCO/PIHP has with respect to ISs and how it affects the MCO's/PIHP's production of performance measures.

Comment: One commenter argued that some States calculate and report MCO-level performance measures and therefore, much of what is contained in the calculating performance measures protocol is not applicable to MCOs, but is applicable to the State.

Response: We recognize that States may have MCOs and PIHPs submit encounter data to them instead of performance measures and, therefore, the State may be the entity calculating the performance measure. We have allowed for this in the quality assessment and performance improvement program requirements specified in § 438.240 of the Medicaid managed care final rule. However, regardless of who calculates the performance measures, MCO and PIHP-level performance measures must be calculated as required by the Medicaid managed care final rule and, if calculated by the MCO/PIHP, must be validated to provide information for the EQR function. We have added clarifying language under § 438.358(b)(2) to recognize that States may be calculating the MCO/PIHP performance measures and in this circumstance the State would provide the information obtained from this activity to the EQRO for the EQR function.

Comment: One commenter suggested combining the validating performance measures protocol and the calculating performance measures protocol to reduce the length and complexity of the two protocols.

Response: We purposefully provided separate protocols for each EQR-related activity. Even though some of the protocols are variations on a theme (for example, validating performance measures and calculating measures) we wanted to provide stand-alone documents for each activity. In addition, though the protocols are variations on a theme, the activities do differ somewhat and we believe the clearest way to present the information is in separate documents.

Comment: One commenter argued that the 30 sample medical record review recommended in the protocol for performance measures not calculated with administrative data only will add tremendous cost, is needlessly intrusive, and is very time consuming.

Response: This aspect of the protocol illustrates what we mean when we say that States must use protocols that are consistent with (but not identical to) our protocols. In this protocol, onsite Activity 4 is the “Assessment of Processes to Produce Numerators.” To be consistent with our protocol, the EQRO must perform this activity (that is, assess the MCOs' or PIHPs' processes to produce the performance measure numerator). In our description of Activity 4, we describe how this activity is to be conducted and state that this activity should include a review of a sample of the medical records used to determine the numerator. Thirty medical records is the number that was included in the private sector protocols we reviewed. However, EQROs may use another sample size and still be consistent with our protocol. Our protocol endorses the policies found in private sector protocols, that require a sufficient number of medical records be reviewed to validate a reported numerator for a given performance measure. As stated previously, however, activities used to provide information for the EQR must be conducted “consistent with” our protocols. “Consistent with” means that the protocols used contain all of the activities and steps included in our protocols. How EQROs and States implement the activities and steps is left to their discretion.

Comment: One commenter suggested we add lab data as a data source to calculating performance measures numerators (page 8, item 4).

Response: We agree with the commenter and have added laboratory data as a possible data source for calculating performance measures.

Comment: One commenter suggested some editorial changes.

Response: We have made editorial changes that were recommended where we thought appropriate and helpful.

Comment: One commenter suggested on page 15 we add “place of service” to the list of claims and encounter data elements to be assessed when assessing the integrity of the MCO's/PIHP's IS.

Response: We agree with the commenter and have added place of service to the list of claims and encounter data elements that may be used to conduct performance measurement.

D. Protocols for Conducting or Validating Performance Improvement Projects and Conducting Focused Studies

Comment: One commenter believes all the activities in this protocol are reasonable.

Response: We agree and retain the activities in the protocol.

Comment: One commenter asked for clarification of why the protocol for conducting performance improvement projects was developed. The commenter questioned the value of this protocol since the EQRO is not affiliated with any MCO and has no way to implement performance improvement initiatives affecting the actual delivery of care. The commenter recommended eliminating this protocol.

Response: This protocol was developed to provide EQROs and States guidance on the activities required when conducting performance projects as an optional EQR-related activity that qualifies for 75 percent FFP. A State may itself, through another State contractor, or through the EQRO, have additional performance improvement projects conducted other than those required to be conducted by the MCO/PIHP under § 438.240(b)(1) of the Medicaid managed care final rule and § 438.358(b)(1) of this rule. As long as the project is conducted consistent with the protocol, the information can be provided to the EQRO and be included as part of the EQR function. If the State itself or other State contractor conducts the activity, the State would not qualify for the 75 percent enhance match. If the EQRO conducts the performance improvement project, the State could claim the enhanced match. We developed separate protocols for the conduct of performance improvement projects and the validation of performance improvement projects to have stand-alone documents.

Comment: One commenter recommended that the focused study protocol be combined with the validating performance improvement projects protocol. The resulting protocol should be an optional protocol to be used at the State's discretion. One commenter recommended that the validating performance improvement projects and conducting performance improvement projects protocols be combined.

Response: We have developed separate protocols for validating and conducting performance improvement projects and for conducting a focused study of health care quality in order to provide stand-alone documents for each of the EQR-related activities. The focused study protocol and the conducting performance improvement projects protocol are to be used at the State's discretion if it decides to include information from these optional EQR-related activities as part of the EQR. In contrast, validating performance improvement projects conducted by MCOs/PIHPs is a mandatory activity. Although these protocols have much in common, there are some differences and we believe it is more helpful to the readers and users of the protocols to present these similar, but different activities in separate documents.

Comment: One commenter argued that the focused study protocol is biased towards proprietary measurement systems, that we advocate the use of indicators that are generally used in the public health community such as those developed by NCQA and the Foundation for Accountability (FACCT). The commenter recommended that the protocol be neutral in tone and approach the topic of performance measure selection from the perspective of State preferences and existing or evolving State-specified systems.

Response: We agree with the commenter that we advocate the use of performance indicators that are generally used in the public health and managed care industry. This is because these measures have been tested for validity and reliability and are widely accepted in the public and private sectors. However, we also, in the performance measures (both conducting and validating) and focused study protocols state that other indicators may be used. We recommend that these indicators be developed on the basis of current clinical practice guidelines or clinical literature derived from health services research or findings of expert or consensus panels.

Comment: One commenter suggested we add appointment availability studies, network assessment studies, open-closed panel reports, member and provider satisfaction survey data, and provider language reports as potential sources of information for selecting study topic for performance improvement projects or focused studies of health care quality.

Response: We agree with the commenter and have revised the potential sources of supporting information section, under Activity “Selecting the Study Topic,” in the performance improvement projects (conducting and validating) and focused studies protocols to include the following: data on appointments and provider networks such as access, open and closed panels, and provider language spoken. Data from surveys was already included in this section in each protocol.

Comment: One commenter suggested we add a discussion of service needs for special needs populations to the list of methods for selecting the study topic.

Response: We recommend in this section that topics should reflect high-volume or high-risk conditions of populations served, including populations with special health care needs such as children in foster care, adults with disabilities, and the homeless. We further state that although these populations may be small, their special health care needs place them at high risk. We believe these provisions address the commenter's concerns and that no change is needed.

Comment: One commenter believes that our rationale for reliable data collection only addresses clinical data collection. The commenter suggested we add a section for service studies such as appointment availability and that methods to implement this include review of appointment books, and “secret shopper” techniques when someone calls to make an appointment. These kinds of indicators require scripts and very clear definitions of items such as acute care, emergent care, and routine care.

Response: We agree with the commenter that we did not include a discussion on data collection issues when using nonclinical data. We have added a paragraph in the performance improvement projects (both conducting and validating) and focused studies protocols to address this issue.

E. Protocol for Validating Encounter Data

Comment: One commenter stated that the protocol does not allow for the fact that encounter data may be used for risk adjusted payment and/or other utilization data analysis purposes.

Response: Accurate and reliable encounter data is crucial to performing any analysis of utilization data, and in particular to the development of capitated payments which are based on utilization data. This protocol specifies processes for assessing the completeness and accuracy of the encounter data MCOs and PIHPs submit to the State. We believe this protocol for validation of encounter data accommodates the multiple purposes for which encounter data are used.

Comment: One commenter stated that this protocol is long, detailed, needlessly prescriptive and biased toward the MEDSTAT and HEDIS models. The commenter also stated that since States generally have encounter data validation processes in place, this protocol will be redundant and should therefore be dropped, reformatted as technical assistance or combined with other protocols to reduce the length and complexity of the protocols.

Response: In developing this protocol (as with all the protocols) we instructed our contractor to draw from existing protocols that have been tested and used in the public and private sectors, and that are consistent with current industry practice. The elements contained in the MEDSTAT and HEDIS tools are consistent with other validation processes reviewed, and contain generic activities and steps that include the essential components of a methodologically sound review of encounter data. By requiring protocols that are “consistent with,” rather than “identical,” we believe that we have allowed for State flexibility while ensuring a minimum standard of quality. Since the validation of encounter data is an optional EQR-related activity, States have the option to conduct this activity or not. Consequently, we do not believe this protocol is redundant, needlessly prescriptive, or biased.

Comment: One commenter believes this protocol should address State data issues and improvements that may impede the ability of MCOs and PHPs to improve their data quality. These issues include the inability of the State to receive MCO and PHP data, unclear data specifications to MCOs and PHPs, and State policies and procedures.

Response: Section 4705(a)(2) of the BBA specifies that EQR be a review of MCOs. Therefore, these protocols focus on MCOs and PIHPs, not on the State. State Medicaid agencies have available to them a variety of approaches that use contractors to strengthen their Medicaid Management Information System (MMIS). Additionally, we have funding opportunities that assist States with improvements to their MMIS. We, therefore, are not modifying this protocol to address State Medicaid agency data issues.

Comment: One commenter asked for clarification about the purpose of the chart on page 11, including how the categories were decided upon, and who will calculate the elements.

Response: The “Acceptable Error Rates Specifications and Identified Areas of Concern Form,” is meant to serve as an example of a tool that an EQRO can use when assessing rates of accuracy and completeness for each data field. This tool can be used at the State's or EQRO's discretion. It may be adapted to meet individual State standards, or a State or EQRO may decide to develop a similar tool. Its purpose is to illustrate that States need to specify what error rate they will determine to be acceptable for the various types of encounter data to be submitted to them. The categories of “encounter type” were determined by the subcontractor that developed this protocol based on its extensive experience as a contractor to us and State Medicaid agencies on the production, assessment, and improvement of encounter data. The acceptable error rates should be specified by the State.

Comment: One commenter recommended against an analysis of mandatory fields (page 16) because these items are generally mandatory and an MCO's submission would not be accepted if any of the fields were not complete.

Response: We do not agree that an MCO's/PIHP's submission would not be accepted if any of the fields were not complete. State Medicaid agencies determine the acceptable levels of missing, surplus, or erroneous data. States also determine the standards for encounter data accuracy and completeness, to which encounter data submitted by MCOs and PIHPs will be compared. This protocol recommends that the encounter data validation process analyze and interpret the data in submitted fields to determine if the information is of the type that was requested by the State Medicaid agency, and if the values are valid and reasonable.

Comment: One commenter believes that because an MCO does not participate in or control the process of documenting the service in the medical record and subsequent billing that is based upon the medical record, there is no possibility for payor misbehavior.

Response: This protocol specifies processes for assessing the completeness and accuracy of encounter data MCOs/PIHPs submit. The protocol references reviews of medical records as an activity that is conducted to verify the accuracy of the automated data submitted, using the medical record as the point of reference. Payor misbehavior is not the issue. The issue addressed by this protocol is the accuracy of the information a provider submits, through the MCO/PIHP to the State, and the extent to which the MCO/PIHP has procedures in place to promote the accuracy and completeness of the data submitted by their providers.

Comment: One commenter believes the acceptable error rates form (page 5) is not information that can be assessed during an onsite visit.

Response: The Acceptable Error Rate form is a tool that can be used by the State or EQRO to document whether the MCO/PIHP has exceeded the acceptable error rate for each encounter type, and whether any concerns have been raised that trigger the need for further investigation. The protocol does not specify at what location (State Medicaid agency offices, MCO or PIHP offices, or EQRO offices) compliance with acceptable error rates is to be determined. The location where this form is to be constructed or used is to be determined by the State.

Comment: One commenter suggested that the protocol address rejected data.

Response: Activity 3, “Analyze Electronic Encounter Data for Completeness and Accuracy,” represents the core of the process the EQRO will use to test the validity of the encounter data. Activity 3 is designed to yield information about the general magnitude of missing encounter data, and should identify problems in the MCO's/PIHP's process for compiling and submitting encounter data. Rejected data should be included in the evidence of and reasons for an MCO's/PIHP's inability to submit encounter data. Additionally, Appendix Z (Information Systems Capabilities Assessment) asks what happens to the encounter if one or more required fields are missing, incomplete, or invalid.

Comment: One commenter suggested that the protocol address additional significant issues in performing data accuracy assessments. The commenter further recommended that it be clear before proceeding if the data are pre- or post-edits and whether they are from the MCO, the State, or from the State's data warehouse.

Response: We do not understand what the commenter is referring to when suggesting that the protocol address additional significant issues in performing accuracy assessments. In response to the second comment, the data that the protocol addresses is MCO/PIHP level data, and where the data resides is unique to each State. The protocol addresses encounter data submitted by the MCO/PIHP to the State. Therefore, the data would include any edits made by the MCO/PIHP. The State will need to identify to the EQRO the extent to which it has performed any edits of the data submitted by the MCO/PIHP.

Comment: One commenter suggested that the protocol address benchmark data that can be used to help determine data completeness.

Response: The use of benchmarks is discussed in a number of the Steps in Activities 2 and 3. The protocol does not specify exact benchmarks that are to be used because benchmarks should be tailored to each State's status with respect to the accuracy and completeness of its encounter data. The protocol instead discusses how the EQRO should use benchmarks for testing the quality of data. Additionally, the protocol indicates the source for some benchmarks, and in some cases, provides instructions for EQROs to develop certain benchmarks.

Comment: One commenter suggested that the protocol address incorporation of vendor data in reporting to the State.

Response: We agree that vendor data should be included when reporting to the State. That is why we reference the importance of vendor data when assessing the MCO's/PIHP's capability to produce accurate and complete encounter data in Activity 2. Activity 2 directs the EQRO to conduct an IS assessment that is consistent with the process described in Appendix Z. Appendix Z includes as elements that impact the accuracy and completeness of encounter data, the MCO's/PIHP's data submission policies, and the contract requirements for vendors and contractors.

F. Information Systems Capabilities Assessment (Appendix Z)

Comment: One commenter believes the level of detail required in the information systems capabilities assessment (ISCA) tool is excessive. The commenter does not believe that the reviewer should have the option of asking for the source code for a variety of computer and report programs. Moreover, the commenter stated that MCOs do not necessarily have the source code because that information may be proprietary and may be the property of a vendor.

Response: We do not agree that the ISCA tool requires an excessive level of detail. A number of public and private sector protocols and tools were examined to promote consistency between this assessment and similar public and private sector activities. We also disagree with the comment that the reviewer does not need the source codes used to perform various calculations, and because these codes are proprietary the MCO/PIHP would not have access to this documentation. The source codes referred to in the protocol are codes used in the programs written by MCO/PIHP staff or by their contractors to calculate continuous enrollment or other calculations using MCO/PIHP administrative data. Consequently, whenever the accuracy of calculations performed by the MCO/PIHP impact on other aspects of the quality measurement; for example, performance measures, the EQRO will require source codes to validate the accuracy of those calculations. These source codes should, therefore, be available to the MCO/PIHP.

Comment: One commenter believes the onsite activities under this Appendix probe policies and procedures not subject to regulation and that they are not relevant to the State MCO contract.

Response: We disagree with the premise that the policies and procedures related to the MCO/PHP ISCA are not subject to regulation. This Appendix relates to three different regulatory provisions. Under § 438.242 of the Medicaid managed care final rule, the State must ensure, through its contracts, that each MCO/PIHP maintains an IS that accurately and completely collects, analyzes, integrates, and reports data on utilization, enrollment and disenrollment. Additionally, § 438.240 stipulates that the State must require MCOs/PIHPs to have an ongoing quality assessment and improvement program for which accurate and complete data is an essential element. Further, in § 438.350 of this final rule, each State is required to provide its EQRO information obtained through methods consistent with these protocols. In our contractor's review of private sector industry and Medicare practices, it was determined that an assessment of an MCO's/PIHP's IS is an essential component of validation of encounter data and performance measurement.

Comment: One commenter believes that this Appendix is outdated and suggested the encounter data protocol should reference current industry available tools.

Response: When we started developing the protocols we used the most recent version of the public and private sector tools referenced. These private and public sector tools have since been updated. However, because we developed the protocols as generic activities and steps to be used in the conduct of the EQR-related activities, we do not agree that the protocols are outdated. Furthermore, in this final rule we allow for use of other protocols, as long as they are consistent (that is, contain the activities and steps identified in these protocols) with those we have developed.

Comment: One commenter believes that States may routinely assess MCO IS capabilities and in these cases this protocol is of limited applicability.

Response: To avoid duplication, in all the protocols calling for an ISCA, we state that the EQRO may use information about the MCO/PIHP ISCA obtained from an ISCA conducted by another party as part of another review such as the validation of performance measures, validation of encounter data, or a review for compliance with standards. If the ISCA was performed by another party as part of another review, the State or EQRO should obtain a copy of the assessment, review it to determine if the findings are current, consistent with this Appendix, and where appropriate, seek more recent or additional information. If a recent assessment has not been conducted, an ISCA that is consistent with this Appendix should be conducted.

G. Protocols for Administering or Validating Surveys

Comment: One commenter argued that the protocol for administering a survey is very prescriptive and the value of such a detailed protocol is questionable particularly when States choose to follow the recommended CAHPS survey method. The commenter asked us to clarify how much latitude there was to follow the CAHPS methodology.

Response: The administration of validation of consumer or provider surveys of quality of care are optional EQR-related activities. If a State elects to have its EQR perform these activities and to qualify for the 75 percent enhanced match, our protocol or a protocol consistent with ours must be used. Our protocol includes generally accepted practices of survey design and implementation. We relied upon, but condensed, generally accepted principles of survey design and administration discussed in textbooks and other health services publications. Although many States use CAHPS surveys (and the CAHPS survey methodology would meet the requirements of this protocol) it was necessary to put forth this protocol to cover those instances when States desired to use a survey other than a CAHPS survey.

Comment: One commenter asked us to clarify the distinctions between the two survey protocols.

Response: The first protocol applies to the situation in which the State or its agent administers a survey, that is, designs and/or conducts a survey. Administration of a survey may include the design and implementation of a new survey or the modification of an existing survey and its implementation.

The second protocol applies to the situation in which the State or its agent validates the use of a survey administered or conducted by another party. The process of validation is necessary to ensure that the survey results are both reliable and valid. In this protocol, survey validation is limited to a review of the survey procedures. The validation process does not include collecting survey data anew from respondents to verify their responses.

Comment: One commenter believes that beta testing all surveys and the additional questions to members and providers would be time consuming and cost prohibitive.

Response: The protocols do not suggest beta testing of all surveys. Instead, they acknowledge the commitment of time and resources and the demands on survey respondents that make such an activity infeasible. The protocol suggests that survey validation be limited to a review of survey procedures.

H. Other Appendices (Attachments to Final Protocols)

Comment: One commenter recommended that we explain the obligations of the State or the EQRO with regard to the documents included in the appendices (for example, what is the role of the documents and how the documents are to be used).

Response: With the exception of Appendix Z, ISCA for MCOs and PIHPs, the appendices (Attachments to the final protocols) provide additional guidance to States and EQROs on how to implement the EQR-related activities. The information contained in the appendices (Attachments to the final protocols) are to be used at the discretion of the State or EQRO based on the particular circumstances of the activity being conducted and other means of obtaining needed information.

I. Section 438.360 (Nonduplication of Mandatory Activities)

Comment: One commenter believes the estimates of the time necessary to collect the information under this provision are too low. In addition, the commenter believes that this function needs to be performed by both professional staff and clerical staff and that a blend of the hourly costs should be used to determine the estimated costs.

Response: As we stated earlier, because we received several comments indicating that this estimate is low but commenters did not provide us with what they believe the correct estimate to be, we have increased the burden hours by 100 percent to 8 hours. We have taken the commenters recommendation and blended the hourly costs to reflect that both professional and clerical staff will partake in this effort.

J. Section 438.362 (Exemption From EQR)

No comments were received on this section.

K. Section 438.364 (EQR Results)

No comments were received on this section.

IV. Provisions of the Final Regulation

For the most part, this final rule adopts the provisions of the December 1, 1999 proposed rule. In response to public comments, we have made clarifying wording changes. Those provisions of this final rule that differ from the provisions of the December 1, 1999 proposed rule follow.

Section 438.310—Basis, Scope, and Applicability

We have revised this section to reference the applicability of this rule to PIHPs. We have added the reference to PIHPs throughout the rule as appropriate.

Section 438.320—Definitions

We have revised this section by adding clarifying language to the definitions for the terms “EQR” and “EQRO” and adding a definition for the term “financial relationship.” The definition of EQR has been revised to clarify that this rule applies to the care provided to Medicaid beneficiaries that receive health care services furnished by MCO and PIHP subcontractors as well as MCOs and PIHPs. This definition has also been revised to clarify that EQR-related activities are not considered part of the EQR function. We have revised the definition of EQRO to mean an organization that conducts the EQR function as well as EQR-related activities. EQR-related activities had not previously been included in the EQRO definition. As a result of this clarifying language, how we use the terms EQR, EQR-related activities, and EQRO needed to be changed in several sections of this rule.

Section 438.350—State Responsibilities

We have revised this section to add clarifying language that the information provided to the EQRO is consistent with the information we require as part the EQR results; for each EQR-related activity that provides information for the EQR, the EQRO must have the objectives of the activity, the methods of data collection and analysis, a description of the data obtained, and the conclusions drawn.

Section 438.352—External Quality Review Protocols

We have revised this section to add clarifying language at paragraph (c) of this section to explain what we meant by each protocol must specify the “detailed procedures” to be followed in collecting the data to promote its accuracy, validity, and reliability. We have changed the wording of “detailed procedures” to “activities and steps” to be consistent with how the EQR protocols have been designed.

Section 438.354—Qualifications of External Quality Review Organizations

We have revised this section to add at paragraph (b)(1) that the EQRO must have “demonstrated experience” as well as knowledge of the Medicaid recipients, policies, data systems, and processes; managed care delivery systems, organizations, and financing; quality assessment and improvement methods, and research design and methodology.

We have revised paragraph (c) of this section to require that all EQROs, as opposed to only State entities that qualify as EQROs, may not deliver any health care services to Medicaid beneficiaries, or conduct on the State's behalf ongoing Medicaid managed care program operations related to the oversight of MCO or PIHP quality of services. This later provision has been revised to apply only to Medicaid managed care operations as opposed to all Medicaid program operations. This provides States the opportunity to contract with a broader group of entities than was provided for in the December 1, 1999 proposed rule.

We have also revised paragraph (c) of this section to add clarifying language to explain how “control” is defined in 48 CFR 19.101. In addition, we have added a provision that prohibits an entity from qualifying as an EQRO if it has a financial relationship with an MCO or PIHP that it will review as an EQRO.

Section 438.356—State Contract Options

We have revised paragraph (a) of this section to clarify that States may only contract with one entity for EQR alone or EQR and other EQR-related activities, but may contract with multiple entities to conduct additional EQR-related activities.

Section 438.358—Activities Related to External Review

We have revised this section by adding cross-references to the Medicaid managed care final rule. We have made these cross-references throughout this rule where appropriate. We had not included these cross-references in the December 1, 1999 proposed rule as the Medicaid managed care final rule had not yet been published.

We have added a general rule under paragraph (a) to clarify that the mandatory and optional EQR related activities can be conducted by the State, the State's agent that is not an MCO or PIHP, or an EQRO.

We have revised paragraph (b)(1) to clarify that information from the validation of performance improvement projects that are underway, as opposed to those being performed, must be obtained from the MCO or PIHP. We have revised paragraph (b)(2) to clarify that information on performance measures can be obtained from either those calculated by the MCO/PIHP and validated by the State or its agent, or those calculated by the State on behalf of the MCO/PIHP. We have also revised (b)(3) by eliminating the reference to specific State standards. These are now referenced in the aggregate by our cross-reference to the Medicaid managed care final rule provision. We have also revised paragraph (c) to clarify that information from optional activities must be from information derived within the preceding 12 months.

Section 438.360—Nonduplication of Mandatory Activities

We have revised this section by removing the word “exempt.” Using this word caused confusion with the “exemption of EQR requirements” under § 438.362. In its place, we provide language that explains that the nonduplication provisions allow States to use information from either a Medicare or accreditation review for certain standards and activities in place of a Medicaid review.

We have also revised this section to allow States to apply this provision to MCOs and PIHPs that provide health care services to commercial consumers of health care as well as Medicare beneficiaries. We have further revised this section to clarify that national accrediting organizations are those organizations that have been approved and recognized for M+C deeming. We have made this clarification throughout the rule as appropriate.

We have restructured this section by revising paragraph (b) so it applies to both M+C and MCOs and PIHPs that provide services to commercial consumers and have revised paragraph (c) to address additional provisions for those MCOs and PIHPs providing services to dually eligible beneficiaries only. Under paragraph (b) and (c), we have added a provision that requires the State in its quality strategy to identify those standards and activities for which it will substitute the Medicare or accreditation review for the Medicaid review. In addition, we require the State to explain the rationale for why the State considers the standards or activities duplicative.

Section 438.362—Exemption From External Quality Review

We have revised paragraph (a)(2) to clarify that the Medicare and Medicaid contract must overlap geographically within the State when it exempts the MCO or PIHP from EQR. The December 1, 1999 proposed rule did not require that the overlap be within the State.

We have revised (b)(1) to clarify that information from Medicare reviews is to be obtained by the State from the MCO or PIHP. The language in the December 1, 1999 proposed rule could have been misinterpreted to mean that the State had to obtain the information from CMS or its agent. We have also revised paragraph (b)(2) to clarify that the MCO or PIHP must provide the State a copy of the accreditation review findings as opposed to ensuring the State receives a copy.

Section 438.364—External Quality Review Results

We have revised paragraph (a)(1) to clarify that in the detailed report, conclusions are drawn as to the timeliness of and access to care as well as the quality of care. We have revised paragraph (a)(1)(iii) to clarify that the detailed report should include a “description” of the data obtained for each EQR-related activity as opposed to the data obtained. We did not intend for the raw data to be provided as part of the EQR results. We have also revised paragraph (a)(2) to require an assessment of the MCO's and PIHP's strengths and weaknesses be addressed as opposed to a “detailed” assessment of the MCO's and PIHP's strengths and weaknesses.

We have revised paragraph (b) to require that the EQR results, upon request, be made available in alternative formats for persons with sensory impairments and that the EQR results be made available through electronic as well as printed copies.

Section 438.370—Federal Financial Participation

We have revised (a) to clarify that 75 percent FFP is also available for the production of the EQR results.

V. Collection of Information Requirements

Under the Paperwork Reduction Act of 1995 (PRA), we are required to provide a 30-day notice in the Federal Register and solicit public comment before a collection of information requirement is submitted to the Office of Management and Budget (OMB) for review and approval. In order to fairly evaluate whether an information collection should be approved by OMB, section 3506(c)(2)(A) of the PRA requires that we solicit comment on the following issues:

• The need for the information collection and its usefulness in carrying out the proper functions of our agency.
• The accuracy of our estimate of the information collection burden.
• The quality, utility, and clarity of the information to be collected.
• Recommendations to minimize the information collection burden on the affected public, including automated collection techniques.

Therefore, we are soliciting public comment on each of these issues for §§ 438.352, 438.360, 438.362 and 438.364 of this document that contain information collection requirements.

We published a notice in the Federal Register on November 23, 2001, to give the public a 60-day period in which to comment. The basic purpose was to afford the public an opportunity to comment on the protocols. We have addressed the comments received in response to this Federal Register notice in section III. above.

For purposes of this requirement, we incorporated Medicaid managed care data from the 2001 Medicaid enrollment report. As of June 2001, there were 329 MCOs (this includes 5 HIOs that must adhere to the EQR requirements of this regulation), and 129 mental health and substance abuse PIHPs.

§ 438.358 (Activities related to EQR)—For each MCO and PIHP, the EQR must use information from the following activities:

(1) Validation of performance improvement projects required by the State to comply with requirements set forth in § 438.240(b)(1) and that were under way during the preceding 12 months.

(2) Validation of MCO or PIHP performance measures reported (as required by the State) or MCO or PIHP performance measure calculated by the State during the preceding 12 months to comply with requirements set forth in § 438.240(b)(2).

(3) A review, conducted within the previous 3-year period, to determine the MCO's or PIHP's compliance with standards (except with respect to standards under §§ 438.240(b)(1) and (2), for the conduct of performance improvement projects and calculation of performance measures, respectively) established by the State to comply with the requirements of § 438.204(g).

In addition, if a State, at its option, wishes to provide additional information to its EQRO, and to have CMS provide 75 percent FFP in the costs of producing this information, then the additional information must be produced through activities identified as optional activities in this final rule and also must be produced in a manner consistent with (as opposed to identical to) the protocols for these six optional activities. These six optional activities are (1) validation of client level data such as claims and encounters, (2) administration or validation of a survey, (3) calculation of performance measures, (4) conduct of performance improvement projects, and (5) conduct of focused studies of quality of care.

The burden associated with this requirement is the time and effort for a State, EQRO, or other State contractor, to conduct and document the findings of the three mandatory activities—the validation of performance improvement projects conducted by the MCO/PIHP, the validation of performance measures calculated by the MCO/PIHP, and a review of MCO/PIHP compliance with structural and operational standards. Each of these activities will need to be conducted on the 329 MCOs and 129 PIHPs that we estimate are currently providing Medicaid services. The types of services provided by these managed care entities and the number of performance improvement projects conducted and performance measures calculated will vary.

We interviewed four EQROs who in 2000 reviewed MCOs/PIHPs in 16 mandatory or voluntary managed care programs in eight States. Based on the information provided by the four EQROs, we confirmed that the hours and costs to conduct these activities vary. The information provided includes: (1) It takes 25 to 138 hours at a cost of $2,000 to $10,000 to validate a performance improvement project conducted by an MCO/PIHP; (2) it takes 12 to 202 hours at a cost of $1,200 to $7,000 to validate a performance measure calculated by an MCO/PIHP; and it takes 200 to 800 hours at a cost of $11,000 to $49,000 to review for MCO/PIHP compliance with structural and operational standards. Based on the submitted information, it takes an average of 65, 53, and 361 hours, respectively, to conduct the above mandatory EQR activities. Therefore, the average total burden associated with this requirement is 479 hours x 458 entities (329 MCOs + 129 PIHPs). Assuming wages of $63 per hour for professionals to comply with the requirement, the cost is $13,821,066.

For the optional EQR activities—validation of client level data (such as claims and encounters), administration or validation of consumer or provider surveys, calculation of performance measures, conduct of performance improvement projects, and conduct of focused studies—we have no data to estimate the hours associated with how long it will take to conduct these activities. We, therefore, estimate that it will take 350 hours to validate client level data and 50 hours to validate consumer or provider surveys. We estimate it will take three times as long to calculate performance measures as it takes on average to validate (159 hours) and three times as long to conduct performance improvement projects and focused studies as it takes on average to validate performance improvement projects (195 hours). We also estimate that it will take three times as long to administer a consumer or provider survey than it takes to validate a survey (150 hours).

Based on 2001 State reported data, we know that of the 42 States that had capitated programs (MCOs or PIHPs) in 2001, 29 (69 percent) had their EQROs validate MCO/PIHP encounter data, 18 (43 percent) had their EQRO administer or validate consumer or provider surveys, 12 (29 percent) had their EQRO calculate performance measures, 16 (38 percent) had their EQRO conduct performance improvement projects, and 32 (76 percent) had their EQRO conduct focused studies. Using the aforementioned percentages and applying them to the number of MCOs and PIHPs, we estimate that States will contract with their EQROs to validate the encounter data of 316 MCOs/PIHPs, administer or validate consumer or provider surveys of 197 MCOs/PIHPs, calculate performance measures of 133 MCOs/PIHPs, conduct performance improvement projects of 174 MCOs/PIHPs, and conduct focused studies of 348 MCOs/PIHPs.

We, therefore, estimate the average total burden associated with conducting each optional EQR activity as follows:

• Validating client level data 350 hours × 316 MCOs/PIHPs = 110,600 hours.
• Validating consumer or provider surveys 50 hours × 98 MCOs/PIHPs (1/2 of 197 MCO/PIHPs that administered or validated surveys) = 4,900 hours.
• Administering consumer or provider surveys 150 hours x 99 MCOs/PIHPs (1/2 of 197 MCO/PIHPs that administered or validated surveys) = 14,850 hours.
• Calculating performance measures 159 hours × 133 MCOs/PIHPs = 21,147 hours.
• Conducting performance improvement projects 195 hours × 174 MCOs/PIHPs = 33,930 hours.
• Conducting focused studies 159 hours × 348 = 55,332 hours.

Assuming a wage of $63 per hour for professionals to comply with the requirement, the cost of conducting the optional EQR activities is (240,759 hours × $63) $15,167,817. We solicit comments specifically on this issue because we had no data on which to base the estimated hours for the conduct of each of the optional EQR activities.

The burden estimate associated with this requirement also includes the time and effort for an MCO/PIHP to prepare the information necessary for the EQRO or other State contractor to conduct the three mandatory activities—the validation of performance improvement projects conducted by the MCO/PIHP, the validation of performance measures calculated by the MCO/PIHP, and a review of MCO/PIHP compliance with structural and operational standards. We estimate that it will take each MCO and PIHP 160 hours to prepare this documentation. We believe one-half of the time preparing the information will be done by professional staff at $63 per hour and the other one-half of the time preparing the information will be done using clerical staff at $12 per hour. Therefore, to comply with the requirement, the cost of compiling the necessary information is (458 MCOs/PIHPs × (80 hours × $63 + 80 hours × $12) $2,748,000.

§ 438.360 (Nonduplication of mandatory activities)—In order to avoid duplication, the State agency may allow the MCO/PIHP to substitute information from a Medicare or accreditation review for the Medicaid review if specified conditions are met. To demonstrate compliance with these requirements an MCO/PIHP must provide to the State agency reports, findings, and other results of the Medicare or private accreditation review. The burden associated with these requirements is the time and effort for an MCO/PIHP to disclose the reports, findings, and other results of the Medicare or private accreditation review to the State agency. Of the 329 MCOs and 129 PIHPs providing Medicaid services, approximately 122 are Medicaid-only MCOs. We believe that there is the potential for States to allow the remaining 336 MCOs/PIHPs to take advantage of the nonduplication provision and that these MCOs/PIHPs will be required to disclose the necessary information to each State agency. We estimate that it will take each MCO 8 hours to disclose the necessary documentation to the State, 4 hours of professional time and 4 hours of clerical time. Therefore, the total burden associated with this requirement is 336 MCOs/PIHPs × 8 hours = 2688 annual burden hours. At $37.50 per hour ($12 + $63/2), the cost will be $100,800.

This section also requires that a State agency provide the reports, findings, and other results of the Medicare or private accreditation review to the appropriate EQRO. We estimate that it will take, on average, 8 hours for a State to disclose the necessary documentation to the appropriate EQRO. The total annual burden associated with this requirement is 2688 hours ($37.50 per hour) and $100,800.

This section also requires a State to include in its quality strategy information concerning the activities or standards for which it is obtaining information from Medicare or an accrediting organization. We believe that the burden for this information collection requirement is included in the burden addressed in the Medicaid managed care rule and approved under OMB number 0938.

§ 438.362 (Exemption from EQR)—Each year, exempted MCOs/PIHPs must provide to the State agency the most recent Medicare review findings reported to the MCO/PIHP. This information must include (1) all data, correspondence, information, and findings pertaining to the MCO's/PIHP's compliance with Medicare standards for access, quality assessment and performance improvement, health services, or delegation of these activities; (2) all measures of the MCO's/PIHP's performance; and (3) the findings and results of all performance improvement projects pertaining to Medicare enrollees.

If an exempted MCO/PIHP has been reviewed by a private accrediting organization and the survey results have been used to either fulfill certain requirements for Medicare external review under 42 CFR part 422, subpart D or to deem compliance with Medicare requirements as provided in § 422.156, the MCO/PIHP must submit a copy of all findings pertaining to its most recent accreditation review to the State agency. These findings must include accreditation survey results of evaluation of compliance with individual accreditation standards, noted deficiencies, corrective action plans, and summaries of unmet accreditation requirements.

The burden associated with these requirements is not applicable for 2 years following the final publication of this regulation. After 2 years, the time and effort for an exempted MCO/PIHP to disclose the findings of its most recent Medicare or private accreditation review to the State agency will be the burden associated with these requirements. We estimate, of the approximately 202 MCOs that potentially may provide Medicare services in addition to Medicaid services, State agencies will allow for approximately 10 percent of the MCOs to be exempt from the EQR requirement. We further estimate that it will take each MCO 8 hours to prepare and submit the necessary documentation to the State agency. Therefore, the total burden associated with this requirement is 10 percent of 202 MCOs × 8 hours = 160 annual burden hours. At a cost of $37.50 ($12 + $63/2) per hour, we assume a total cost of $6,000.

§ 438.364 (EQR results)—The EQRO responsible for the EQR function will be required to provide to the State agency a detailed technical report that describes for each mandatory and optional activity undertaken for the EQR, the objectives, technical methods of data collection and analysis, a description of the data obtained, conclusions drawn from the data, and the manner in which the conclusions were drawn as to the quality of the care furnished by the MCO/PIHP. In addition, the report must include: (1) An assessment of each MCO's/PIHP's strengths and weaknesses with respect to the quality, timeliness, and access to health care services furnished to Medicaid beneficiaries; (2) recommendations for improving the quality of health care services furnished by each MCO/PIHP; (3) as the State agency determines methodologically appropriate, comparative information about all MCOs/PIHPs, and (4) an assessment of the degree to which each MCO/PIHP has addressed effectively the recommendations for quality improvement, as made by the EQRO during the previous year's EQR.

The burden associated with this requirement is the time and effort for an EQRO to submit to a State agency a detailed technical report for each EQR conducted. We estimate that it will take an EQRO 200 hours to prepare and submit the necessary documentation to the State agency. Therefore, the total burden associated with this requirement is 458 technical reports (329 MCOs + 129 PIHPs) × 200 hours = 91,600 annual burden hours. Assuming wages of $63 per hour for professionals to comply with this requirement, the cost is $5,770,800.

This section also requires each State agency to provide copies of technical reports, upon request, to interested parties such as participating health care providers, enrollees and potential enrollees of the MCO/PIHP, beneficiary advocate groups, and members of the general public.

The burden associated with this requirement is the time and effort for a State agency to disclose copies of a given technical report to interested parties. We estimate that on average, it will take a State agency 8 hours to disclose the required information. Therefore, the total burden associated with this requirement is 329 MCOs + 129 PIHPs × 25 requests per MCO or PIHP × 8 hours = 91,600 annual burden hours and a cost ($12 per hour) of $1,099,200.

The information collection requirements contained in this final rule will be submitted to OMB for review. In accordance with the Paperwork Reduction Act, these requirements will not go into effect until approved by OMB.

If you comment on any of these information collection and record keeping requirements, please mail 3 copies directly to the following:

Centers for Medicare & Medicaid Services, Office of Information Services, Security and Standards Group, Division of CMS Enterprise Standards, Room N2-14-26, 7500 Security Boulevard, Baltimore, MD 21244-1850; Attn: Julie Brown, HCFA-2015-F; and

Office of Information and Regulatory Affairs, Office of Management and Budget, Room 10235, New Executive Office Building, Washington, DC 20503, Attn: Brenda Aguilar, CMS Desk Officer.

VI. Regulatory Impact Analysis

A. Overall Impact

We have examined the impacts of this rule as required by Executive Order 12866 (September 1993, Regulatory Planning and Review) the Regulatory Flexibility Act (RFA) (September 19, 1980, Pub. L. 96-354), section 1102(b) of the Social Security Act, the Unfunded Mandates Reform Act of 1995 (Pub. L. 104-4), and Executive Order 13132. Executive Order 12866 directs agencies to assess all costs and benefits of available regulatory alternatives and, when regulation is necessary, to select regulatory approaches that maximize net benefits, including potential economic, environmental, public health and safety effects, distributive impacts, and equity. A regulatory impact analysis (RIA) must be prepared for major rules with economically significant effects ($100 million or more in any one year).

The RFA requires agencies to analyze options for regulatory relief of small entities. For purposes of the RFA, we prepare a regulatory flexibility analysis unless we certify that a rule will not have a significant economic impact on a substantial number of small entities. Small entities include small businesses, nonprofit organizations, and governmental agencies. Most hospitals and other providers and suppliers are small entities, either by nonprofit status or by having revenues of $5 to $25 million or less annually. Individuals and States are not included in the definition of a small entity.

Section 1102(b) of the Act requires us to prepare a regulatory impact analysis for any proposed rule that may have a significant impact on the operations of a substantial number of small rural hospitals. This analysis must conform to the provisions of section 604 of the RFA. For purposes of section 1102(b) of the Act, we define a small rural hospital as a hospital that is located outside a Metropolitan Statistical Area and has fewer than 100 beds.

The Unfunded Mandates Reform Act (Pub. L. 104-4) requires that agencies prepare an assessment of anticipated costs and benefits before proposing any rule that may result in an annual expenditure by State, local, and tribal governments, in the aggregate, or by the private sector, of $110 million or more. This rule does not impose any mandates on State, local, or tribal governments, or the private sector that will result in an annual expenditure of $110 million or more.

Under Executive Order 13132, we are required to adhere to certain criteria regarding Federalism in developing regulations. We have determined that this regulation will not significantly affect States rights, roles, and responsibilities. Section 1903(a)(30)(C) of the Act currently requires an EQR for each contract a State has with a section 1903(m) organization. In accordance with section 4705 of the BBA, this rule will establish requirements and procedures for EQR of Medicaid MCOs. We require States to ensure that an annual EQR is performed by a qualified EQRO for each contracting MCO, the EQRO has adequate information to carry out the review, and that the results of the reviews are made available to interested parties such as participating health care providers, enrollees, advocate groups, and the general public. We also require that these EQR provisions apply to PIHPs and certain entities with comprehensive risk contracts that have been exempted from the requirements of section 1903(m) of the Act. We believe this is consistent with the intent of the Congress in enacting the quality provisions of the BBA. This rule would not require State agencies to dismantle EQR mechanisms that they have used to meet section 1902 (a)(30)(C) of the Act and which they have found to be effective and efficient. Rather, this rule would provide States greater flexibility in the types of entities they may use to conduct EQR.

We worked closely with States in developing this regulation. Specifically, in accordance with section 1932(c)(2)(A)(ii) of the Act, which requires the Secretary to consult with States to establish a method for identifying entities qualified to conduct EQR, we met with States and other stakeholders under the auspices of the NASHP to establish a criteria to identify qualified entities. Most of the recommendations made at this meeting have been incorporated into this rule. For recommendations not accepted, an explanation was provided in the December 1, 1999 proposed rule.

In addition, section 1932(c)(2)(A)(iii) of the Act requires the Secretary to coordinate with the NGA in contracting with an independent quality review organization to develop protocols to be used in EQR. To meet this requirement, we issued a request for proposal for one or more contractors to develop a set of review protocols for EQROs to use in the conduct of EQRs. Two State representatives selected by the NGA were members of the panel that reviewed and rated responding proposals. Moreover, part of the development of the EQR protocols includes convening an expert panel for review and comment of the protocols. State representatives were included in this process.

B. Anticipated Effects

In publishing this final rule, we considered two main alternatives. The first was to allow this final rule to be published, incorporating public comments on the proposed rule. The second alternative was to implement the provisions of the BBA as written, without expanding the regulations beyond the statutory language. We believe this final rule as written was the appropriate alternative to choose. Used in conjunction with the Medicaid Managed Care final rule published June 14, 2002, this final rule is a necessary tool for States to use to create and maintain strong, viable Medicaid managed care programs that deliver high quality health care in their State marketplaces and health care delivery systems. Further, we felt this final rule was necessary to implement the Congress' directive to the Secretary to establish a method for identifying entities qualified to conduct EQR.

We do not anticipate that the provisions in this final rule will have a substantial economic impact on most hospitals, including small rural hospitals. The BBA provisions include some new requirements on State agencies and MCOs, but not directly on individual hospitals. The impact on individual hospitals will vary according to each hospital's current and future contractual relationships with MCOs. Furthermore, the impact will also vary according to each hospital's current procedures and level of compliance with existing law and regulation pertaining to Medicaid managed care. For these reasons, this final rule will not have a significant impact on the operations of a substantial number of hospitals. The only other small entity affected by these regulations would be the EQROs. However, this rule does not impose additional burdens on them. Instead, the rule offers these organizations the benefit of opportunities for additional revenues. Thus we certify that this rule will not have a significant economic impact on a substantial number of small entities.

We do not anticipate a significant increase in Medicaid expenditures as a result of the publication of these regulations for the following reasons. First, approximately 42 States are currently obtaining 75 percent enhanced FFP for EQR activities carried out by QIOs and organizations that meet the requirements to contract with Medicare as a QIO. Permitting these State agencies to claim 75 percent matching for EQR activities conducted by the additional types of entities allowed by these regulations would therefore not result in increased costs to the extent that State agencies switch from QIO or organizations that meet the requirements to contract with Medicare as a QIO to these other entities. Moreover, we believe that, by expanding the pool of organizations available to conduct EQR, State agencies may be able to negotiate savings compared to current costs of dealing with PRO and PRO-like organizations. Additional savings may be realized through opportunities afforded by the final rule to coordinate EQR activities with quality reviews conducted for other purposes. Additional costs may arise where State agencies currently conduct quality review activities at 50 percent Federal matching rate that would now qualify for 75 percent, and from new EQR activities undertaken as a result of the BBA requirements.

In addition, even though we extend this requirement to PIHPs, again we do not expect this to significantly increase Medicaid expenditures. PIHP costs account for approximately 5 percent of the payments we make to capitated arrangements. Furthermore, State agencies currently conduct quality review activities on PIHPs at a 50 percent Federal matching rate. Additional costs may arise for States' quality review activities that would now qualify for 75 percent and for new quality review activities undertaken as a result of the activities required in this rule.

Although we cannot quantify these various cost and savings effects, we believe that their net impact would be well below the $100 million threshold for a major rule, and therefore that a regulatory impact analysis is not required. We do not believe that this final rule will cause MCOs to devote significantly more time to collect, organize and prepare for EQR than is already required by States. While the scope of work for EQR may be different under this final rule, we do not believe that the cost difference will be significant and States may actually be able to achieve savings since we are expanding the pool of organizations available to conduct EQR. Further, additional savings may also be realized through opportunities afforded by this rule to coordinate EQR activities with other quality and oversight activities. We acknowledge with the increased opportunity to contract with other qualified entities to conduct EQR, more States may avail themselves the 75 percent match for EQR activities. However, we do not believe this would represent a significant cost impact.

C. Conclusion

For these reasons, we are not preparing analyses for either the RFA or section 1102(b) of the Act because we have determined, and we certify, that this rule will not have a significant economic impact on a substantial number of small entities or a significant impact on the operations of a substantial number of small rural hospitals.

In accordance with the provisions of Executive Order 12866, this regulation was reviewed by the Office of Management and Budget.

List of Subjects

42 CFR Part 433

• Administrative practice and procedure
• Child support
• Claims
• Grant programs-health
• Medicaid
• Reporting and record keeping requirements

42 CFR Part 438

• Grant Programs—health
• Managed care entities
• Medicaid
• Quality assurance
• Reporting and record keeping requirements

For the reasons set forth in the preamble, the Centers for Medicare & Medicaid Services amends 42 CFR chapter IV as set forth below.

PART 433—STATE FISCAL ADMINISTRATION

A. Amend part 433 as set forth below.

1. The authority citation for part 433 continues to read as follows:

Authority: Sec. 1102 of the Social Security Act (42 U.S.C. 1302).

2. In § 433.15, add a new paragraph (b)(10) to read as follows:

§ 433.15

Rates of FFP for administration.

* * * * *

(b) * * *

(10) Funds expended for the performance of external quality review or the related activities described in § 438.358 of this chapter when they are performed by an external quality review organization as defined in § 438.320 of this chapter: 75 percent.

B. Add a new subpart E to part 438 to read as set forth below.

PART 438—MANAGED CARE

Subpart E—External Quality Review

438.310

Basis, scope, and applicability.

438.320

Definitions.

438.350

State responsibilities.

438.352

External quality review protocols.

438.354

Qualifications of external quality review organizations.

438.356

State contract options.

438.358

Activities related to external quality review.

438.360

Nonduplication of mandatory activities.

438.362

Exemption from external quality review.

438.364

External quality review results.

438.370

Federal financial participation.

Authority: Sec. 1102 of the Social Security Act (42 U.S.C. 1302).

Subpart E—External Quality Review

§ 438.310

Basis, scope, and applicability.

(a) Statutory basis. This subpart is based on sections 1932(c)(2), 1903(a)(3)(C)(ii), and 1902(a)(4) of the Act.

(b) Scope. This subpart sets forth requirements for annual external quality reviews of each contracting managed care organization (MCO) and prepaid inpatient health plan (PIHP), including—

(1) Criteria that States must use in selecting entities to perform the reviews;

(2) Specifications for the activities related to external quality review;

(3) Circumstances under which external quality review may use the results of Medicare quality reviews or private accreditation reviews; and

(4) Standards for making available the results of the reviews.

(c) Applicability. The provisions of this subpart apply to MCOs, PIHPs, and to health insuring organizations (HIOs) that began on or after January 1, 1986 that the statute does not explicitly exempt from requirements in section 1903(m) of the Act.

§ 438.320

Definitions.

As used in this subpart—

EQR stands for external quality review.

EQRO stands for external quality review organization.

External quality review means the analysis and evaluation by an EQRO, of aggregated information on quality, timeliness, and access to the health care services that an MCO or PIHP, or their contractors furnish to Medicaid recipients.

External quality review organization means an organization that meets the competence and independence requirements set forth in § 438.354, and performs external quality review, other EQR-related activities as set forth in § 438.358, or both.

Financial relationship means—

(1) A direct or indirect ownership or investment interest (including an option or nonvested interest) in any entity. This direct or indirect interest may be in the form of equity, debt, or other means and includes any indirect ownership or investment interest no matter how many levels removed from a direct interest; or

(2) A compensation arrangement with an entity.

Quality, as it pertains to external quality review, means the degree to which an MCO or PIHP increases the likelihood of desired health outcomes of its enrollees through its structural and operational characteristics and through the provision of health services that are consistent with current professional knowledge.

Validation means the review of information, data, and procedures to determine the extent to which they are accurate, reliable, free from bias, and in accord with standards for data collection and analysis.

§ 438.350

State responsibilities.

Each State that contracts with MCOs or PIHPs must ensure that—

(a) Except as provided in § 438.362, a qualified EQRO performs an annual EQR for each contracting MCO or PIHP;

(b) The EQRO has sufficient information to use in performing the review;

(c) The information used to carry out the review must be obtained from the EQR-related activities described in § 438.358.

(d) For each EQR-related activity, the information must include the elements described in § 438.364(a)(1)(i) through (a)(1)(iv);

(e) The information provided to the EQRO in accordance with paragraph (c) of this section is obtained through methods consistent with the protocols established under § 438.352; and

(f) The results of the reviews are made available as specified in § 438.364.

§ 438.352

External quality review protocols.

Each protocol must specify—

(a) The data to be gathered;

(b) The sources of the data;

(c) The activities and steps to be followed in collecting the data to promote its accuracy, validity, and reliability;

(d) The proposed method or methods for validly analyzing and interpreting the data once obtained; and

(e) Instructions, guidelines, worksheets, and other documents or tools necessary for implementing the protocol.

§ 438.354

Qualifications of external quality review organizations.

(a) General rule. The State must ensure that an EQRO meets the requirements of this section.

(b) Competence. The EQRO must have at a minimum the following:

(1) Staff with demonstrated experience and knowledge of—

(i) Medicaid recipients, policies, data systems, and processes;

(ii) Managed care delivery systems, organizations, and financing;

(iii) Quality assessment and improvement methods; and

(iv) Research design and methodology, including statistical analysis.

(2) Sufficient physical, technological, and financial resources to conduct EQR or EQR-related activities.

(3) Other clinical and nonclinical skills necessary to carry out EQR or EQR-related activities and to oversee the work of any subcontractors.

(c) Independence. The EQRO and its subcontractors are independent from the State Medicaid agency and from the MCOs or PIHPs that they review. To qualify as “independent”—

(1) A State agency, department, university, or other State entity may not have Medicaid purchasing or managed care licensing authority; and

(2) A State agency, department, university, or other State entity must be governed by a Board or similar body the majority of whose members are not government employees.

(3) An EQRO may not—

(i) Review a particular MCO or PIHP if either the EQRO or the MCO or PIHP exerts control over the other (as used in this paragraph, “control” has the meaning given the term in 48 CFR 19.101) through—

(A) Stock ownership;

(B) Stock options and convertible debentures;

(C) Voting trusts;

(D) Common management, including interlocking management; and

(E) Contractual relationships.

(ii) Deliver any health care services to Medicaid recipients;

(iii) Conduct, on the State's behalf, ongoing Medicaid managed care program operations related to oversight of the quality of MCO or PIHP services, except for the related activities specified in § 438.358; or

(iv) Have a present, or known future, direct or indirect financial relationship with an MCO or PIHP that it will review as an EQRO.

§ 438.356

State contract options.

(a) The State—

(1) Must contract with one EQRO to conduct either EQR alone or EQR and other EQR-related activities; and

(2) May contract with additional EQROs to conduct EQR-related activities as set forth in § 438.358.

(b) Each EQRO must meet the competence requirements as specified in § 438.354(b).

(c) Each EQRO is permitted to use subcontractors. The EQRO is accountable for, and must oversee, all subcontractor functions.

(d) Each EQRO and its subcontractors performing EQR or EQR-related activities must meet the requirements for independence, as specified in § 438.354(c).

(e) For each contract, the State must follow an open, competitive procurement process that is in accordance with State law and regulations and consistent with 45 CFR part 74 as it applies to State procurement of Medicaid services.

§ 438.358

Activities related to external quality review.

(a) General rule. The State, its agent that is not an MCO or PIHP, or an EQRO may perform the mandatory and optional EQR-related activities in this section.

(b) Mandatory activities. For each MCO and PIHP, the EQR must use information from the following activities:

(1) Validation of performance improvement projects required by the State to comply with requirements set forth in § 438.240(b)(1) and that were underway during the preceding 12 months.

(2) Validation of MCO or PIHP performance measures reported (as required by the State) or MCO or PIHP performance measure calculated by the State during the preceding 12 months to comply with requirements set forth in § 438.240(b)(2).

(3) A review, conducted within the previous 3-year period, to determine the MCO's or PIHP's compliance with standards (except with respect to standards under §§ 438.240(b)(1) and (2), for the conduct of performance improvement projects and calculation of performance measures respectively) established by the State to comply with the requirements of § 438.204(g).

(c) Optional activities. The EQR may also use information derived during the preceding 12 months from the following optional activities:

(1) Validation of encounter data reported by an MCO or PIHP.

(2) Administration or validation of consumer or provider surveys of quality of care.

(3) Calculation of performance measures in addition to those reported by an MCO or PIHP and validated by an EQRO.

(4) Conduct of performance improvement projects in addition to those conducted by an MCO or PIHP and validated by an EQRO.

(5) Conduct of studies on quality that focus on a particular aspect of clinical or nonclinical services at a point in time.

(d) Technical assistance. The EQRO may, at the State's direction, provide technical guidance to groups of MCOs or PIHPs to assist them in conducting activities related to the mandatory and optional activities that provide information for the EQR.

§ 438.360

Nonduplication of mandatory activities.

(a) General rule. To avoid duplication, the State may use, in place of a Medicaid review by the State, its agent, or EQRO, information about the MCO or PIHP obtained from a Medicare or private accreditation review to provide information otherwise obtained from the mandatory activities specified in § 438.358 if the conditions of paragraph (b) or paragraph (c) of this section are met.

(b) MCOs or PIHPs reviewed by Medicare or private accrediting organizations. For information about an MCO's or PIHP's compliance with one or more standards required under § 438.204(g), (except with respect to standards under §§ 438.240(b)(1) and (2), for the conduct of performance improvement projects and calculation of performance measures respectively) the following conditions must be met:

(1) The MCO or PIHP is in compliance with standards established by CMS for Medicare+Choice or a national accrediting organization. The CMS or national accreditation standards are comparable to standards established by the State to comply with § 438.204(g) and the EQR-related activity under § 438.358(b)(3).

(2) Compliance with the standards is determined either by—

(i) CMS or its contractor for Medicare; or

(ii) A private national accrediting organization that CMS has approved as applying standards at least as stringent as Medicare under the procedures in § 422.158.

(3) The MCO or PIHP provides to the State all the reports, findings, and other results of the Medicare or private accreditation review applicable to the standards provided for in § 438.204(g); and the State provides the information to the EQRO.

(4) In its quality strategy, the State identifies the standards for which the EQR will use information from Medicare or private accreditation reviews, and explains its rationale for why the standards are duplicative.

(c) Additional provisions for MCOs or PIHPs serving only dually eligibles. The State may use information obtained from the Medicare program in place of information produced by the State, its agent, or EQRO with respect to the mandatory activities specified in § 438.358 (b)(1) and (b)(2) if the following conditions are met:

(1) The MCO or PIHP serves only individuals who receive both Medicare and Medicaid benefits.

(2) The Medicare review activities are substantially comparable to the State-specified mandatory activities in § 438.358(b)(1) and (b)(2).

(3) The MCO or PIHP provides to the State all the reports, findings, and other results of the Medicare review from the activities specified under § 438.358(b)(1) and (b)(2) and the State provides the information to the EQRO.

(4) In its quality strategy, the State identifies the mandatory activities for which it has exercised this option and explains its rationale for why these activities are duplicative.

§ 438.362

Exemption from external quality review.

(a) Basis for exemption. The State may exempt an MCO or PIHP from EQR if the following conditions are met:

(1) The MCO or PIHP has a current Medicare contract under part C of title XVIII or under section 1876 of the Act, and a current Medicaid contract under section 1903(m) of the Act.

(2) The two contracts cover all or part of the same geographic area within the State.

(3) The Medicaid contract has been in effect for at least 2 consecutive years before the effective date of the exemption and during those 2 years the MCO or PIHP has been subject to EQR under this part, and found to be performing acceptably with respect to the quality, timeliness, and access to health care services it provides to Medicaid recipients.

(b) Information on exempted MCOs or PIHPs. When the State exercises this option, the State must obtain either of the following:

(1) Information on Medicare review findings. Each year, the State must obtain from each MCO or PIHP that it exempts from EQR the most recent Medicare review findings reported on the MCO or PIHP including—

(i) All data, correspondence, information, and findings pertaining to the MCO's or PIHP's compliance with Medicare standards for access, quality assessment and performance improvement, health services, or delegation of these activities;

(ii) All measures of the MCO's or PIHP's performance; and

(iii) The findings and results of all performance improvement projects pertaining to Medicare enrollees.

(2) Medicare information from a private, national accrediting organization that CMS approves and recognizes for Medicare+Choice deeming.

(i) If an exempted MCO or PIHP has been reviewed by a private accrediting organization, the State must require the MCO or PIHP to provide the State with a copy of all findings pertaining to its most recent accreditation review if that review has been used for either of the following purposes:

(A) To fulfill certain requirements for Medicare external review under subpart D of part 422 of this chapter.

(B) To deem compliance with Medicare requirements, as provided in § 422.156 of this chapter.

(ii) These findings must include, but need not be limited to, accreditation review results of evaluation of compliance with individual accreditation standards, noted deficiencies, corrective action plans, and summaries of unmet accreditation requirements.

§ 438.364

External quality review results.

(a) Information that must be produced. The State must ensure that the EQR produces at least the following information:

(1) A detailed technical report that describes the manner in which the data from all activities conducted in accordance with § 438.358 were aggregated and analyzed, and conclusions were drawn as to the quality, timeliness, and access to the care furnished by the MCO or PIHP. The report must also include the following for each activity conducted in accordance with § 438.358:

(i) Objectives.

(ii) Technical methods of data collection and analysis.

(iii) Description of data obtained.

(iv) Conclusions drawn from the data.

(2) An assessment of each MCO's or PIHP's strengths and weaknesses with respect to the quality, timeliness, and access to health care services furnished to Medicaid recipients.

(3) Recommendations for improving the quality of health care services furnished by each MCO or PIHP.

(4) As the State determines, methodologically appropriate, comparative information about all MCOs and PIHPs.

(5) An assessment of the degree to which each MCO or PIHP has addressed effectively the recommendations for quality improvement made by the EQRO during the previous year's EQR.

(b) Availability of information. The State must provide copies of the information specified in paragraph (a) of this section, upon request, through print or electronic media, to interested parties such as participating health care providers, enrollees and potential enrollees of the MCO or PIHP, recipient advocacy groups, and members of the general public. The State must make this information available in alternative formats for persons with sensory impairments, when requested.

(c) Safeguarding patient identity. The information released under paragraph (b) of this section may not disclose the identity of any patient.

§ 438.370

Federal financial participation.

(a) FFP at the 75 percent rate is available in expenditures for EQR (including the production of EQR results) and EQR-related activities set forth in § 438.358 conducted by EQROs and their subcontractors.

(b) FFP at the 50 percent rate is available in expenditures for EQR-related activities conducted by any entity that does not qualify as an EQRO.

(Catalog of Federal Domestic Assistance Program No. 93.778, Medical Assistance)

Dated: August 6, 2002.

Thomas A. Scully,

Administrator, Centers for Medicare & Medicaid Services.

Approved: October 3, 2003.

Tommy G. Thompson,

Secretary.

Footnotes

1.  In the Medicaid managed care final rule under § 438.240(c)(2) we permit States to calculate performance measures on the MCO's/PIHP's behalf in place of the MCO/PIHP calculating and reporting performance measures to the State. Under this circumstance, the validation of MCO/PIHP performance measures is not required as a mandatory activity but the State must submit the State-calculated performance measures to the EQRO for the EQR function as specified under § 438.358(b)(2). This issue is addressed later in the preamble in response to a comment.