Donald Therasse - Comment

July 17, 2009 Division of Dockets Management (HFA-305) Food and Drug Administration 5630 Fishers Lane, Rm. 1061 Rockville, MD 20852. Via Internet to: http://www.regulations gov Re: FDA-2009-N-0192-0002 Dear Sir(s)/Madame(s), We are pleased to submit comments on the Sentinel Initiative contractual report entitled “Evaluation of Signal Detection Methods For Use In Prospective Post- Licensure Medical Product Safety Surveillance” [Document ID: FDA-2009-N-0192- 0002]. We support the Food and Drug Administration (FDA) in their efforts to develop the Sentinel Initiative as required by Section 905 of the Food and Drug Administration Amendments Act (FDAAA) of 2007. We also applaud FDA in soliciting input and comments from stakeholders and making this information available to the public. We look forward to the continued development of the plans for implementing the Sentinel Initiative. We also look forward to providing additional comments and insights once more specific information regarding the implementation of the Sentinel Initiative is outlined. As the Sentinel Initiative is envisioned to strengthen FDA’s ability to monitor the performance of marketed products there are a number of elements that will need to be addressed in order to have a successful and sustainable Sentinel system. These include topics such as governance, operations, database models, data sources, scientific methods, communications, privacy, and legal considerations. We would like to commend Nelson and colleagues in their efforts to evaluate existing signal detection methods for potential use in the Sentinel Initiative. Overall, we found the report to be very informative and a nice summary of possible methods. We agree with the author’s recommendation that sequential testing methods offer a useful framework which could be applied to the Sentinel system to efficiently monitor product safety. Although the authors recommended several methods not be pursued further (e.g., SPRT, CUSUM, PRR data mining, and Bayesian data updating), we suggest that, due to lack of comparative results, these methods need to be evaluated and compared relative to their performance when utilizing various observational data sources. Listed below are more specific comments regarding the report. Specific Comments on Sentinel Initiative – Evaluation of Signal Detection Methods 1) Definitions: As part of defining the scope of the Sentinel Initiative, we recommend that definitions be provided for relevant terms such as signals, active surveillance, epidemiological cohort study, etc. (section 1.2). 2) Adjusting for Confounding Factors: As the document states, the maxSPRT method is limited in providing adjustments for confounding factors. Alternative methods should be proposed for active surveillance, such as propensity score adjustment/matching and prior event rate ratio (PERR). In addition, generalized estimation equations (GEE) should be considered if the data are correlated (section 2.1.5). 3) Other Methods: The authors briefly outline other methods for consideration in section 6. We recommend other methods be evaluated for performance, including methods for controlling for multiplicity across drug/event combinations such as false discovery rate or Bayesian methods. Assessment of such methods may be useful, especially when data mining approaches have the potential to identify a number of false positive signals (section 6). 4) Terminology Clarification: It will also be important to clarify additional terms such as “false and true” signals (section 7.3). A signal is something that needs further assessment. Some signals turn out to be true effects and some turn out to be false effects; another interpretation could be the signal was “confirmed” or “refuted.” 5) Signal Confirmation Methods: The methodologies stated for signal confirmation focus on one method of confirming signals (i.e., a two-phase sampling design; section 7.3). While this may be a useful technique, there are many other ways to confirm or refute a signal and it would be useful to include additional methods. For example, one could conduct a similar study in a different database, or even conduct a clinical trial, nonclinical study, an alternative pharmacoepidemiological study, registry etc. We appreciate the opportunity to comment on the current report regarding possible signal detection methods for the Sentinel Initiative. Please let us know if you have questions or need additional clarification. We would also welcome the chance to provide additional comments and insights as more detailed information is released on the Sentinel Initiative. Sincerely, Donald G. Therasse, M.D. Vice President, Global Patient Safety Lilly Research Laboratories Eli Lilly and Company