2018-24785. Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Disclosures of Descriptive Presentations in Professional Oncology Prescription Drug Promotion

AGENCY:

Food and Drug Administration, HHS.

ACTION:

Notice.

SUMMARY:

The Food and Drug Administration (FDA or Agency) is announcing that a proposed collection of information has been submitted to the Office of Management and Budget (OMB) for review and clearance under the Paperwork Reduction Act of 1995.

DATES:

Fax written comments on the collection of information by December 14, 2018.

ADDRESSES:

To ensure that comments on the information collection are received, OMB recommends that written comments be faxed to the Office of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer, Fax: [phone], or emailed to [e-mail]. All comments should be identified with the OMB control number 0910-NEW and title “Disclosures of Descriptive Presentations in Professional Oncology Prescription Drug Promotion.” Also include the FDA docket number found in brackets in the heading of this document.

FOR FURTHER INFORMATION CONTACT:

Ila S. Mizrachi, Office of Operations, Food and Drug Administration, Three White Flint North, 10A-12M, 11601 Landsdown St., North Bethesda, MD 20852, 301-796-7726, [e-mail].

SUPPLEMENTARY INFORMATION:

In compliance with 44 U.S.C. 3507, FDA has submitted the following proposed collection of information to OMB for review and clearance.

Disclosures of Descriptive Presentations in Professional Oncology Prescription Drug Promotion

OMB Control Number—0910-NEW

I. Background

Section 1701(a)(4) of the Public Health Service Act (42 U.S.C. 300u(a)(4)) authorizes FDA to conduct research relating to health information. Section 1003(d)(2)(C) of the Federal Food, Drug, and Cosmetic Act (FD&C Act) (21 U.S.C. 393(d)(2)(C)) authorizes FDA to conduct research relating to drugs and other FDA regulated products in carrying out the provisions of the FD&C Act.

Under the FD&C Act and implementing regulations, promotional labeling and advertising about prescription drugs are generally required to be truthful, non-misleading, and to reveal facts material to the presentations made about the product being promoted (see sections 502(a) and (n), and 201(n) of the FD&C Act (21 U.S.C. 352(a) and (n), and 321(n)); see also 21 CFR 202.1). As a part of the ongoing evaluation of FDA's regulations in this area, FDA is proposing to study the impact of disclosures as they relate to presentations of preliminary and/or descriptive scientific and clinical data in promotional labeling and advertising for oncology products. The use of disclosures is one method of communicating information to healthcare professionals about scientific and clinical data, the limitations of that data, and practical utility of that information for use in treatment. These disclosures may influence prescriber comprehension and decision making and may affect how and what treatment they prescribe for their patients.

Pharmaceutical companies market directly to physicians through means that include publishing advertisements in medical journals, exhibit booths at physician meetings or events, sending unsolicited promotional materials to doctors' offices, and presentations (“detailing”) by pharmaceutical representatives (Ref. 1). Research suggests that detail aids sometimes contain carefully extracted data from clinical studies that, taken out of context, can exaggerate the benefits of a drug (Ref. 2) or contribute to physicians prescribing the drug for an inappropriate patient population.

Promotional labeling and advertising for cancer drugs deserve specific attention. Oncology drugs represented 26 percent of the 649 compounds under clinical trial investigation from 2006 to 2011 (Ref. 3). The past decade has seen a dramatic rise in the number of oncology drugs brought to market. In the past 18 months, over 22 percent of new drug approvals at FDA were new cancer drugs. In that time period, FDA approved 16 cancer drugs as new molecular entities or new therapeutic biologics out of a total of 72 (this does not include approvals of benign hematology products or biological license application approvals of blood reagents, or assays and anti-globulin products used in testing kits) (Refs. 4 and 5). Although overall survival remains the gold standard for demonstrating clinical benefit of a cancer drug, several additional endpoints including progression free survival, disease-free or recurrence-free survival, or durable response rate (including hematologic response endpoints) are accepted for either regular or accelerated approval depending on the magnitude of effect, safety profile, and disease context (Ref. 6). In addition to the endpoints upon which FDA approval of these products may be based, pharmaceutical companies typically assess many other endpoints to further explore the effects of their products. Some trials are designed to allow for formal statistical analyses of these additional endpoints; however, in many cases these endpoints are strictly exploratory and support only the reporting of descriptive results. For clinicians who are not specifically trained in clinical trial design, interpreting these endpoints may be challenging. Pharmaceutical companies invest heavily in the development and distribution of promotional materials to make oncologists aware of favorable clinical trial results.

When communicating scientific and clinical data, a specific statement that modifies or qualifies a claim (referred to for the purposes of this document as a disclosure) could be used to convey the limitations of the data and practical utility of the information for treatment. Much of the prior research on disclosures in this topic area has been limited to the dietary supplement arena with consumers (Refs. 7 to 10). Disclosures in professional pieces could influence prescriber comprehension as well as subsequent decision making; however, no published data exist regarding how prescribers use and understand scientific claims in conjunction with qualifying disclosures.

The proposed study seeks to address the following research questions:

1. Do disclosures mitigate potentially misleading presentations of preliminary and/or descriptive data in oncology drug product promotion?

2. Does the language (technical, non-technical) of the disclosure influence the effectiveness of the disclosure?

3. Does the presence of a general statement about the clinical utility of the data in addition to a specific disclosure influence processing of claims and disclosures?

4. Do primary care physicians (PCPs) and oncologists differ in their processing of claims and disclosures about preliminary and/or descriptive data?

5. Which disclosures do physicians prefer?

To address these questions, FDA has designed a study that will be conducted in three independent phases, each phase examining a data display in a promotional piece for a unique oncology or hematology product. Independent variables will include: (1) Specific disclosure (technical, non-technical, none), (2) general statement (present, absent), and (3) specialty (PCPs, oncologists). Each phase will have the following design:

Sample	General statement	Specific disclosure
		Technical	Non-technical	No disclosure
Oncologists	Present	•	•	Control.
	Absent	•	•
PCPs	Present	•	•	Control.
	Absent	•	•

Specific disclosures will include material information specifically related to the particular data display in question. As such, each specific disclosure may include clinical or statistical information related to the trial design, the statistical analysis plan of the trial, or any other material statistical or clinical information necessary for evaluation or interpretation of the data. The team developing the disclosures includes social science analysts, pharmacists, oncological medical officers, statisticians, and an oncology nurse. An example of the general statement is “This presentation includes exploratory information of uncertain clinical utility and should be interpreted cautiously when used to make treatment decisions.”

Outcome (dependent) variables will focus on the assessment of the data display as a whole as well as attention to the disclosure, if present. Specifically, we will examine recognition of the clinical endpoint in the data display, comprehension of the data display, perceptions of the strength of the data, and the perceived credibility of the promotional piece. We will also look at attention to the specific disclosure and the general statement, prescriber decisions, and prescriber preferences. Preferences will be determined by a secondary task at the end of the questionnaire that shows each participant all disclosure options and asks them to choose their preferred version.

Oncologists and PCPs will be recruited to participate via the internet. We plan to conduct one pretest with 90 participants and one study with 2,115 participants, both of which are expected to take approximately 20 minutes. Voluntary participants will view professionally developed promotional pieces that mimic currently available promotion and answer questions.

In the Federal Register of Monday, June 19, 2017 (82 FR 27845), FDA published a 60-day notice requesting public comment on the proposed collection of information (see above). Comments received along with our responses to the comments are provided below. Comments that are not PRA-relevant or do not relate to the proposed study are not included. For brevity, some public comments are paraphrased and therefore may not reflect the exact language used by the commenter. We assure commenters that the entirety of their comments was considered even if not fully captured by our paraphrasing. The following acronyms are used here: FRN = Federal Register Notice; DTC = direct-to-consumer; HCP = healthcare professional; PCP = primary care physicians; FDA = Food and Drug Administration; OPDP = FDA's Office of Prescription Drug Promotion.

The first public comment responder (regulations.gov tracking number 1k1-8xz7-mwcd) included eight individual comments, to which we have responded.

Comment 1: “It is unclear why FDA has chosen to conduct a study focused on oncology therapeutics and those medical specialists who prescribe such products.” [verbatim] All prescription drug products are treated the same according to regulations; therapeutic intent and prescriber type do not invoke alternate regulatory approaches.

Response: As we described in the 60-day Federal Register notice, promotional activities for oncology drugs are frequent and pervasive. Promotional labeling and advertising for cancer drugs deserve specific attention. Oncology drugs represented 26 percent of the 649 compounds under clinical-trial investigation from 2006 to 2011 (Ref. 3). The past decade has seen a dramatic rise in the number of oncology drugs brought to market. In the past 18 months, over 22 percent of new drug approvals at FDA were new cancer drugs. In that time period, FDA approved 16 cancer drugs as new molecular entities or new therapeutic biologics out of a total of 72 (this does not include approvals of benign hematology products or biological license application approvals of blood reagents, or assays and anti-globulin products used in testing kits) (Refs. 4 and 5). Although overall survival remains the gold standard for demonstrating clinical benefit of a cancer drug, several additional endpoints including progression free survival, disease-free or recurrence-free survival, or response rate (including hematologic response endpoints) are accepted for either regular or accelerated approval depending on the magnitude of effect, safety profile, and disease context (Ref. 6). In addition to the endpoints upon which FDA approval may be based, pharmaceutical companies typically assess many other endpoints to further explore the effects of their products. Some trials are designed to allow for formal statistical analyses of these additional endpoints; however, in many cases these endpoints are strictly exploratory and support only the reporting of descriptive results. For clinicians who are not specifically trained in clinical trial design, interpreting these endpoints can be challenging. Pharmaceutical companies invest heavily in the development and distribution of promotional materials to educate oncologists about favorable clinical trial results.

As another public comment responder (regulations.gov tracking number 1k1-8y3p-o6qb) notes, “We agree with the FDA's assessment that dedicated research is necessary regarding oncology drug promotion, particularly given that a significant proportion of the drug development pipeline is comprised of oncology products . . .”

Comment 2: FDA should use a more targeted approach, including a monadic design with 100 oncologists split into two experimental conditions.

Response: To clarify the study design, we are testing two variations of disclosure (specific disclosure: Technical, non-technical), two variations of general statement (general statement: Present or absent), plus a control (control: No specific disclosure). Participants will be healthcare professionals who are members of one of two medical populations and will be randomly assigned to one condition. Because we are examining the effects of multiple variables and their interactions, the necessary sample sizes will be larger than those suggested in this comment based on power analyses. We have, however, changed the study design based on multiple comments and will now examine only oncologists and primary care physicians.

Comment 3: The length of the survey looks long—at 17 pages, it appears that it will take approximately 30-40 minutes to complete.

Response: We have tested the survey in-house with individuals unfamiliar with the research project, and it appears that this survey will take approximately 15 minutes to complete.

Comment 4: Instead of using recall as a measure, respondents should be allowed to have access to the materials while answering questions to better approximate their actual experiences.

Response: It is an open question as to whether having the materials in front of them better approximates actual HCP experiences. In past discussions with HCPs, some have reported that they do refer back to materials that sales representatives leave, and others report that they do not receive leave-behind materials or do not refer to them again. In any case, we have a mixture of recall and comprehension questions in our questionnaire. For the recall questions, respondents will not be able to access the materials. They will, however, be able to review the materials while answering the comprehension questions.

Comment 5: Why is FDA examining non-oncologists at all? Why are you screening out oncology for specialists in question SPECIALTY2?

Response: HCPs of all types are exposed to prescription drug promotion. Depending on location (e.g., rural areas) and type of clinical setting, some non-oncologists may have a need to consider oncologic prescription drugs to treat their patients. We agree that oncologists are the most relevant population to study in this research. However, we also want to know whether specific education and experience influence the processing of claims, data, and disclosures. Upon further review, we agree that nurse practitioners and physician assistants without oncology experience are not a necessary group to investigate to answer our particular research questions. We intend to use PCPs as a control group to understand whether specific advanced training influences the understanding of preliminary and/or descriptive oncology data. Some PCPs may have experience with oncology prescriptions, particularly in rural areas. We will not eliminate PCPs without oncology experience, but we will measure oncology prescribing experience and use this variable as a covariate in our studies.

Comment 6: FDA should screen for the prescribing of oncologic products.

Response: Although we do not intend to screen out physicians without oncology prescribing experience, we will measure this variable and use this information to determine whether it plays a role in the responses of PCPs.

Comment 7: From this point (ENDPOINT) responses may be based on the ability of respondents to recall information vs. the absence/presence of disclosures. If FDA continues with this design, the Agency should be prepared to control for this in the study design.

Response: Because this is an experimental design with random assignment to condition, any fatigue with questions that may affect the recall of information should fall out evenly across conditions. Therefore, any differences would be the result of our manipulations, in this case, the presence and form of disclosures. We have given thought to the ordering of the questions so that the most important questions are asked in the beginning of the survey rather than toward the end.

Comment 8: The answer to this question (CAUTIOUS) may be influenced more by personal and subjective opinion vs. the content of the disclosure.

Response: Because of the experimental design with random assignment to condition, personal and subjective opinions should be evenly and randomly spread across experimental conditions. However, upon further review, we have determined that this question has limited utility and we will delete it.

The second public comment responder (regulations.gov tracking number 1k1-8y3p-o6qb) included one individual comment. They reported that they support the study specifically and OPDP's overall research efforts generally, and they agree that oncology deserves special attention. We thank this commenter for taking the time to provide this comment to us.

The third public comment responder (regulations.gov tracking number 1k1-8y5u-5vp0) included eight individual comments, to which we have responded.

Comments 1 and 2: The commenter supports FDA social science research and this specific project, as well as the Disclosures study (Docket No. FDA-2017-N-0558). “FDA's collective research indicates a considered, objective updating of the FDA's advertising regulations, including the use of disclosures to prevent misleading claims in advertisements for oncology products, is timely . . . . Enabling disclaimers would be one way to enable innovators to advertise new oncology therapeutics for their approved uses in ways which would be non-misleading.”

Response: Thank you for your support.

Comment 3: The commenter suggests making sure that primary care physicians and advanced practitioners have experience in the oncology field—otherwise, it seems useless to include less knowledgeable respondents whose answers are more speculative. Overall, they question whether advanced practitioners are appropriate for this study at all.

Response: We have removed advanced practitioners from the design. We will measure the oncology prescribing experience of the PCPs in our sample, but we will not eliminate those who do not have specific oncology training. One of our research questions is whether specific training and experience in oncology influences the understanding of preliminary oncology data. To do that, we need to include a group of practitioners who may not have specific training and experience in oncology, but who are licensed practitioners permitted by law to prescribe oncology drugs, and who, in some cases, may do so.

Comment 4: If the only data being presented for BENEFICIAL, EVIDENCE1 and EVIDENCE2 are the endpoints for the disclosure without presenting overall survival or more clinically validated data, we suggest removing these questions.

Response: The pieces include other clinically validated data as would be typical in an existing piece for an oncology indication.

Comment 5: Remove CONFUSING2 because it asks physicians to speculate.

Response: As this item is a perception measure, as opposed to an accuracy measure, it is reasonable to consider some level of speculation. Moreover, in cognitive testing, HCPs responded without difficulty.

Comment 6: For SCRIPT4, add an “I don't know” option instead of instructing respondents to “make your best guess.”

Response: This item was cognitively tested and participants expressed no difficulty answering it.

Comment 7: Those who respond “not at all familiar” to FAMILIAR should skip BTKNOW1, BTKNOW2, and ACCEL.

Response: We agree with this comment. Those who respond “not at all familiar” to FAMILIAR will skip the three items mentioned above.

Comment 8: BTDV1 and BTDV2 present incomplete data and therefore it is unclear how this will be a useful question. The commenter suggests either adding an “I need more information” option or removing the question.

Response: These items present incomplete data but we have provided enough data that HCPs should be able to make a choice. HCPs in cognitive testing exhibited no difficulty with the question. There is no existing data on perceptions of FDA's “breakthrough” designation and this item will provide at least rudimentary information. Please note that each respondent will see only one of the items. These items are carefully crafted to avoid order effects and alphabetical effects.

The fourth public commenter (regulations.gov tracking number 1k1-8y5u-koc0) included 15 individual comments, to which we have responded.

Comment 1 (summarized): The commenter is concerned with the Agency's recent approaches to studies in this area. FDA has proposed to undertake projects in a variety of disparate topics without articulating a clear, overarching research agenda or adequate rationales on how the proposed research related to the goal of further protecting public health. Within the last year, the Agency has increased such efforts at an exponential pace. At times, FDA proposes new studies seemingly without fully appreciating its own previous research published on the Office of Prescription Drug Promotion (OPDP) website. Proposed studies are often unnecessary in light of existing data. The commenter suggests that the Agency publish a comprehensive list of its prescription drug advertising and promotion studies from the past five years and articulate a clear vision for its research priorities for the near future. Going forward, FDA should use such priorities to explain the necessity and utility of its proposed research and should provide a reasonable rationale for the proposed research.

Response: OPDP's mission is to protect the public health by helping to ensure that prescription drug information is truthful, balanced, and accurately communicated, so that patients and healthcare providers can make informed decisions about treatment options. OPDP's research program supports this mission by providing scientific evidence to help ensure that our policies related to prescription drug promotion will have the greatest benefit to public health. Toward that end, we have consistently conducted research to evaluate the aspects of prescription drug promotion that we believe are most central to our mission, focusing in particular on three main topic areas: Advertising features, including content and format; target populations; and research quality. Through the evaluation of advertising features we assess how elements such as graphics, format, and disease and product characteristics impact the communication and understanding of prescription drug risks and benefits; focusing on target populations allows us to evaluate how understanding of prescription drug risks and benefits may vary as a function of audience; and our focus on research quality aims at maximizing the quality of research data through analytical methodology development and investigation of sampling and response issues. Because we recognize the strength of data and the confidence in the robust nature of the findings is improved through the results of multiple converging studies, we continue to develop evidence to inform our thinking. We evaluate the results from our studies within the broader context of research and findings from other sources, and this larger body of knowledge collectively informs our policies as well as our research program. Our research is documented on our homepage, which can be found at: https://www.fda.gov/​aboutfda/​centersoffices/​officeofmedicalproductsandtobacco/​cder/​ucm090276.htm. The website includes links to the latest Federal Register notices and peer-reviewed publications produced by our office. The website maintains information on studies we have conducted, dating back to a survey of DTC attitudes and behaviors conducted in 1999.

Comment 2: FDA should provide more detail about the study to stakeholders. “It is not clear from this description whether the study will yield useful information to evaluate whether disclosures provide appropriate contextual information in certain communications, whether such disclosures can be made more effective, and where the disclosures are necessary to ensure communications are truthful and non-misleading.”

Response: We have described the purpose of the study, the design, the population of interest, and have provided the questionnaire to numerous individuals upon request. These materials have proven sufficient for others to comment publicly, and for academic experts to peer-review the study successfully. Our full stimuli are under development during the PRA process. We do not make draft stimuli public during this time because of concerns that this may contaminate our participant pool and compromise the research.

Comment 3: The Agency should wait until it has completed its broader study on disclosures more generally. This study is duplicative of other studies.

Response: As we discussed in the 60-day Federal Register notice, oncological products deserve specific attention as they account for nearly a quarter of new drug approvals and can involve the assessment of complicated endpoints. Moreover, they have specific disclosures that are unique to their products and deserve particular study. The other disclosures study (Docket No. FDA-2017-N-0558) will provide important information about a variety of disclosures in different medical conditions. One research study cannot answer all questions or study all aspects of an issue. These two studies will be complementary but not redundant. Please also refer to our response to comment 1 from the first commenter above.

Comment 4: Given that FDA grants approval based on certain preliminary and descriptive data, and that various limitations as to the underlying data must already be communicated to prescribers, there appears to be limited utility in researching disclosures regarding such data.

Response: We disagree that FDA grants approval on preliminary or descriptive data. The evidentiary standard is substantial evidence. While we recognize that no single development program can answer all questions about a particular drug in all populations, it is not accurate to describe the evidence supporting approval as descriptive or preliminary. What is potentially unique about oncology products is that many are approved under accelerated approval, in which the substantial evidence of benefit is on a surrogate endpoint that is reasonably likely to predict a clinical outcome. There remains some residual uncertainty regarding whether the effect on a surrogate endpoint will directly correlate with a clinical benefit; however, there is a requirement that confirmatory evidence of clinical benefit be obtained after approval. This residual uncertainty about the relationship of the surrogate endpoint to the clinical benefit is communicated to prescribers through the FDA-required labeling (e.g., inclusion of a limitation of use in the Indications and Usage section of the FDA-required labeling). In addition, reliance on a surrogate endpoint under accelerated approval is only done for serious diseases when the evidence indicates that the product provides a meaningful therapeutic benefit to patients over existing treatments (21 CFR 314.500).

However, this study does not focus on endpoints that formed the basis for approval. This study focuses on promotional displays of preliminary and/or descriptive data. It has not been established whether and how current disclosure-type additions to promotion are adequately communicating the limitations around this type of data, and that is the purpose of the current study. Given the importance of these limitations, it is crucial to make sure that promotional materials directed at to prescribers convey limitations appropriately. Past research has shown that simply including a statement somewhere in a promotional piece does not grant it automatic usefulness (Refs. 7 to 10).

Comment 5: FDA notes that, “[a]lthough overall survival remains the gold standard for demonstrating clinical benefit of a drug, several additional endpoints are accepted as surrogates . . . [including] disease-free survival, objective response rate, complete response rate, progression-free survival, and time to progression.” The Agency further states that “[f]or clinicians who are not specifically trained in clinical trial design, interpreting these endpoints may be challenging.” FDA does not cite any sources for this claim, and there is no basis for thinking that clinicians do not have a thorough understanding of the data limitations described in presentations of preliminary or descriptive scientific and clinical data. This is especially true of oncologists.

Response: This statement was not intended to be a claim, but rather a statement of concern. Studies report that physicians lack sufficient critical knowledge and skills to evaluate evidence based medicine (EBM) and may be influenced by the way study results are presented (Refs. 11 to 13). FDA recently conducted a systematic review of research related to prescribers' training and critical appraisal skills related to clinical trials (Ref. 14). The study found that extant physician knowledge and skills regarding certain statistical concepts and trial designs were in the middle of the possible outcome score range, at levels below those considered mastery, even after interventions designed to increase knowledge and skills. Evidence suggested that clinical credentials affect understanding and use of clinical data. Physicians with formal training in biostatistics, epidemiology, clinical research, or EBM demonstrated higher levels of knowledge and appraisal skills than those with usual medical education and training.

Comment 6: The specific disclosures outlined by FDA include “clinical or statistical information related to the trial design, the statistical analysis plan of the trial, or any other material statistical or clinical information necessary for evaluation or interpretation of the data.” The breadth of the proposed specific disclosures appears burdensome, unnecessary, and overwhelming for the purposes of the proposed survey.

Response: These concepts were provided as examples of the types of information that may be necessary for the accurate evaluation or interpretation of the data. This statement was not meant to imply that all of these concepts would be included in disclosures used in this study.

Comment 7: PCPs and non-oncology mid-level practitioners will provide much less utility in their survey responses regarding such disclosures.

Response: We have changed the design. See previous comments and responses.

Comment 8: The Agency proposes to conduct its survey via electronic media. FDA should consider testing non-electronic media, including printed sales aids, as these forms are often reviewed by the proposed study subjects.

Response: To clarify, the stimuli presented will consist of mock print materials in .pdf format, administered via the internet. Conducting the study in person would require a greater expenditure of resources without appreciable benefits.

Comment 9: The Agency should consider using a consistent sliding scale format for all survey responses. Just within pages 7-9 of the survey, FDA proposes numerous different schemes for survey responses: (1) “Not at all beneficial—Extremely beneficial;” (2) “Completely agree—Completely disagree;” (3) “No evidence—Strong (or conclusive) evidence;” (4) “Not at all complex—Extremely complex;” (5) “Not at all confusing—Extremely confusing;” and (6) additional responses in which subjects are asked to agree with certain statements. The variety in response options is confusing in format and could potentially introduce error. To the extent possible, FDA should make the response format consistent throughout the survey. Further, the Agency should ensure the sliding scale format consistently provides an odd number of responses to permit a “neutral” response. Certain questions (e.g., the IMPROVE question on page 7) provide six choices, not permitting a neutral response.

Response: Although one scale throughout would be easier for respondents, it will not necessarily provide better data. When a series of adjacent questions have the same response options, respondents may use a response mechanism known as anchoring and adjusting when reporting (Ref. 15). Respondents use their response to the initial survey question on a topic as the “cognitive anchor,” and then adjust up or down based on subsequent questions (Ref. 16). Anchoring and adjusting is more likely to occur for questions when respondents have some level of uncertainty in their answer (Ref. 17), which would be expected in this study. Epley and Gilovich (Ref. 17) found that when respondents use an anchoring and adjusting strategy, they often adjust insufficiently. Respondents start with the response they used for the first item and then search for the next value that is “close enough.” This can result in responses to adjacent items being more similar than responses to the same items if they used an item-specific scale (Not at all beneficial to Extremely beneficial; Not at all complex to Extremely complex). Using the same scale across all survey questions would artificially increase the correlations between all questions making it more difficult to identify differences based on the stimuli or respondent characteristics. Furthermore, use of item-specific scales compared with agree-disagree scales reduces primacy effects (tendency of respondents to select options at the beginning of the list) (Ref. 18), and increases reliability and validity (Ref. 19). Careful consideration was made to use agree-disagree scales only when item-specific scales would not be appropriate (e.g., presenting patient vignettes) or unnecessarily complex (e.g., asking about “complex terminology, statistical terms, or jargon,” inquiring about “strong” evidence).

In terms of neutral points, given the focus of the questions, we believe that offering a neutral response option is not necessary to measure opinions and attitudes accurately. Consequently, our objective is to force a selection and have participants make at least a weak commitment in either a positive or negative direction. Of concern is that offering a neutral midpoint could potentially encourage “satisficing”—cuing participants to choose a neutral response because it is offered (Ref. 20). Additionally, providing a midpoint leads to the loss of information regarding the direction in which people lean (Ref. 21). Research has found that neither format (either with or without a neutral point) is necessarily better or produces more valid or reliable results (Ref. 22). Instead, it should be left to the researcher to determine the goals of the study. During cognitive testing, a majority of participants were satisfied with the response options and all participants felt comfortable choosing a response in the absence of a midpoint.

Use of a midpoint is an issue we have examined in previous studies and we determined that we achieve valid and reliable responses without a midpoint. To increase consistency with measures used in previous studies, and in support of the arguments presented above, we are opting to exclude a midpoint. Finally, if a participant does not feel that they can choose a response because of a lack of a neutral option, they will be able to skip the question.

Comment 10: In the BENEFICIAL question on page 7 of the survey, it is unclear what relevance the subject's perception of clinical benefit of a drug has in studying FDA's proposed research purpose.

Response: For prescription drug products, advertisers must ensure that both the benefits and limitations are appropriately conveyed. If limitations are not appropriately conveyed, viewers may have an inflated view of the benefits of the product, relative to its risks. This question investigates this issue.

Comment 11: In a study setting, subjects may be prone to read and pay attention to more or all of the information presented. Subjects also are more aware of the importance of their responses. The Agency should address what efforts it will take to avoid response bias by study subjects.

Response: We initially had this concern many years ago when OPDP began conducting research. However, since that time, we have seen no evidence of this bias. In fact, we often deal with the opposite problem—ensuring that respondents spend a minimum amount of time looking at mock materials. Moreover, cognitive testing participants have told us that they would not spend extra time on materials if they were answering questions without an interviewer in the room. Individuals, especially HCPs, are busy, and we believe our experiments do not overestimate the amount of time participants spend on actual materials.

Comment 12: Although the draft survey did not contain Informed Consent text, the Agency should ensure that this text does not state or imply that the survey is being conducted on behalf of the U.S. Food and Drug Administration. Such a statement could potentially influence subjects' responses to study questions. Instead, this information might be provided at the conclusion of the study.

Response: We will ensure that all materials reference the U.S. Department of Health and Human Services rather than FDA.

Comment 13: The CAUTIOUS question on page 8 should be rephrased or omitted. Subjects may be biased to respond that they interpret all data with caution, regardless of the underlying scientific evidence presented in study stimuli.

Response: We agree with this comment and will delete this item.

Comment 14: The DECISIONS question on page 8 should be omitted. How survey participants “feel about the data presented” will be highly dependent on their external experience in making prescribing decisions. This question thus may lead to highly variable results.

Response: Because this is an experimental design with random assignment to conditions, external experiences in making prescribing decisions should be randomly scattered across experimental conditions. Thus, we will be able to infer causation to our manipulations of disclosures if we find any differences across experimental conditions. We believe the presence and form of the disclosure may influence this dependent variable and believe it will reveal important information about how HCPs process the data.

Comment 15: The PREFERENCE and PREFERWHY questions on page 16 should be moved to the beginning of the survey or omitted altogether. Subjects' responses regarding their preference in sales aid disclosure statements will be heavily influenced by earlier portions of the survey.

Response: We have given careful thought to the ordering of the questions in the questionnaire. Because preference is of secondary interest to us, we have included it after our primary outcome variables, so that it does not influence them. We recognize that prior questions may influence these measures and will interpret them with that caveat in mind.

FDA estimates the burden of this collection of information as follows:

Table 1—Estimated Annual Reporting Burden ¹

Activity	Number of respondents	Number of responses per respondent	Total annual responses	Average burden per response 2	Total hours
Pretest Study Screener Completes	150	1	150	0.03 (2 minutes)	5
Main Study Screener Completes	3,525	1	3,525	0.03 (2 minutes)	106
Pretest Study	90	1	90	0.33 (20 minutes)	30
Main Study	2,115	1	2,115	0.33 (20 minutes)	698
Total					839
1 No capital costs or operating and maintenance costs are associated with collection of this information.
2 Burden estimates of less than 1 hour are expressed as a fraction of an hour in decimal format.

II. References

The following references marked with an asterisk (*) are on display at the Dockets Management Staff (HFA-305), Food and Drug Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852, and are available for viewing by interested persons between 9 a.m. and 4 p.m., Monday through Friday; they also are available electronically at https://www.regulations.gov. References without asterisks are not on public display at https://www.regulations.gov because they have copyright restriction. Some may be available at the website address, if listed. References without asterisks are available for viewing only at the Dockets Management Staff. FDA has verified the website addresses, as of the date this document publishes in the Federal Register, but websites are subject to change over time.

*1. Johar, K., “An Insider's Perspective: Defense of the Pharmaceutical Industry's Marketing Practices,” Albany Law Review, 76:299-334, 2012-2013.

*2. Wick, C., M. Egger, S. Trelle, et al., “The Characteristics of Unsolicited Clinical Oncology Literature Provided by Pharmaceutical Industry,” Annals of Oncology, 18(9):1580-1582, 2007.

* 3. Fisher, J.A., M.D. Cottingham, and C.A. Kalbaugh, “Peering Into the Pharmaceutical `Pipeline': Investigational Drugs, Clinical Trials, and Industry Priorities,” Social Science & Medicine, 131:322-330, 2015.

4. Centerwatch, “FDA Approved Drugs for Oncology,” https://www.centerwatch.com/​drug-information/​fda-approved-drugs/​therapeutic-area/​12/​oncology (accessed on October 5, 2018).

5. https://www.fda.gov/​BiologicsBloodVaccines/​DevelopmentApprovalProcess/​BiologicalApprovalsbyYear/​ucm596371.htm.

* 6. Beaver, J.A., L.J. Howie, L. Pelosof, et al., “A 25-Year Experience of U.S. Food and Drug Administration Accelerated Approval of Malignant Hematology and Oncology Drugs and Biologics: A Review,” JAMA Oncology, 4:849-856, 2018.

* 7. Dodge, T. and A. Kaufman, “What Makes Consumers Think Dietary Supplements Are Safe and Effective? The Role of Disclaimers and FDA Approval,” Health Psychology, 26:513-517, 2007.

* 8. Dodge, T., D. Litt, and A. Kaufman, “Influence of the Dietary Supplement Health and Education Act on Consumer Beliefs About the Safety and Effectiveness of Dietary Supplements,” Journal of Health Communication, 16(3):230-244, 2011.

* 9. Mason, M.J., D.L. Scammon, and X. Fang, “The Impact of Warnings, Disclaimers, and Product Experience on Consumers' Perceptions of Dietary Supplements,” The Journal of Consumer Affairs, 41(1):74-99, 2007.

* 10. France, K.R. and P.F. Bone, “Policy Makers' Paradigms and Evidence From Consumer Interpretations of Dietary Supplement Labels,” The Journal of Consumer Affairs, 39(1):27-51, 2005.

* 11. Ghosh, A.K. and K. Ghosh, “Translating Evidence-Based Information into Effective Risk Communication: Current Challenges and Opportunities,” The Journal of Laboratory and Clinical Medicine, 145(4):171-180, 2005.

* 12. Harewood, G.C. and L.M. Hendrick, “Prospective, Controlled Assessment of the Impact of Formal Evidence-Based Medicine Teaching Workshop on Ability to Appraise the Medical Literature,” Irish Journal of Medical Science, 179(1):91-94, 2010.

* 13. Fritsche, L., T. Greenhalgh, Y. Falck-Ytter, et al., “Do Short Courses in Evidence Based Medicine Improve Knowledge and Skills? Validation of Berlin Questionnaire and Before and After Study of Courses in Evidence Based Medicine,” British Medical Journal, 325(7376):1338-1341, 2002.

* 14. Kahwati, L., D. Carmondy, N. Berkman, et al., “Prescribers' Knowledge and Skills for Interpreting Research Results: A Systematic Review,” Journal of Continuing Education in the Health Professions, 37(2):129-136, 2017.

* 15. Tversky, A. and D. Kahneman,” Judgment Under Uncertainty: Heuristics and Biases,” Science, 185(4157):1124-1131, 1974.

* 16. Gehlbach, H. and S. Barge, “Anchoring and Adjusting in Questionnaire Responses,” Basic and Applied Social Psychology, 34(5):417-433, 2012.

* 17. Epley, N. and T. Gilovich, “The Anchoring-and-Adjustment Heuristic: Why the Adjustments are Insufficient,” Psychological Science, 17(4):311-318, 2006.

* 18. Höhne, J.K. and D. Krebs, “Scale Direction Effects in Agree/Disagree and Item-Specific Questions: A Comparison of Question Formats,” International Journal of Social Research Methodology, 21(1):91-103, 2017.

* 19. Saris, W.E., M. Revilla, J.A. Krosnick, et al., “Comparing Questions with Agree/Disagree Response Options to Questions with Item-Specific Response Options” Survey Research Methods, 4:61-79, 2010.

* 20. Krosnick, J.A. and S. Presser, “Question and Questionnaire Design,” In: Handbook of Survey Research (pp. 263-314). Bingley, United Kingdom: Emerald Group Publishing Limited, 2010.

21. Converse, J.M. and S. Presser, Survey Questions: Handcrafting the Standardized Questionnaire, (No. 63). Thousand Oaks, CA: SAGE Publications, 1986.

22. DeVellis, R.F., Scale Development: Theory and Applications, (Vol. 26). Thousand Oaks, CA: SAGE Publications, 2016.

Dated: November 7, 2018.

Leslie Kux,

Associate Commissioner for Policy.