2018-17360. Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Experimental Study of Risk Information Amount and Location in Direct-to-Consumer Print Ads

AGENCY:

Food and Drug Administration, HHS.

ACTION:

Notice.

SUMMARY:

The Food and Drug Administration (FDA) is announcing that a proposed collection of information has been submitted to the Office of Management and Budget (OMB) for review and clearance under the Paperwork Reduction Act of 1995 (the PRA).

DATES:

Fax written comments on the collection of information by September 13, 2018.

ADDRESSES:

To ensure that comments on the information collection are received, OMB recommends that written comments be faxed to the Office of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer, Fax: [phone], or emailed to [e-mail]. All comments should be identified with the OMB control number 0910—New and title “Experimental Study of Risk Information Amount and Location in Direct-to-Consumer Print Ads.” Also include the FDA docket number found in brackets in the heading of this document.

FOR FURTHER INFORMATION CONTACT:

Ila S. Mizrachi, FDA PRA Staff, Office of Operations, Food and Drug Administration, Three White Flint North, 10A-12M, 11601 Landsdown St., North Bethesda, MD 20852, 301-796-7726, [e-mail].

SUPPLEMENTARY INFORMATION:

In compliance with 44 U.S.C. 3507, FDA has submitted the following proposed collection of information to OMB for review and clearance.

Experimental Study of Risk Information Amount and Location in Direct-to-Consumer Print Ads

OMB Control Number 0910—NEW

I. Background

Section 1701(a)(4) of the Public Health Service Act (42 U.S.C. 300u(a)(4)) authorizes FDA to conduct research relating to health information. Section 1003(d)(2)(C) of the Federal Food, Drug, and Cosmetic Act (the FD&C Act) (21 U.S.C. 393(d)(2)(C)) authorizes FDA to conduct research relating to drugs and other FDA regulated products in carrying out the provisions of the FD&C Act.

Section 502(n) of the FD&C Act (21 U.S.C.352(n)) specifies that advertisements (ads) for prescription drugs and biological products must provide a true statement of information “in brief summary” describing the advertised product's “side effects, contraindications and effectiveness.” This is clarified further in the prescription drug advertising regulations. The brief summary shall include a true statement of information relating to side effects, contraindications, warnings, precautions, and any such information under such headings as cautions, special considerations, important notes, etc., as well as effectiveness (§ 202.1(e)(1)). The prescription drug advertising regulations also specify that the phrase side effect and contraindication refers to all of the categories of risk information contained in the required, approved, or permitted product labeling written for health professionals, including the side effects, warnings, precautions, and contraindications (§ 202.1(e)(3)(iii)). Ads must also “present a fair balance between information relating to side effects and contraindications and effectiveness . . .” An ad must present true information relating to side effects and contraindications in comparable depth and detail with the claims for effectiveness or safety (§ 202.1(e)(5)(ii)).

To fulfill the regulatory requirements for fair balance and the brief summary, sponsors have typically included risk information about the product in direct-to-consumer (DTC) print ads both in the main part of the ad where the product claims appear, and in a separate brief summary page. The section of the main ad where the risks appear is often referred to as the “Important Safety Information” (ISI). Including risks in both the ISI and the brief summary may have advantages. Some research has found that repetition of information improves recall, especially for older adults (Ref. 1). This might result in improved recall for risks that appear both in the ISI and brief summary. However, it is possible that risks appearing on the main page in the ISI may be more likely to be read than risks appearing in the brief summary. Based on FDA survey research, about 27 percent of consumers surveyed in 2002 reported reading half or more of the brief summary in DTC print ads (Ref. 2). In comparison, when asked how much of the “main” ad they read, about 78 percent reported reading “all” or “almost all” of the main body portion of the ad.

One potential downside to including the same warnings in both the ISI and again in the brief summary is the potential to overwarn consumers. Overwarning is the concept that individuals are exposed to so many warnings in the course of daily life that they are less likely to pay attention to any one particular warning (Ref. 3). In terms of presenting risk information, detailing too many risks may lead consumers to discount all risks, or miss the most important risk information. Similarly, habituation follows when readers see the same warning repeatedly. Upon seeing a particular warning repeatedly, consumers may cease to pay attention to it (Refs. 4-6). Even if a warning has features that make it noticeable, it still has the potential for habituation with repeated exposure (Ref. 5). Although researchers caution against habituation and overwarning, there appears to be limited empirical research in the area of DTC advertising for prescription drugs for the logical supposition that seeing repeated warnings will lead to increased selectivity and reduced attention by recipients over time. Of note, the Office of Prescription Drug Promotion (OPDP) is studying the presentation of risk information in the context of DTC TV ads (“Disclosure Regarding Additional Risks in Direct-to-Consumer Prescription Drug Television Advertisements,” OMB control number 0910-0785).

OPDP plans to investigate, through empirical research, various combinations of the ISI and brief summary. We propose to test two levels of the ISI (short versus long) and the presence of a consumer brief summary (absent versus present) in two different medical conditions (overactive bladder (OAB) and rheumatoid arthritis). The consumer brief summary will follow the draft recommendations for language, readability, content, and format described in “Brief Summary and Adequate Directions for Use: Disclosing Risk Information in Consumer-Directed Print Advertisements and Promotional Labeling for Prescription Drugs: Guidance for Industry, Revised Draft Guidance” (Ref. 7). The “long” ISI is a selection of risks from the brief summary and is typical of what would appear in current DTC ads for each condition. The “short” ISI was created by applying the ideas from recent FDA work on the major statement in broadcast ads (see Refs. 8 and 9). Figures 1 and 2 describe the study design. This will be investigated in DTC print ads for prescription drugs.

This project is designed to use eye-tracking technology. Eye-tracking technology is an effective method to determine the extent to which consumers attend to risk information presented in DTC print ads. This technology allows researchers to unobtrusively detect and measure where a participant looks while viewing a print ad and for how long, and the pattern of their eye movements may indicate attention to and processing of information in the ad.

We plan to collect descriptive eye-tracking data on voluntary participants' attention to the following: (1) The ISI, (2) the brief summary, and (3) the indication and benefit claims. All participants will be 18 years of age or older. We will exclude individuals who are trained as healthcare professionals, employees of the U.S. Department of Health and Human Services (HHS), or who work in pharmaceutical, advertising, or marketing settings because their knowledge and experiences may not reflect those of the typical consumer. We will also exclude individuals who have photosensitive epilepsy; use a medical device that is sensitive to infrared light; or wear various kinds of eyeglasses, hard contact lenses, or colored contact lenses, or have certain vision disorders.

To examine differences between experimental conditions, we will conduct inferential statistical tests such as analysis of variance. With the sample size described in this document, we will have sufficient power to detect small-to-medium sized effects in the main study.

We plan to conduct one 60-minute pilot study with 40 participants and two 60-minute studies with 200 voluntary participants each (50 participants in each cell), for a total of 400 main study voluntary participants. The studies will be conducted in person in at least five different cities across the United States. These locations include Chicago, IL, Tampa, FL, Phoenix, AZ, Houston, TX, and Marlton, NJ. The pilot study and main studies will have the same design and will follow the same procedure. Participants who self-identify as having one of the medical conditions of interest will be randomly assigned to one of four test conditions. In Study 1, the ad will be for a fictitious drug to treat rheumatoid arthritis. In Study 2, the ad will be for a fictitious drug to treat OAB. After obtaining consent, we will explain the study procedure to participants and calibrate the eye-tracking device. To collect eye-tracking data, we will use an unobtrusive glasses-based real-world eye tracker with a minimum speed of 50 hertz. The test images will be presented on paper and sized similarly to how they would appear in print materials such as magazines. To simulate normal ad viewing, participants will view two ads. One of the ads will be the study ad. The non-study ad will be for a consumer product unrelated to health. Only eye-tracking data from the study ad will be analyzed. Next, participants will complete a questionnaire that assesses risk perceptions, risk recall, efficacy perceptions, efficacy recall, and covariates such as demographics and health literacy. In the pilot study, participants will also answer questions as part of a debriefing interview to assess the study design and questionnaire.

In the Federal Register of June 19, 2017 (82 FR 27842), FDA published a 60-day notice requesting public comment on the proposed collection of information. Five public comments were received. Comments received along with our responses to the comments are provided below. For brevity, some public comments are paraphrased and therefore may not reflect the exact language used by the commenter. We assure commenters that the entirety of their comments was considered even if not fully captured by our paraphrasing in this document. The following acronyms are used here: FRN = Federal Register Notice; DTC = direct-to-consumer; FDA and the Agency = Food and Drug Administration; OPDP = FDA's Office of Prescription Drug Promotion.

(Comment 1a, regulations.gov tracking number 1k1-8xet-419m (verbatim)) The research methodology that is outlined here, does not take into consideration prior exposure to ads and the fact that it is known to take about seven exposures to anything before the information sticks. Exposing the respondents to an hour-long eye-tracking research study does not take this into consideration.

(Response) We are not testing long-term retention of information. We are recruiting participants who have the medical condition of interest and may currently be under treatment. Also, Question 21 asks about familiarity with treatments for the targeted condition, which can be used as a covariate in analyses. We do not expect participants to have prior exposure to advertising for the product in the study because the ad is for a fictional product.

(Comment 1b (verbatim)) A sample of 400 is what is considered robust for comparative analysis. Although you will have enough to do some comparison with 200 respondents in each group, it would be better to increase to 400 per group.

(Response) Analysis will be conducted within medical condition. This yields a sample size within each study of 200, which will be used to examine the main effect of length of ISI, the main effect of the presence of a brief summary, and the interaction effects of the two. The sample size of 200 was determined through a power analysis using an alpha level of 0.05, a power of 0.90 and a medium effect size (f = 0.25). The power to detect a medium effect size (f = 0.25) is 0.999 given an alpha of 0.05 if the sample size for each study was increased to 400. The increase in sample size would not substantially improve our ability to detect differences.

(Comment 1c (verbatim, edited for length)) It seems like the research is front loaded to give the answer that the FDA is looking for—give less information to consumers so that they think less about the side effects of the product and buy more product. Consumers should be given all the information to make an informed choice by themselves not determined by what the FDA or other governmental organization feels is what they can handle.

(Response) Please see our responses to Comments 2i and 5a. This research is intended to develop scientific evidence to help inform policy decisions and ensure that our policies related to prescription drug promotion will have the greatest benefit to public health. OPDP seeks to ensure that prescription drug promotional materials provide truthful, balanced and accurately communicated information that helps patients make informed decisions about their treatment options. In each study, the ads will all include the same risk concepts and we will measure comprehension of these risks. We will vary the amount of detail about each risk concept in the ISI section of the ad and we will test the effects of repeating information across the ISI and the consumer brief summary.

(Comment 2a, regulations.gov tracking number 1k1-8xz7-z732 (verbatim)) Do the exclusion criteria adequately account for all potential subjects that have vision impairments that can affect how their eyes move as they read? Additional exclusions may be needed to address these (e.g. blindness in one eye, artificial eye, etc.).

(Response) The study design currently calls for excluding potential participants with vision impairments that interfere with the capabilities of the eye-tracking glasses. This includes wearing regular glasses, bifocals, trifocals, progressive lenses, hard contact lenses, and colored contact lenses. We will also add exclusion criteria for potential participants who have cataracts, amblyopia (lazy eye/blind in one eye), strabismus (cross-eyed), mydriasis (permanent pupil dilation), nystagmus (involuntary eye movements), an ocular prosthesis (glass eye), and who are designated as legally blind.

(Comment 2b (verbatim)) Consider adding an arm to the design that shows an ad without any specific risk content or a brief summary, but alternatively consists of a statement that informs a potential patient that the drug in question has risks, including serious risks, associated with its use, and that it is very important that a patient talk with his/her doctor about these risks, prior to use, to determine if the drug is appropriate for the patient. It would be interesting to see what type of recall and what type of eye movement data would occur for this type of statement.

(Response) FDA regulations state that prescription drug advertisements must contain “a true statement of information in brief summary relating to side effects, contraindications (. . .[to] include side effects, warnings, precautions, and contraindications and include any such information under such headings as cautions, special considerations, important notes, etc.) and effectiveness” (§ 202.1(e)(1)). Additionally, advertisements must also “present a fair balance between information relating to side effects and contraindications and . . . effectiveness. . . .” (§ 202.1(e)(5)(ii)). We decline the suggestion to test the proposed statement at this time.

(Comment 2c (verbatim)) Question 1: The relevance of asking a subject to assess how many risks are presented in comparison to how many benefits is not apparent. We recommend that FDA consider deleting the question or alternatively rewording it to get data on how many risks the subjects think are presented in the ad. Response options should be quantitative, such as: No risks, 1-3 risks, 4-6 risks, >6 risks.

(Response) The purpose of Question 1 is to assess participants' initial impressions of balance of risks versus benefits in the ad. Additionally, Question 4 has been revised based on the results of cognitive testing to collect risks that participants can recall. This provides both a quantitative measure and an accuracy evaluation. We believe this approach will yield richer data as far as how many risks the participant recalls from the ad.

(Comment 2d (verbatim)) Question 4: If subjects are going to be asked to recall, using free text, the risks presented in the ad, it would similarly be interesting to add a similar question to recall, using free text, which benefits were presented in the ad.

(Response) The questionnaire contains several questions about benefit/efficacy (Questions 3, 10, and 11). We also have questions that measure the perceived risk/benefit tradeoff (Questions 1, 18, and 19). Although it would be interesting from a conceptual standpoint to include an open-ended recall question about product benefits, our focus in this study is on the risk information. Further, we are concerned about adding length to the questionnaire as we have worked to minimize the burden of the collection of information on respondents.

(Comment 2e (verbatim)) Questions 8, 10, and 11: Suggest rewording the questions so that they describe the likelihood that a person taking the drug experiences a side effect or a benefit.

(Response) The items used in this section were developed through scale validation research. Thus, we prefer to retain them in their original form.

(Comment 2f (verbatim)): Questions 12-15: It may be confusing for the reader to discern differences between the terms “main ad”, “page following the main ad”, and “advertisement”. These terms might need to be accompanied by further explanatory text.

(Response) Cognitive testing revealed participants did have difficulty discerning the differences in the ad components based on the descriptive terms provided. To address this problem and help with data quality, thumbnail images will be provided next to Questions 13-15, so that participants will have a visual cue of what portion of the ad the question is asking about without allowing them to re-read the ad stimulus.

(Comment 2g (verbatim)) Questions 16 and 17: Randomize the order in which the personal involvement adjectives/tasks are presented to minimize bias.

(Response) Question 16 is The Personal Involvement Inventory, a validated measure with high internal consistency (coefficient α = .88) and has been used in prior studies to provide useful information about personal relevance (Refs. 10 and 11). The author of the inventory confirmed that it was developed and has been administered without randomization of these items. For the current study, values across items will be averaged in order to produce an overall personal involvement score for comparison across participants. Since this question is a validated measure and will be used only as a moderator variable, the item order will not change. Question 17 is a measure of self-efficacy, which will serve as an additional outcome of interest. We will randomize Question 17.

(Comment 2h (verbatim)) Question 18: It is not clear what the term “leave” means. It may mean “take time off from work.” Please clarify.

(Response) Question 18 was developed through scale validation research. “Leave” does in fact mean “take time off from work.” We did not encounter any confusion on the part of respondents during cognitive testing of the questionnaire. We prefer to retain this question in its original form.

(Comment 2i (verbatim)) Question 19: A consumer should not be expected to make a risk/benefit assessment of a drug simply by reading an ad. Such an assessment can occur only after a patient has had a discussion with his/her healthcare provider. Thus, we suggest deletion of this question.

(Response) An important purpose of communicating the drug's specific risk and benefit information in DTC advertising is to position consumers as active and well-informed participants in their health care decisionmaking. FDA seeks to improve our understanding of what baseline judgements about product risks and benefits individuals make on the basis of advertising. Question 19 does not indicate that FDA expects that the advertisement will be the sole basis for individuals to assess benefit and risk or make ultimate healthcare decisions. Rather, Question 19, which was developed through scale validation research, measures one aspect of the consumer's perception of the drug's risk-benefit tradeoff. Further, we did not encounter any confusion on the part of respondents during cognitive testing of the questionnaire.

(Comment 2j (verbatim)) Questions 28-33: We note these questions assess the ability of the respondent to answer questions using an ice cream nutrition facts label. We assume the inclusion of these questions is to assess how well respondents are capable of comprehending complex numeric information. However, we note that some respondents may not be able to comprehend and apply numeric information or be motivated to do so, regardless of how it appears. The format may not matter when this is the case. Therefore, we suggest that FDA consider analyzing results based on those who can vs. cannot answer the ice cream questions. Alternatively, the ice cream questions could be used at the start of the survey to screen out those who are unable to answer the questions, thereby further focusing the sample on persons who are able to comprehend numeric presentations likely to be found in drug promotion.

(Response) Questions 28-33 make up the Newest Vital Sign (NVS), developed by Pfizer. (See https://www.pfizer.com/​health/​literacy/​public-policy-researchers/​nvs-toolkit). The NVS is a valid and reliable measure of health literacy and numeracy that was used and recommended by two studies (Refs. 12 and 13). In this study, the NVS will be used as a covariate that measures risk of low health literacy/numeracy. It is important that potential participants of various health literacy levels are included, because level of health literacy/numeracy of the individual has been shown to play a particularly strong role in viewing and processing health information (Ref. 14).

For the stated reasons, no change to the analysis or use of the questions to filter the sample of participants is planned.

(Comment 3a, regulations.gov tracking number 1k1-8y5u-ecif (verbatim)) One omitted variable in the study design is recall after viewing the ad and ISI/brief summary. It would seem potential negative effects of overwarning and habituation would be even more apparent after a lapse of time. The commenter suggests incorporating a parameter to capture this, for example, including a re-contact option to test recall and interpretation after a period of 2-4 days. For this recall option, we suggest that a quota of ~ 30 respondents per cell in order to ensure a robust sample for statistical testing.

(Response) Question 4 captures open-ended recall of risks and negative effects. The comment proposes an interesting research idea. However, testing long-term retention of information is beyond the scope of this study.

(Comment 3b (summarized)) The commenter suggests ensuring a representative sample of respondents with the conditions of interest is collected (~ 30 per cell). Analysis of these respondents compared to those without the conditions would act as a control.

(Response) The study design calls for only including individuals who have the medical condition targeted for each study. This is based on the rationale that, relative to the general population, individuals who suffer from a specific medical condition pay more attention to DTC ads related to that medical condition (Refs. 15-17). Thus, we do not plan to add a general population sample.

(Comment 3c (verbatim)) Neither the full stimuli nor specific examples of the disclosure language were provided. The lack of access to these makes full interpretation of the study objectives difficult as well as leaves us unable to provide suggestions or comments on the stimuli to be tested.

(Response) We have described the purpose of the study, the design, the population of interest, and have provided the questionnaire to numerous individuals upon request. The brief summary for each ad contains a summary of the product risks, side effects, and contraindications. The “long” ISI is a selection of risks from the brief summary and is typical of what would appear in current DTC ads for each condition. The “short” ISI was created by applying the ideas from recent FDA work on the major statement in broadcast ads (see Refs. 16 and 17). Our full stimuli are under development during the PRA process. We do not make draft stimuli public during this time because of concerns that this may contaminate our participant pool and compromise the research.

(Comment 3d (summarized)) The commenter suggests that the data and information collected with eye-tracking be used as secondary evidence of attention. This is due to both difficulty of interpretation inherent in eye-tracking data along with subjectivity introduced by the ad copy stimuli under examination, as stimuli can be manipulated to increase/decrease attractiveness to a respondents' eye. The commenter believes these limitations make use of this data to direct policy difficult. Additionally, the briefing document does not expand upon exactly how the eye-tracking data will be analyzed other than tracking attention. There are various ways to analyze eye-tracking data, such as order of attention, number of multiple viewings, and possibly pupil dilation as a measure of attention. The commenter has traditionally added qualitative elements to its use of eye-tracking technology in research, by discussing what the respondent saw after viewing the stimuli and even reviewing a respondents' eye-tracking map with them to get further insights.

(Response) To clarify, two types of data will be collected in each study. Both data types are considered useful evidence. Self-report measures will be collected via a web-based questionnaire, and physical measures of attention will be collected via eye-tracking glasses. Existing research has relied on self-report measures to determine how much and what parts of the risk and benefit information consumers are reading. Because of the known unreliability of self-report measures (Ref. 18), research is needed to accurately determine what and how much consumers are reading when they see risk and benefit statements in prescription drug ads.

During the debriefings for the pilot study, respondents will be shown their eye-gaze data and asked to comment on the elements of the stimuli they attended to, the elements they did not attend to, and why. These data in aggregate form will be reviewed to determine whether to modify the stimuli prior to the main studies. Eye-tracking data (both heat maps and gaze plots) will be used in the analyses to identify general patterns across participants and to investigate how those relate to questionnaire measures.

(Comment 3e (verbatim)) The FRN states the location of risk information is also an objective of the study. The commenter assumes this “location” testing will be via testing risk information communicated in stimuli having the ISI plus the Brief Summary against stimuli having the ISI only. If this is inaccurate, then we are not sure the study design as described in the FRN adequately tests for a variable of “location.” If varying location of risk information beyond ISI versus ISI + Brief Summary is desired, the commenter suggests this be tested in a subsequent study or that the proposed study better specify variation of “location.”

(Response) The commenter has correctly interpreted the study design. We are not manipulating where the information appears on the page. Location, as used here, refers to the presence of information in both the brief summary and the ISI, or just the ISI. Within each medical condition, we have endeavored to maintain consistency of where the information appears on the page, and the order of the information, across experimental conditions.

(Comment 3f (summarized)) Through the survey, the commenter suggests maintaining a single scale for all rating questions. For example, the commenter generally employs a 5-point scale, which includes a midpoint, and is defined at each point. In the current questionnaire, the scales switch from 5-point to 6-point scales which could cause confusion among some respondents. If the 6-point scales are included explicitly to omit a neutral mid-point, the commenter suggests that each of the points are defined to ensure that respondents know what the point on the scale they are choosing means (similarly to what is provided in Question 20 onwards).

(Response) Many of the items used in the survey were developed through scale validation research (i.e., Questions 8-11, 18, and 19). These items utilize a six-point scale, so we have attempted to use six-point scales where possible. In other cases, however, we are using items that have been used in prior FDA studies (i.e., Questions 1 and 24) or are established measurement inventories (Question 16 is the Personal Involvement Inventory; Ref. 11). Changing the scale range or altering the scale to add definitions to each scale point would preclude comparison with prior study results. Thus, we prefer to maintain the scale ranges currently in use.

(Comment 3g (summarized)) For Questions 8-11, the commenter suggests adding a “Don't know” option as respondents might not be able to assess likelihood of side effects, seriousness of side effects, efficacy, and potential improvement based on the information presented in the ad. The current range of answer choices may force inaccurate or speculative responses; a “Don't Know” answer would be a legitimate choice and informative for the study. The commenter's standard practice is to provide a “Don't Know” option whenever it could be a valid answer.

(Response) The items used in this section were developed through scale validation research. Thus, we prefer to retain them in their original form, for this study, though we will consider this for future measurement studies.

(Comment 3h (verbatim)) For Question 12, without ability to review the stimuli, it is unclear what content will appear in Area A, B, C and D. It is also unclear whether the content will be the same across all 4 stimuli ads or whether content will change location in the ad.

(Response) We have endeavored to maintain consistency of information location across conditions. Area A is the part of the ad with a picture. Areas B, C, and D are all sections of the ISI.

(Comment 3i (summarized)) The commenter wonders what the utility of asking Question 16 is as the question appears to be out of scope with the objectives of the study. Whether or not the ad is important, boring, or relevant to the respondent seems irrelevant to the stated goals. We suggest removing the question.

(Response) Please see our response to Comment 2g.

(Comment 3j (summarized) In Question 18, the inclusion of “. . . outweigh all the things I have to do to obtain it (appointments, prescriptions, leave)” seems out of scope when considering the objectives of the study. The commenter suggests removing the question.

(Response) This question measures one aspect of product benefits, the benefit-inconvenience tradeoff, which is an important component of drug product perceptions. Additionally, please see our response to Comment 2h.

(Comment 3k (summarized)) For Question 19, the commenter suggests a minor adjustment to the wording. Instead of saying “The benefits of [DRUG NAME] outweigh any side effects it may have”, the commenter suggests saying “. . . any side effects it is described/indicated as having”. “May have” could be interpreted subjectively by respondents to include side effects not in the ISI and brief summary.

(Response) Question 19 is a validated question so it will be retained as is. Cognitive testing revealed no comprehension or reporting issues for this question.

(Comment 3l (verbatim)) For Questions 22-23 pertaining to respondent perception of condition. There does not appear to be any skip logic to ensure that only those with one of the specified conditions can answer those questions. These questions should not be asked of those who do not suffer from one of the specified conditions.

(Response) We intend to recruit individuals who self-identify as having either OAB or rheumatoid arthritis. Those individuals will be assigned to view an ad that treats their medical condition. The questionnaire will contain questions relevant to that medical condition only.

(Comment 4a, regulations.gov tracking number 1k1-8y4d-os71 (summarized)) The commenter recommends that greater emphasis be placed on the recall/questionnaire metric rather than the eye-tracking metric. The eye-tracking data will determine if there is indeed a direct correlation between the length (amount) of the risk information and length of time spent looking at that information; however, it will not differentiate between what content and format is more effective for communicating that risk information. The commenter suggests that FDA include in the questionnaire (and/or debriefing interview) specific inquiries regarding the repetitiveness of the risk information in order to further explore the link between the amount and placement of risk information and the ultimate recall of this information.

(Response) Please see our response to Comment 3d. In addition, we will add a question regarding repetitiveness to the questionnaire.

(Comment 4b (summarized)) The commenter believes it is important that the fictitious drugs in this study have safety profiles reflecting the complex safety profiles of actual, currently-approved and promoted products.

(Response) The DTC ads to be used in this research were developed using actual ads for these medical conditions. Additionally, we consulted with expert reviewers in OPDP on content and format to ensure the stimuli are realistic.

(Comment 4c (summarized)) The “short” versus “long” ISI should be defined explicitly. The commenter believes it is critical to know the specific ISI content (“short” and “long”) in order to fully understand the study results. Additionally, OPDP examples of adequate “short” and “long” ISIs used in the context of print ads would be valuable templates for industry, especially given the lack of consensus in acceptable utilization of “short” iterations of ISI as observed in past OPDP advisory comments and Warning Letters.

(Response) Please see our responses to comments 1c and 3c. This study is not intended to provide specific guidelines on what content should be included in the ISI.

(Comment 4d (summarized)) The commenter proposes the content of the brief summary be stated as well so as to understand what risk information is repeated from the ISI and what impact this may have on the study results.

(Response) We have described how the consumer brief summary will be constructed in the Background section. Please see our responses to Comment 1c and 3c.

(Comment 4e (summarized)) The commenter questions the utility of including the control, consumer product ad if the eye-tracking data is not utilized. FDA should clarify if the questionnaire will assess the recall of the control ad. The commenter recommends FDA fully evaluate the data from the control ad in order to provide appropriate context for the results obtained from the study, health-related ads.

(Response) The purpose of participants viewing the consumer product ad, otherwise known as the warm-up ad, is to orient them to the ad-viewing task. In addition, the warm-up ad permits the research team to do an initial review and adjustment of the eye-tracking equipment as needed before the study task begins. Therefore, there is no plan to analyze the warm-up ad data as it is not relevant to the focus of the study, and is mainly a procedure to orient the participant to the eye-tracking task.

(Comment 5a, regulations.gov tracking number 1k1-8y60-6g3m (summarized)) The commenter is concerned with the Agency's recent approaches to studies in this area. FDA has proposed to undertake projects in a variety of disparate topics without articulating a clear, overarching research agenda or adequate rationales on how the proposed research related to the goal of further protecting public health. Within the last year, the Agency has increased such efforts at an exponential pace. At times, FDA proposes new studies seemingly without fully appreciating its own previous research published on the OPDP website. Proposed studies are often unnecessary in light of existing data. The commenter suggests that the Agency publish a comprehensive list of its prescription drug advertising and promotion studies from the past 5 years and articulate a clear vision for its research priorities for the near future. Going forward, FDA should use such priorities to explain the necessity and utility of its proposed research and should provide a reasonable rationale for the proposed research.

(Response) OPDP's mission is to protect the public health by helping to ensure that prescription drug information is truthful, balanced, and accurately communicated, so that patients and health care providers can make informed decisions about treatment options. OPDP's research program supports this mission by providing scientific evidence to help ensure that our policies related to prescription drug promotion will have the greatest benefit to public health. Toward that end, we have consistently conducted research to evaluate the aspects of prescription drug promotion that we believe are most central to our mission, focusing in particular on three main topic areas: Advertising features, including content and format; target populations; and research quality. Through the evaluation of advertising features we assess how elements such as graphics, format, and disease and product characteristics impact the communication and understanding of prescription drug risks and benefits; focusing on target populations allows us to evaluate how understanding of prescription drug risks and benefits may vary as a function of audience; and our focus on research quality aims at maximizing the quality of research data through analytical methodology development and investigation of sampling and response issues.

Because we recognize the strength of data and the confidence in the robust nature of the findings is improved through the results of multiple converging studies, we continue to develop evidence to inform our thinking. We evaluate the results from our studies within the broader context of research and findings from other sources, and this larger body of knowledge collectively informs our policies as well as our research program. Our research is documented on our homepage, which can be found at: https://www.fda.gov/​aboutfda/​centersoffices/​officeofmedicalproductsandtobacco/​cder/​ucm090276.htm. The website includes links to the latest FRNs and peer-reviewed publications produced by our office. The website maintains information on studies we have conducted, dating back to a survey of DTC attitudes and behaviors conducted in 1999.

(Comment 5b (The commenter provided a summary of their comments followed by a more detailed description of the same comments. For brevity, the summary of comments has been omitted and only the specific comments [5b through 5t] are provided below. The commenter's full comments may be accessed at regulations.gov via tracking number 1k1-8y13-m7td) (summarized)) The PRA Notice states there has been little empirical research for the logical supposition that seeing repeated warnings will lead to increased selectivity and reduced attention. This is not correct. As some authors have commented, “[h]abituation has been found in a variety [of] contexts and domains.” The commenter is aware of at least three empirical research studies, none cited in the PRA Notice, that demonstrate the “habituation effect is a robust phenomenon.” This effect has been documented in “studies involving different contexts and response measures.”

(Response) We thank the commenter for pointing out this mischaracterization. We have revised our introduction to clarify that whereas there is an overall body of research relating to habituation, there is limited, if any, research on habituation in the specific context of DTC print advertising for prescription drugs.

(Comment 5c (summarized)) FDA should clarify whether the proposed study will adopt the brief summary format outlined in “Guidance for Industry—Brief Summary and Adequate Directions for Use: Disclosing Risk Information in Consumer-Directed Print Advertisements and Promotional Labeling for Prescription Drugs” (Draft Guidance).

(Response) We plan to utilize the Question and Answer consumer-friendly format described in the referenced draft guidance.

(Comment 5d (summarized)) The commenter requests that the Agency make available for public comment the study stimuli, including the non-study ad for a consumer product unrelated to health. In particular, the commenter wishes to provide comments on: (1) What constitutes “short” and “long” length for the ISI and (2) the content, format, and design of the Brief Summary.

(Response) Please see our responses to Comments 1c, 3c, 4c, and 4e.

(Comment 5e (summarized)) The Agency proposes to use eye tracking technology “to determine how risk presentations in DTC print ads are perceived.” The commenter encourages the Agency to use this technology in conjunction with other inputs (for example, qualitative research) to understand why subjects are looking at a portion of the proposed materials, rather than to draw conclusions that such portions were viewed. Additionally, an explanation of the use of eye tracking technology should also be included during the subject enrollment process.

(Response) FDA plans to collect and analyze eye-tracking (physical measures of attention) data in conjunction with other measures, including self-report measures of attention, recall, and comprehension. The recall measures will be collected via qualitative (open-ended) questions. To avoid the potential for priming effects, the goals of the eye-tracking component of the study will not be explained to recruited individuals before they report for their in-person sessions. However, participants will be made aware of the eye-tracking component during the informed consent process. Please also see our response to Comment 3d.

(Comment 5f (summarized)) Recall Questions. FDA should capture whether subjects comprehend that there are side effects and negative outcomes, even if the subject does not recall information on the specifics. The commenter suggests adding a question concerning whether subjects were aided in the recall of information by the “short” or “long” ISI format.

(Response) Questions 4a-c capture recall of risk in an open-ended format. Our approach involves random assignment to experimental conditions; each participant will see only one version of the stimuli. Because participants will not be aware there is another, different format, asking them their impressions of the long versus the short format is not feasible.

(Comment 5g (verbatim)) Recall questions (e.g., Question 4) ask test subjects to identify specific side effects and negative outcomes of the featured drug products. It is not clear why such questions are necessary for the research purpose of the study.

(Response) An important purpose of communicating the drug's specific risk and benefit information in DTC advertising is to position consumers as active and well-informed participants in their health care decision-making. In this study, we are investigating how different presentations of risk information impact perception and comprehension of drug risks and benefits. These questions are designed to provide information to help us identify effective ways to communicate risk and benefit information in DTC advertising. See our response to Comment 2b for additional context.

(Comment 5h (verbatim)) The questionnaires do not define certain key terms (e.g., risk, side effect). Subjects may interpret these terms based on different standards. FDA might consider providing user-friendly definitions.

(Response) We appreciate the importance of ensuring uniform interpretation of terms. In cognitive interviews preceding this work, we assessed whether individuals interpret key terms similarly and made revisions where necessary. We have also considered the additional time (burden) that would be required to complete the survey if every term were defined in the pilot and main study. With these factors in mind, we have chosen not to provide additional definitions.

(Comment 5i (summarized)) The commenter recommends that: (1) FDA replace the phrase “negative outcomes” with “risks and warnings” and (2) insert “possible” before the phrase “side effects.”

(Response) We have deleted “negative outcomes” from the question wording in Question 2 and Question 4b. Also, please see our response to Comment 3g concerning the proposal to reword the previously validated question.

(Comment 5j (verbatim)) The Agency should consider changing the sliding scale to an odd number system to permit a “neutral” response. Most questions (e.g., Questions 2-3, Questions 8-11) provide six choices, not permitting a neutral response.

(Response) Please see our response to Comment 3f.

(Comment 5k (verbatim)) FDA should reconsider the inclusion of the perceived efficacy likelihood (Question 10) and perceived efficacy magnitude (Question 11) questions. It is not apparent what utility these specific questions have in the context of the study.

(Response) We note that this comment is the opposite of Comment 2d, which suggests adding recall questions about product benefits. Although the main focus of this research is on the risk information, an important purpose of communicating the drug's specific risk and benefit information in DTC advertising is to position consumers as active and well-informed participants in health care decision-making. These questions will allow us to assess the impact of our study variables on perception and comprehension of drug benefits.

(Comment 5l (summarized)) The commenter supports a study design that includes an analysis of whether the inclusion of the brief summary, along with a short or long ISI, presents duplicative information to the user, and therefore, introduces overwarning.

(Response) We thank the commenter for their support of research. We reiterate that the purpose of the study is to examine how various means of presenting risk information impact consumer comprehension and perceptions of product information.

(Comment 5m (verbatim)) FDA states that it will conduct the studies in person in at least five different cities across the United States. The Agency should address what efforts it will take to avoid enrichment of the sample population when selecting cities.

(Response) We interpret the commenter's request for FDA to address how it will “avoid enrichment of the sample population when selecting cities” to mean that FDA should address how it will avoid collecting data in cities where the medical conditions are more prevalent than in other cities. This is not the aim of collecting data in five different cities. Rather, the cities have been selected to represent metropolitan areas in various geographic areas of the United States, including the West, Southwest, Midwest, Southeast, and the mid-Atlantic. These locations include Chicago, IL, Tampa, FL, Phoenix, AZ, Houston, TX, and Marlton, NJ. Due to the low population prevalence rate of the two medical conditions and the need to conduct sessions with 40 individuals with the condition in each of 5 areas, testing in rural areas is not feasible.

(Comment 5n (summarized)) Study participants diagnosed with one of the medical conditions of interest may be more prone to pay attention and read information concerning prescription drugs for these conditions. Additionally, the study setting may prompt participants to pay closer attention to stimuli. FDA should clarify how it plans to limit such response biases.

(Response) The study method randomly assigns each participant to an experimental condition, ensuring that potential pre-existing biases will be evenly distributed across the conditions. The only aspect of the participants' experiences that will be varied in the study will be the manipulations that we have described. Thus, given the experimental design of the study, if we find differences between and among conditions, we can be reasonably sure that the manipulations caused the differences. Similarly, any individual differences in attention or ability should be spread across experimental conditions. We have not found in the past that our participants spend an inordinate amount of time viewing stimuli, but we will be careful to place the research in context when we interpret the data.

(Comment 5o (verbatim)) An “FDA employee” category, similar to S6 and S7, should be added to the Screener Survey. These individuals should also be terminated from the study.

(Response) We have added a category to exclude employees of HHS, which includes employees of FDA.

(Comment 5p (verbatim)) S2 and S3 of the Screener Survey should be rewritten as follows: “Has a doctor or other health care professional ever diagnosed you with overactive bladder (OAB)?”

“Has a doctor or other health care professional ever diagnosed you with rheumatoid arthritis (RA)?”

(Response) We will leave the wording of the screener questions S2 and S3 as-is. Cognitive testing results in various contexts have indicated comprehension and reporting errors associated with using the more formal phrase “. . . diagnosed you with . . . [condition].” Common practice is to use the wording “. . . ever told you . . . .”

(Comment 5q (verbatim)) Question 16 of the Questionnaire and P1 of the Pilot Study should be deleted. Whether a subject considers the study stimuli to be “Exciting/Unexciting” or “Boring/Interesting” or whether the subject “likes” the study stimuli has no apparent relevance to FDA's study goals.

(Response) Please see our response to Comment 2g.

(Comment 5r (verbatim)) Questions 12-17 should be the first questions of the Questionnaire. A subject will likely answer these questions most accurately immediately after reviewing the study stimuli and before answering other questions that could influence these answers.

(Response) FDA agrees that it is important to position certain questions where they will be answered in close proximity to the ad-viewing time, which may improve reporting accuracy. However, the decision was to place the questions that assess recall and recognition of risks (Questions 4-7) earliest in the question sequence, so as to minimize memory decay and contamination of responses by exposure to questions covering other constructs (risk likelihood, risk magnitude). The attention (Question 12) and ad reading (Questions 13-15) measures will be retained in their current order (in the first half of the questionnaire).

(Comment 5s (verbatim)) Question 18 should include considerations for prescription drug access.

(Response) Please see our response to Comment 2h.

(Comment 5t (summarized)) It is unclear how FDA plans to utilize the non-study ad (related to ice cream). Questions 27-32 appear very different in nature, substance, purpose, format, and length than the questions concerning the drug ad. FDA should clearly articulate the purpose of this stimulus and how it will be used in analyzing study results (if at all). If the sole purpose is to “stimulate normal ad viewing,” the commenter encourages adding another one to two non-study ads.

(Response) The comment suggests that the nutrition facts label was interpreted as the “non-study ad.” That is not the case. The ice cream nutrition facts label and accompanying questions (Questions 27-33) are included in the questionnaire as skills-based measures of health literacy and numeracy and have been adapted for self-administration in these studies. Please see our response to Comment 2j.

FDA estimates the burden of this collection of information as follows:

Table 1—Estimated Annual Reporting Burden ¹

Activity	Number of respondents	Number of responses per respondent	Total annual responses	Average burden per response	Total hours
Pilot Screener	120	1	120	.03 (2 minutes)	4
Study 1 Screener	600	1	600	.03 (2 minutes)	18
Study 2 Screener	600	1	600	.03 (2 minutes)	18
Completes, Pilot	40	1	40	1	40
Completes, Study 1	200	1	200	1	200
Completes, Study 2	200	1	200	1	200
Total					480
1 There are no capital costs or operating and maintenance costs associated with this collection of information.

II. References

The following references are on display with the Dockets Management Staff (HFA-305), Food and Drug Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852 and are available for viewing by interested persons between 9 a.m. and 4 p.m., Monday through Friday; they are also available electronically at https://www.regulations.gov. FDA has verified the website addresses, as of the date this document publishes in the Federal Register, but websites are subject to change over time.

1. McGuire, L.C., “Remembering What the Doctor Said: Organization and Older Adults' Memory for Medical Information.” Experimental Aging Research, vol. 22, pp. 403-428, 1996.

2. Aikin, K.J., J.L. Swasy, and A.C. Braman, “Patient and Physician Attitudes and Behaviors Associated with DTC Promotion of Prescription Drugs: Summary of FDA Survey Research Results” (2004). Available at https://www.fda.gov/​AboutFDA/​CentersOffices/​OfficeofMedicalProductsandTobacco/​CDER/​ucm151498.htm.

3. Warnings and Risk Communication (2005). Wogalter, M.S., D.M. DeJoy, and K.R. Laughery (Eds.). Philadelphia: Taylor & Francis, Inc.

4. Conzola, V.C. and M.S. Wogalter, “A Communication-Human Information Processing (C-HIP) Approach to Warning Effectiveness in the Workplace.” Journal of Risk Research, vol. 4(4), pp. 309-322, 2001.

5. Wogalter, M.S. and K.R. Laughery, “Warning! Sign and Label Effectiveness.” Current Directions in Psychological Science, vol. 5(2), pp. 33-37, 1996.

6. Wogalter, M.S., T.L. Smith-Jackson, B.J. Mills, and C.S. Paine, “The Effects of Print Format in Direct-to-Consumer Prescription Drug Advertisements on Risk Knowledge and Preference.” Drug Information Journal, vol. 36(3), pp. 693-705, 2002.

7. Brief Summary and Adequate Directions for Use: Disclosing Risk Information in Consumer-Directed Print Advertisements and Promotional Labeling for Prescription Drugs. Revised Draft Guidance. Available at https://www.fda.gov/​downloads/​Drugs/​GuidanceComplianceRegulatoryInformation/​Guidances/​UCM069984.pdf.

8. “Content of Risk Information in the Major Statement in Prescription Drug Direct-to-Consumer Broadcast Advertisements; Establishment of a Public Docket; Request for Information and Comments.” August 21, 2017, 82 FR 39598.

9. Betts, Kevin R., et al., “Serious and Actionable Risks, Plus Disclosure: Investigating an Alternative Approach for Presenting Risk Information in Prescription Drug Television Advertisements.” Research in Social and Administrative Pharmacy, 2017.

10. Bhutada, N.S., B.L. Rollins, and M. Perri III, “Impact of Animated Spokes-Characters in Print Direct-to-Consumer Prescription Drug Advertising: An Elaboration Likelihood Model Approach.” Health Communication, vol. 32, pp. 391-400, 2017.

11. Zaichkowsky, J.L., “The Personal Involvement inventory: Reduction, Revision, and Application to Advertising.” Journal of Advertising, vol. 23, pp. 59-70, 1994.

12. Mackert, M., S.E. Champlin, K.E. Pasch, and B.D. Weiss, “Understanding Health Literacy Measurement Through Eye Tracking.” Journal of Health Communication, vol. 18, pp. 185-196, 2013.

13. Chiang, K.P. and A. Jackson, “The Impact of Health Literacy on Involvement and Attitude Toward Pharmaceutical Print Ads.” International Journal of Healthcare Management, vol. 9(1), pp. 47-57, 2016.

14. An, S. and N. Muturi, “Subjective Health Literacy and Older Adults' Assessment of Direct-to Consumer Prescription Drug Ads.” Journal of Health Communication, vol. 16(3), pp. 242-255, 2011.

15. Ball, J.G., D. Manika, and P. Stout, “Consumers Young and Old: Segmenting the Target Markets for Direct-to-Consumer Prescription Drug Advertising.” Health Marketing Quarterly, vol. 28(4), pp. 337-353, 2011.

16. Christensen, T.P., F.J. Ascione, and R.P. Bagozzi, “Understanding How Elderly Patients Process Drug Information: A Test of a Theory of Information Processing.” Pharmaceutical Research, vol. 14, pp. 1589-1596, 1997.

17. Mehta, A. and S.C. Purvis, “Consumer Response to Print Prescription Drug Advertising.” Journal of Advertising Research, vol. 43(2), pp. 194-206, 2003.

18. Paulhus, D.L. and S. Vazire, “The Self-Report Method.” Handbook of Research Methods in Personality Psychology, vol. 1, pp. 224-239, 2007.

Dated: August 8, 2018.

Leslie Kux,

Associate Commissioner for Policy.