E9-31200. Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Experimental Study: Presentation of Quantitative Effectiveness Information to Consumers in Direct-to-Consumer Television and ...

AGENCY:

Food and Drug Administration, HHS.

ACTION:

Notice.

SUMMARY:

The Food and Drug Administration (FDA) is announcing that a proposed collection of information has been submitted to the Office of Management and Budget (OMB) for review and clearance under the Paperwork Reduction Act of 1995.

DATES:

Fax written comments on the collection of information by February 4, 2010.

ADDRESSES:

To ensure that comments on the information collection are received, OMB recommends that written comments be faxed to the Office of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer, FAX: [phone], or e-mailed to [e-mail]. All comments should be identified with the OMB control number 0910-New and title Experimental Study: Presentation of Quantitative Effectiveness Information to Consumers in Direct-to-Consumer (DTC) Television and Print Advertisements for Prescription Drugs. Also include the FDA docket number found in brackets in the heading of this document.

FOR FURTHER INFORMATION CONTACT:

Liz Berbakos, Office of Information Management (HFA-710), Food and Drug Administration, 5600 Fishers Lane, Rockville, MD 20857, 301-796-3792, [e-mail].

SUPPLEMENTARY INFORMATION:

In compliance with 44 U.S.C. 3507, FDA has submitted the following proposed collection of information to OMB for review and clearance.

Experimental Study: Presentation of Quantitative Effectiveness Information to Consumers in Direct-to-Consumer (DTC) Television and Print Advertisements for Prescription Drugs—(OMB Control Number 0910-New)

I. Background

The Federal Food, Drug, and Cosmetic Act (the act) requires that manufacturers, packers, and distributors (sponsors) who advertise prescription human and animal drugs, including biological products for humans, disclose in advertisements certain information about the advertised product's uses and risks.^[1] By its nature, the presentation of this information is likely to evoke active trade-offs by consumers, i.e., comparisons with the perceived risks of not taking treatment, and comparisons with the perceived benefits of taking a treatment (Ref. 1). FDA has an interest in fostering safe and proper use of prescription drugs, an activity that engages both risks and benefits. Therefore, an examination of ways to improve consumers' understanding of this information is central to this regulatory task.

Under the act, FDA engages in a variety of communication activities to ensure that patients and health care providers have the information they need to make informed decisions about treatment options, including the use of prescription drugs. FDA regulations (21 CFR 201.57) describe the content of required product labeling, and FDA reviewers ensure that labeling contains accurate and complete information about the known risks and benefits of each drug.

FDA regulations require that prescription drug advertisements that make (promotional) claims about a product also include risk information in a “balanced” manner (21 CFR 202.1(e)(5)(ii)), both in terms of the content and presentation of the information. This balance applies to both the front, display page of an advertisement, as well as including information “in brief summary” about the advertised product's “side effects, contraindications, and effectiveness”^[2] usually, but not always, on a separate page. However, beyond the “balance” requirement there is limited guidance and research to direct or encourage sponsors to present benefit claims that are informative, specific, and reflect clinical effectiveness data.

FDA has recently provided guidance to sponsors about ways to present risk information in prescription drug advertisements (Ref. 2). This guidance notwithstanding, research addressing specifically how to present benefit and efficacy information in prescription drug advertisements is limited. For example, “benefit claims,” broadly defined, appearing in advertisements are often presented in general language that does not inform patients of the likelihood of efficacy and are often simply variants of an “intended use” statement. One content analysis of DTC advertising by Woloshin and Schwartz (2001) (Ref. 3) found that information about product benefits and risks is often presented in an unbalanced fashion. The researchers classified the “promotional techniques” used in the advertisements. Emotional appeals were observed in 67 percent of the ads while vague and qualitative benefit terminology was found in 87 percent of the ads. Only 9 percent contained data. However, for risk information, half the advertisements used data to describe side-effects, typically with lists of side-effects that generally occurred infrequently. Similarly, a content analysis by Frosch et al. (2007) (Ref. 4) found that only a small proportion of product-claim ads gave specific information about the population prevalence of the medical condition being advertised. The authors criticize DTC for presenting “best-case scenarios that can distort and inflate consumers' expectations about what prescription drugs can accomplish” (see p. 12 of Frosch et al.) (Ref. 4) without disclosing how many consumers are likely to experience that benefit.

Some research has proposed that providing quantitative information about product efficacy enables consumers to make better choices about potential therapy. One possible format (termed the “drug facts” box by its creators) for this information has recently received attention (Refs. 5, 6, and 7). In these studies, the drug facts box format contained information about the product's efficacy and safety in terms of rate (how many people in the clinical trial experienced a benefit or side effect compared to placebo). As expected, this study showed that consumers who were provided efficacy information used it. Participants receiving efficacy information (without other potentially valuable information about the drug) were more likely to correctly choose the product with the higher efficacy than consumers who saw the brief summary that did not contain this information.

Although these results are intriguing, additional research is necessary to uncover important information about how consumers understand effectiveness information about prescription drug products from direct-to-consumer advertisements. For example, the research to date does not address whether simply adding efficacy rate information and qualitative summations to a consumer-friendly brief summary would enable consumers to find and report the correct answer, or if the presentation of information in a chart format itself increases comprehension.

Further, these data cannot address the best way in which to convey numerical information; percents were used but another format, such as frequencies, may be more effective at communicating quantitative information. Previous research shows that individuals have great difficulty processing numerical concepts (e.g., Beyth-Marom, 1982; Bowman, 2002; Cohen, Ferrell, and Johnson, 2002) (Refs. 8, 9, and 10). A few studies have attempted to determine what different formats makes these concepts least troublesome (e.g., Fagerlin, Wang, and Ubel, 2005; Lipkus, 2007) (Refs. 11 and 12), however, most research into the communication of numerical concepts concentrates on risk information. We are not aware of research looking into the integration of quantitative information about effectiveness or benefits into the body of the advertisement itself. The addition of this information may help consumers make better health care decisions, provided they can understand it.

It is also not known if ways of communicating product efficacy work equally well across print and television DTC media. To our knowledge, research on presenting quantitative information in risk communication has been conducted exclusively with static modalities. The ideal format for presenting quantitative information may vary as a function of presentation. The amount of mental processing capacity each individual can devote to understanding a message varies depending on how long individuals have to look at the material and whether the material is self-paced or presented at an uncontrollable speed. As a result, some forms of quantitative information may lend themselves to print, rather than broadcast. This particular understanding is crucial to the risk-benefit tradeoff that patients must make with the consultation of a health care professional in order to achieve the best health outcomes.

The proposed study will examine: (1) Various ways of communicating quantitative efficacy in DTC print ads and (2) whether the findings translate to DTC television ads.

In the Federal Register of June 22, 2009 (74 FR 29490), FDA published a 60-day notice requesting public comment on the proposed collection of information. FDA received four comments.

II. Comments on the Information Collection

In the following section, we outline the observations and suggestions raised in the comments and provide our responses.

(Statement 1) All four comments expressed support for the research to explore issues of quantitative benefit information. They all described the collection of data as a worthy endeavor which will provide useful information on how best to communicate information in DTC ads.

(Statement 2) Two comments suggested enhancing or supplementing the existing behavioral intention questions (questions 13a through d in the questionnaire).

(Response) We took this as an opportunity to examine our behavioral intention questions thoroughly. We decided to maintain three of our four behavioral intention questions but remove one of them because of possible redundancy. We also added a new item to this question on the basis of a comment from one of our peer reviewers. Although we took seriously the suggestion to inquire about use of the Internet, one of our existing questions already covers this issue. In the interest of brevity, we have decided to streamline this section.

(Statement 3) One comment suggested including some questions about the risk/benefit tradeoff.

(Response) We plan to do so and these questions can be seen in questions 23a through d of the questionnaire. We labeled this variable “attitude toward drug” because it is easier to analyze and interpret using this term.

(Statement 4) Three comments suggested adding different types of participants to our sample, including: (1) A general population sample, (2) a sample of participants suffering from a medical condition that they can diagnose themselves, and (3) samples of at least three different medical conditions.

(Response) We selected high cholesterol because it is prevalent in the population and is commonly advertised DTC. We think adding a medical condition that is symptomatic or can otherwise be self-diagnosed is an excellent suggestion. We hope to explore the research questions in the current study in a variety of other medical conditions in future research.

(Statement 5) Two comments suggested comparing the test ad with either the standard of care or with multiple other comparators instead of simply comparing it to placebo.

(Response) In response, we remind readers that this is the first study to examine issues of quantitative benefit information in print and television DTC ads and that existing literature paints a grim picture of the amount of numerical information viewers may be likely to absorb. Thus, we are using the simplest comparison for this first study. We agree that future studies should examine other types of comparisons; however, we remind readers that only comparisons that are in the approved product labeling can be displayed in promotional pieces.

(Statement 6) One comment recommended the use of the Newest Vital Sign health literacy test.

(Response) We examined this test and considered it for use in our design, but ultimately decided against it for a number of reasons. First, we would have to modify the test so that it could be administered over the Internet rather than in person. It is unclear how some aspects of the test could be altered in such a way. Second, the test takes approximately 3 minutes when administered in person and may take as long or longer to administer via computer. We believe that numeracy is the key component of health literacy that will influence the results of our study, and we have devoted considerable space in the questionnaire to its measurement (see questions 29a through f, 30a through d, and 31a through d of the questionnaire). Because of time constraints and the key role of numeracy, we will maintain our current questions to thoroughly examine numeracy and provide basic information on health literacy. We will also include a one-item subjective health literacy item (see question 28 in the questionnaire). We will continue to examine the Newest Vital Sign measure for future research.

(Statement 7) Two comments expressed concern that our study does not address the role of the health care provider and overstates the decisions that consumers can make about their prescription drugs.

(Response) We agree that the health care provider is the best person to interpret clinical data and that the consumer or patient does not make the final prescribing decision. Nonetheless, DTC is currently directed at consumers in such a way that they have information about the risk side of the risk/benefit tradeoff but no specific information about the benefit side. This study is designed to assess whether adding specific benefit information will help consumers understand how well the product works, which may ultimately result in better-informed conversations with their health care providers.

(Statement 8) One comment suggested looking at the results of this study in conjunction with the results of another study we are conducting concerning the role of distraction in television ads in order to inform the development of future research.

(Response) This is an excellent suggestion that shows a strong understanding of the Division of Drug Marketing, Advertising and Communications' (DDMAC) long-term research goals. We plan to use the results of these two studies, in part, to strengthen the development of our future research.

(Statement 9) One comment recommended the inclusion of open-ended recall questions in the questionnaire.

(Response) We have included some open-ended questions in the revised questionnaire (see questions 4 and 15 in the questionnaire).

(Statement 10) One comment suggested including questions about perceptions of safety and efficacy. A related comment suggested using personal framing rather than asking about “the average person.”

(Response) We have included questions about safety and efficacy perceptions and these are shown in the revised questionnaire (see questions 15, 16, 17, and 20 in the questionnaire). We combed through the questionnaire to determine the best framing for each question. Where possible we added personalizing language, but in portions of the questionnaire that measure recall of the words in the ad, we mimicked the language of the ad (see questions 14a through h and 18a through i in the questionnaire).

(Statement 11) One comment suggested copy testing our mock ad before it is included in the protocol.

(Response) This is an excellent suggestion that cannot be implemented due to limited resources. Nevertheless, we conducted extensive pretesting of the stimuli ad for a previous project and applied the same procedures and concepts to the creation of the current mock ad. Moreover, we conducted limited cognitive testing (of fewer than nine people) to address such issues and these interviews provided some assurance that our ads were acceptable as were the ads for the other project.

(Statement 12) One comment suggested that we show the ads to participants as they would view them at home, i.e., in a clutter reel of ads for the television component and in a group of magazine ads in the magazine component.

(Response) Although embedding our stimuli within other ads would more closely mimic real viewing, we have several research questions to answer before we reach that point. We are not confident participants will understand any numerical information even when specifically directing them to one ad because this type of information seems to be so difficult for people to understand. We need to establish the basic parameters of statistical and visual information presentation before we can manipulate the realism of the situation and begin to examine other issues such as stopping power and attention.

(Statement 13) One comment recommended against using the Internet to administer the study and instead suggested the use of a mall-intercept protocol.

(Response) Although we recognize that one study cannot address all questions and repeat that the current study is planned to be the first among future studies, we do require several experimental conditions to answer basic presentation and comprehension questions. The resources necessary to conduct this study using a mall-intercept procedure give us less than half of the participants we are currently utilizing. Given that we are using a nationally representative, random digit dialing-based Internet panel to collect our experimental data, we feel that we are obtaining the best value for our funds. We do not feel that the tradeoffs in terms of external validity regarding mall-intercepts are favorable to that method.

(Statement 14) One comment recommended including an analysis plan for review, specifically one that addresses what result(s) would support a conclusion that the test ad has achieved a balanced presentation.

(Response) In response to the first part of this comment, we have included an analysis plan in this current document. In response to the second part of this comment, the primary research question in this study is not whether the information is balanced, but simply how well participants can understand numerical benefit information. Although we will address questions of balance and risk/benefit tradeoff in our questionnaire (see questions 23a through d in the questionnaire), our main dependent variables concern the recall and understanding of the benefit information, independent of the other information in the ad. Secondarily, we will examine recall and comprehension of risk information to assess whether it is affected by the inclusion of benefit information and the form the benefit information takes. Finally, we will look at the intersection of benefit and risk information, primarily in risk and benefit perception questions. Our main analyses, however, involve the understanding of benefit information and not in the balance of benefit and risk information. That is an excellent suggestion for future research.

(Statement 15) One comment expressed concern that high efficacy may not be the only reason to select one drug over another.

(Response) We agree. The current research is not designed to examine the multiple factors that a physician or a consumer considers when prescribing or deciding to take a drug. The scope of this project is to investigate the presentation of quantitative benefit information. We have chosen to vary the efficacy of the product (high versus low) as a simple method for determining whether viewers can understand how well the product works when this information is presented in different forms. We maintain that the efficacy of the drug is a major consideration in this decision and therefore represents a reasonable variable to use in this study.

(Statement 16) One comment was concerned that data presentation, and in particular the relative frequency presentation, would confuse consumers.

(Response) This comment reflects the very reason we are conducting the study. Before considering the idea of adding quantitative benefit information to DTC advertising, we want to ensure that we are not causing people to become more confused about their options. We have included the relative frequency condition specifically because we believe consumers do have trouble understanding this format. Sponsors have expressed interest in using this format in their ads and therefore this is a particularly important experimental condition for testing.

(Statement 17) One comment suggested that we ask questions about participant age and education.

(Response) We ask these and other demographic questions in this study (see questions 39 through 45 in the questionnaire).

(Statement 18) One comment mentioned that subjective measures of drug efficacy may confuse viewers.

(Response) We will define high and low efficacy quantitatively based on the range of efficacy currently found in the drug class. We will ask perception questions on Likert scales (e.g., strongly agree to strongly disagree) as well as numerical scales.

(Statement 19) One comment suggested that we are basing our entire study on an outdated study from 2001.

(Response) First, we provided information about the 2001 study to provide background information because it is relevant to the current study but have not based our entire research on it. Second, it is unclear what basic principles of human communication will have changed in the 8 years that have passed since the publication of this one study. Finally, although this one study shows that researchers in the field are investigating similar issues, no research currently exists to answer our research questions about the understanding of quantitative information in print and television DTC advertisements.

(Statement 20) One comment suggested that 20 minutes is not adequate for participants to complete this study.

(Response) We have completed similar studies in the past within 20 minutes. We will conduct cognitive testing before the administration of the study to ensure that the protocol can be completed within 20 minutes. Interviews lasting longer than 20 minutes have shown that participants tend not to want to spend that much time on them. Therefore, we will maintain the study at 20 minutes or less.

III. Revised Study

Based in part on these comments, further research discussions, and the input of three external reviewers, we propose the following revised design, hypotheses, and analysis plan.

A. Overview

This study will be conducted in two concurrent parts: One examining quantitative information in DTC print advertisements and the other examining such information in DTC television advertisements. Three factors will be examined: Drug efficacy, statistical format, and visual format.

We will investigate two levels of drug efficacy (low versus high), defined by a quantifiable, objective metric that can be conveyed in graphical representations of the drug versus the comparator reference drug (in this case, placebo). Specifically, high efficacy will be defined by a large, noticeable difference compared with no treatment; whereas low efficacy will be defined by a minimal difference between the drug and no treatment. We will examine two levels of efficacy to determine whether participants can accurately distinguish between these levels within various formats.

We will investigate five statistical formats, defined as the type of statistical information conveyed: Frequency, percent, frequency plus percent, relative frequency, and frequency plus relative frequency. Based on existing literature, we will use the frequency statistical format in all of our visual formats for consistency.

Visual format is defined as various methods through which efficacy can be visually represented. We have chosen to investigate four different formats: Pie chart, bar chart, table, and pictograph.

Additionally, we will have a control condition with no specific efficacy information provided. Please see the sample stimuli for the operationalization of each of these conditions. The factors will be combined in a partially crossed factorial design as follows:

		Statistical Format
		Frequency	Percent	Frequency + Percent	Relative Frequency	Frequency + Relative Frequency
Efficacy	Low
	High

and

		Visual Format
		None	Pie Chart	Bar Chart	Table	Pictograph
Efficacy	Low
	High

+ 1

No Statistical Format/No Efficacy

B. Procedure

This study will be administered over the Internet. A total of 2,250 interviews involving print ads will be completed. Participants in this part of the study will be randomly assigned to view one version of the magazine promotion page and the brief summary page of a prescription drug ad. Following their perusal of this document, they will answer questions about their recall and understanding of the benefit and risk information, their perceptions of the benefits and risks of the drug, and their intent to ask a doctor about the medication.

A total of 2,250 interviews involving television ads will be completed. Participants in this part of the study will be randomly assigned to view one version of a television ad twice and answer the same questions described in the previous paragraph.

For both parts, demographic and health care utilization information will be collected. The entire procedure is expected to last approximately 20 minutes. This will be a one-time (rather than annual) information collection.

C. Participants

Data will be collected using an Internet protocol. Participants will all have reported that a health care professional has diagnosed them with high cholesterol and will represent a range of education levels. Because the task presumes basic reading abilities, all selected participants must speak English as their primary language. Participants must be 18 years or older.

D. Hypotheses

1. Preface

The proposed research has two main objectives. First, we plan to test several statistical formats to determine whether the presentation of efficacy information in different formats affects perceptions of efficacy. The risk communication literature suggests that presenting numerical risk information as an absolute frequency (e.g., N out of 100) may be the most easily understood format (Fagerlin et al., 2007) (Ref. 13). Percent, and a combination of absolute frequency and percent, represent increasingly complex statistical formats; however, they may not differ from the baseline of absolute frequency for average consumers. In contrast, the risk communication literature suggests that presenting numerical risk information as a relative frequency (e.g., 10 times higher) is a markedly more complex statistical format that biases perceptions (Fagerlin et al., 2007) (Ref. 13). Thus, presenting efficacy information as a relative frequency, compared to absolute frequency, may affect perceptions of efficacy. Presenting the combination of absolute frequency and relative frequency may mitigate this effect.

Second, we plan to test several visual formats to determine whether the presentation of a visual format, in conjunction with the presentation of absolute frequency information, affects perceptions of efficacy. The risk communication literature suggests that the addition of visual formats such as bar charts, tables, and pictographs increase peoples' understanding of numerical information (Ancker et al., 2006; Lipkus and Hollands, 1999) (Refs. 14 and 15). However, not all visual formats are always helpful; for instance, pie charts may only help when people are comparing proportions (Lipkus, 2007) (Ref. 12). Thus, presenting efficacy information with a bar chart, table, and pictograph—but not necessarily with a pie chart—may affect people's understanding of efficacy information, in comparison to when there is no visual format.

Measuring numeracy will allow us to assess the magnitude of these effects across participants. Similarly, the separate TV and print portions of the study will allow us to assess the magnitude of these effects across these modalities.

2. Specific Hypotheses

a. Efficacy effects in print and TV ads.

(1) Behavioral intentions, attitude toward drug, and perceived efficacy will be higher in high efficacy conditions than in low efficacy conditions.

(2) We will explore whether there are differences between the no efficacy condition (control) and the low and high efficacy condition on behavioral intentions, attitude toward drug, and perceived efficacy.

(3) Benefit accuracy will be higher in the low and high efficacy conditions than in the no efficacy condition. There will be no difference between the low and high efficacy conditions.

(4) The effects tested in hypotheses (1) and (2), explained previously in section III.D.2 of this document, will be modified by numeracy, such that high numeracy participants will be more likely to show these effects than will low numeracy participants.

(5) Risk recall will not differ by efficacy level (no, low, high).

(6) Perceived risk will be lower in the high efficacy condition compared with the low efficacy condition because, according to the Affect Heuristic (Slovic and Peters, 2006) (Ref. 16), people perceive things that are more beneficial as less risky.

b. Statistical format effects in print and TV ads.

(1) We will test competing hypotheses for behavioral intentions, attitude toward drug, and perceived efficacy.

(1a) Overestimation hypothesis: The first hypothesis rests on the assumption that in the absence of any quantitative information people overestimate the effectiveness of drugs. Accordingly, we would predict that behavioral intentions, attitude toward drug, and perceived efficacy will be higher for participants in the no statistical format condition, compared to all other statistical format conditions. Support for this interpretation will be found if estimates of the benefits are higher in the no statistical format condition than in all other statistical format conditions.

(1b) Peripheral cue hypothesis: The competing hypothesis rests on the assumption that any statistical information will be used as a peripheral cue; that is, participants will not process the quantitative information provided in the various statistical formats but will rather view it as “scientific proof” of the drug's efficacy. Accordingly, we would predict that behavioral intentions, attitude toward drug, and perceived efficacy will be lower for participants in the no statistical format condition, compared to all other statistical format conditions. Support for this interpretation will be found if, in addition to perceived efficacy effects, estimates on attitude toward the ad “peripheral cue” measures—ratings of how believable, persuasive, informative, etc., the ad is—are lower in the no statistical format condition than in all other statistical format conditions.

(2) Based on the risk communication literature, we predict that the absolute frequency, percent, and absolute frequency and percent conditions may not differ on behavioral intentions, attitude toward drug, or perceived efficacy. However, we predict that behavioral intentions, attitude toward drug, and perceived efficacy will be higher in the relative frequency condition than in the absolute frequency, percent, absolute frequency + percent, and absolute frequency + relative frequency conditions.

(3) The effects tested in hypotheses (1) and (2) will be modified by numeracy. (See sections III.D.1 through 2 of this document.) For instance, we expect that the difference between the relative frequency and the absolute frequency + relative frequency conditions will be greater for high numeracy participants than for low numeracy participants (because high numeracy participants will be more likely to use the additional information provided by the absolute frequency).

(4) Benefit accuracy will be lowest in the no statistical format condition and highest in the absolute frequency condition (Slovic, Monahan, and MacGregor, 2000) (Ref. 17). Tests of other relations between statistical formats will be exploratory. For instance, we might see information overload with some formats (e.g., absolute frequency and relative frequency) which impedes benefit accuracy.

(5) The effects tested in hypothesis (4) will be modified by numeracy, such that low numeracy participants will show greater differences in benefit accuracy across statistical formats than will high numeracy participants (Peters, Vastfjall, et al., 2006) (Ref. 18).

(6) We expect that risk recall will not differ by statistical format, but we will conduct exploratory analyses to determine whether information overload impedes risk recall.

(7) We expect that perceived risk will be lowest in the relative frequency condition if perceived benefit is indeed highest in this condition (see Slovic and Peters, 2006, reference 16 of this document).

c. Visual format effects in print and TV ads.

(1) We will test competing hypotheses for benefit accuracy, behavioral intentions, attitude toward drug, and perceived efficacy.

(1a) Visual information facilitation hypothesis: The first hypothesis rests on the assumption that participants will, to the extent possible, process and use the information in the visual formats. The risk communication literature suggests that visual representations of risk can increase understanding, and that people have a more difficult time processing this kind of information in pie charts, as compared to other visual formats. Therefore, our first hypothesis is that benefit accuracy will be higher in the bar chart, table, and pictograph conditions—but not necessarily the pie chart condition—than in the no visual format condition. Tests of other relations between visual formats will be exploratory.

(1b) Information overload hypothesis: Alternatively, there may be no differences across visual formats on behavioral intentions, attitude toward drug, perceived efficacy, or benefit accuracy if the visual serves as a distraction or is too much information to process.

(1c) Peripheral cue hypothesis: Behavioral intentions, attitude toward drug, and perceived efficacy—but not benefit accuracy—may be higher in all visual conditions than in the no visual condition if the visual information serves as a peripheral cue.

(2) The effects tested in hypothesis (1) will be modified by numeracy. For instance, we expect that high numeracy participants will be more likely to process the information in the visual formats, and thus more likely to show the pattern of effects outlined in 1a, compared to low numeracy participants.

(3) We expect that perceived risk and risk recall will not differ by visual format but we will conduct exploratory analyses to determine whether information overload impedes risk recall.

E. Analysis Plan

We will conduct the following statistical analyses separately for the print and television versions of the ad.

Efficacy effects in print and TV ads: We will conduct Analysis of Variance (ANOVAs) to test whether the no statistical format/no efficacy condition differs from the low and high efficacy condition on the dependent measures (i.e., benefit accuracy, behavioral intentions, attitude toward drug, perceived efficacy, perceived risk, and risk recall, peripheral cue measures). We will conduct these analyses both with and without covariates (e.g., demographic and health characteristics) included in the model. In addition, we will test whether any main effects are moderated by other measured variables (e.g., numeracy, demographic, and health characteristics). If the main effect of efficacy is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see section III.D of this document). In addition, the main effect of efficacy (low vs. high) and any interaction it has with statistical format or visual format will be tested in the ANOVAs presented in the following two sections.

Statistical format effects in print and TV ads: We will conduct ANOVAs to test whether the no statistical format/no efficacy condition differs from the other statistical format conditions on the dependent measures. In addition, we will examine the main effect of statistical format in ANOVAs predicting our dependent measures from statistical format, efficacy level, and their interaction. We will conduct these analyses both with and without covariates included in the model. In addition, we will test whether any main effects are moderated by other measured variables. If the main effect of statistical format is significant, we will conduct pairwise-comparisons statistical tests to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses. (See section III.D of this document.)

Visual format effects in print and TV ads: To test our hypotheses regarding visual format, we will examine the main effect of visual format in ANOVAs predicting our dependent measures from visual format, efficacy level, and their interaction. We will conduct these analyses both with and without covariates included in the model. In addition, we will test whether any main effects are moderated by other measured variables. If the main effect of visual format is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses. (See section III.D of this document.)

The total annual estimated burden imposed by this collection of information is 1,755 hours for this one-time collection (table 1 of this document).

Table 1.—Estimated Annual Reporting Burden¹

Activity	No. of Respondents	Annual Frequency per Response	Total Annual Responses	Hours per Response	Total Hours
Screener	9,000	1	9,000	2/60	270
Questionnaire	4,500	1	4,500	20/60	1,485
Total					1,755
1 There are no capital costs or operating and maintenance costs associated with this collection of information.

These estimates are based on FDA's experience with previous consumer studies.

IV. References

The following references have been placed on display in the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852, and may be seen by interested persons between 9 a.m. and 4 p.m., Monday through Friday.

1. Schwartz, L., S. Woloshin, W. Black, et al., The Role of Numeracy in Understanding the Benefit of Screening Mammography, Annals of Internal Medicine, 127(11), 966-72, 1997.

2. Draft Guidance for Industry: Presenting Risk Information in Prescription Drug and Medical Device Advertising, available at http://www.fda.gov/​downloads/​Drugs/​GuidanceComplianceRegulatoryInformation/​Guidances/​UCM155480.pdf.

3. Woloshin, S. and L. Schwartz, Direct to Consumer Advertisements for Prescription Drugs: What Are Americans Being Told, Lancet, 358, 1141-46, 2001.

4. Frosch, D.L., P.M. Krueger, R.C. Hornik, et al., Creating Demand for Prescription Drugs: A Content Analysis of Television Direct-to-Consumer Advertising, Annals of Family Medicine, 5(1), 6-13, 2007.

5. Schwartz, L.M., S. Woloshin, H.G. Welch, The Drug Facts Box: Providing Consumers With Simple Tabular Data on Drug Benefit and Harm, Medical Decision Making, 27, 655-692, 2007.

6. Schwartz, L.M., S. Woloshin, H.G. Welch, Communicating Drug Benefits and Harms Wth a Drug Facts Box: Two Randomized Trials, Annals of Internal Medicine, 150, 516-527, 2009.

7. Woloshin, S., L.M. Schwartz, H.G. Welch, The Value of Benefit Data in Direct-to-Consumer Drug Ads, Health Affairs, Web Exclusive Supplement, W4-234-245, 2004.

8. Beyth-Marom, R., How Probable is Probable? A Numerical Translation of Verbal Probability Expressions, Journal of Forecasting, 1, 257-269, 1982.

9. Bowman, M.L., The Perfidity of Percentiles, Archives of Clinical Neuropsychology, 17, 295-303, 2002.

10. Cohen, D.J., J.M. Ferrell, N. Johnson, What Very Small Numbers Mean, Journal of Experimental Psychology: General, 131, 424-442, 2002.

11. Fagerlin, A., C. Wang, P.A. Ubel, Reducing the Influence of Anecdotal Reasoning on People's Health Care Decisions: Is a Picture Worth a Thousand Statistics?, Medical Decision Making, 25, 398-405, 2005.

12. Lipkus, I., Numeric, Verbal, and Visual Formats of Conveying Health Tasks: Suggested Best Practices and Future Recommendations, Medical Decision Making, 27, 697-713, 2007.

13. Fagerlin, A., P.A. Ubel, D.M. Smith, et al., Making Numbers Matter: Present and Future Research in Risk Communication, American Journal of Health Behavior, 31, Supplement 1: S47-56, 2007.

14. Ancker, J.S., Y. Senathirajah, R. Kukafka, et al., Design Features of Graphs in Health Risk Communication: A Systematic Review, Journal of the American Medical Information Association, 13, 608-618, 2006.

15. Lipkus, I., J.G. Hollands, The Visual Communication of Risk, Journal of the National Cancer Institute Monographs, 25, 149-163, 1999.

16. Slovic, P. and E. Peters, Risk Perception and Affect, Current Directions in Psychological Science, 15, 322-325, 2006.

17. Slovic, P., J. Monahan, DG MacGregor, Violence Risk Assessment and Risk Communication: The Effects of Using Actual Cases, Providing Instruction, and Employing Probability Versus Frequency Formats, Law and Human Behavior, 24, 271-96, 2000.

18. Peters, E., D. Vastfjall, P. Slovic, et al., Numeracy and Decision Making, Psychological Science, 17, 407-13, 2006.

Dated: December 23, 2009.

David Horowitz,

Assistant Commissioner for Policy.

Footnotes

1.  For prescription drugs and biologics, the act requires advertisements to contain “information in brief summary relating to side effects, contraindications, and effectiveness” (section 502(n) of the act (21 U.S.C. 352(n)).

2.  See section 502(n) of the act.