2015-14125. Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Establishment of a Tobacco User Panel

AGENCY:

Food and Drug Administration, HHS.

ACTION:

Notice.

SUMMARY:

The Food and Drug Administration (FDA) is announcing that a proposed collection of information has been submitted to the Office of Management and Budget (OMB) for review and clearance under the Paperwork Reduction Act of 1995.

DATES:

Fax written comments on the collection of information by July 10, 2015.

ADDRESSES:

To ensure that comments on the information collection are received, OMB recommends that written comments be faxed to the Office of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer, FAX: [phone], or emailed to [e-mail]. All comments should be identified with the OMB control number 0910-NEW and title “Establishment of a Tobacco User Panel”. Also include the FDA docket number found in brackets in the heading of this document.

FOR FURTHER INFORMATION CONTACT:

FDA PRA Staff, Office of Operations, Food and Drug Administration, 8455 Colesville Rd., COLE-14526, Silver Spring, MD 20993-0002, [e-mail].

SUPPLEMENTARY INFORMATION:

In compliance with 44 U.S.C. 3507, FDA has submitted the following proposed collection of information to OMB for review and clearance.

Establishment of a Tobacco User Panel—(OMB Control Number 0910-NEW)

The Food and Drug Administration's Center for Tobacco Products (CTP) proposes to establish a high quality, probability-based, primarily Web-based, panel of 4,000 tobacco users. The panel will include individuals who can participate in up to 8 studies over a 3-year period to assess consumers' responses to tobacco marketing, warning statements, product labels, and other communications about tobacco products. CTP proposed the establishment of the panel of consumers because currently existing Web-based panels have a number of significant limitations.

First, most existing consumer panels are drawn from convenience samples that limit the generalizability of study findings (Ref. 1). Second, although at least two probability-based panels of consumers exist in the United States, there is a concern that responses to the studies using tobacco users in these panels may be biased due to panel conditioning effects (Refs. 2 and 3). That is, consumers in these panels complete surveys so frequently that their responses may not adequately represent the population as a whole. Panel conditioning has been associated with repeated measurement on the same topic (Ref. 4), panel tenure (Ref. 2), and frequency of the survey request (Ref. 3). This issue is of particular concern for tobacco users who represent a minority of the members in the panels, and so may be more likely to be selected for participation in experiments and/or surveys related to tobacco products. Third, a key benefit of the Web panel approach is that the surveys can include multimedia, such as images of tobacco product packages, tobacco advertising, new and existing warning statements and labels, and potential reduced harm claims in the form of labels and print advertisements. Establishing a primarily Web-based panel of tobacco users through in-person probability-based recruitment of eligible adults and limiting the number of times individuals participate in tobacco-related studies will result in nationally representative and unbiased data collection on matters of importance for FDA.

With this submission, FDA seeks approval from OMB to establish the Tobacco User Panel, a nationally representative, primarily Web-based panel of 4,000 current tobacco users. Data collection activities will involve pilot testing of panel recruitment and management procedures and systems, mail and in-person household screening, in-person recruitment of tobacco users, enrollment of selected household members, administration of a baseline survey, and panel maintenance surveys, following all required informed consent procedures for panel members. Once the panel is established, panel members will be asked to participate in up to eight experimental and observational studies over the 3-year panel commitment period. The first of these studies (Study 1) is included in this information collection request; approval for the remainder of the studies will appear in future requests. The current request also seeks approval to conduct up to two rounds of cognitive testing of new survey items and up to two focus groups to further refine study protocols, as needed. With this clearance, study investigators will be able to use the OMB approved data collection methods where appropriate to plan and implement the national panel.

The overall purpose of the proposed data collection is to collect information from a representative sample of tobacco users to provide data that may be used to develop and support FDA's policies related to tobacco products, including their labels, labeling, and advertising. Data will be collected from the panel primarily through the use of randomized experimental designs, however, there may be data collected through the use of other methods, such as surveys, interviews, or online group discussions. Given the limitations on the existing Web-based panels, it is important to develop a new panel of tobacco users that balances the need to conduct experiments while limiting the number of tobacco-related studies per year so as to not bias study results.

FDA estimates the burden of this collection of information as follows:

In the Federal Register of October 16, 2014 (79 FR 62160), FDA published a 60-day notice requesting public comment on the proposed collection of information. FDA received three comments, however only two were PRA related. Within those submissions, FDA received multiple comments which the Agency has addressed.

(Comment) One comment asked FDA for the opportunity to review the data collection plans and instruments including the sample design, data collection methodology, and panel performance evaluation plan.

(Response) All the instruments and background documents including our plan for evaluating panel performance have been uploaded to the docket for easy access. The documents included are the data collection plans and methodology (Supporting Statement Part A), copies of the survey instruments used to screen and recruit panel members, as well as the first experimental or observation study (Study 1), and the proposed sample design (Supporting Statement Part B).

(Comment) One comment asked FDA to provide additional details about the proposed sample design and FDA's approach to issues such as nonresponse of subjects and conditioning effects.

(Response) The proposed sample design is described in detail in Supporting Statement, Part B. Briefly, we propose a multi-stage area sample based on an address-based sampling frame. The probabilities (single, joint, and the overall selection probability) will be measurable at each stage.

The issues of non-response and conditioning effects are real challenges but they should be considered separately from the sample design. These are issues faced in the field once the sample has been selected and contacted. We have proposed several strategies for reducing non-response in the recruitment of panel members, the primary one being in-person recruitment which we believe will lead to significantly larger recruitment rates than we would achieve if we contacted sample members via mail, telephone, or web. We will describe our plans to reduce the non-response bias in future individual studies as part of the OMB submissions for these studies. We consider the issue of conditioning effects as part of our overall panel management plan, which is described in Supporting Statement, Part A.

(Comment) One comment stated that FDA suggests that not every panelist will be eligible to participate in every study to minimize the potential for “conditioning” effects. However, this approach to participation is inconsistent with the requirement that every individual in the population has a non-zero probability of being in the sample. FDA will need to make trade-offs to balance these two interests. FDA could consider drawing data from similar respondents, as long as FDA knows that there are no important hidden differences between the respondents that may affect their responses.

(Response) We will draw the original sample with known, non-zero, and, to the extent possible, equal probabilities. The same will apply to any additional samples drawn for the panel to replace attrition. Furthermore, any subsample drawn from the panel for specific studies will also result in known probabilities of selection. We will derive a strategy of spreading the survey-taking load over all panel members to avoid excessive burden on any single member or group of members. We will implement this strategy by randomly selecting each subsample, but at the same time keeping track of each member's survey-taking activity. As the number and frequency of survey-taking for a given member increases, their probability of selection will decrease, a strategy that we will implement using probability proportion to size sampling. This strategy will lead to known and measurable selection probabilities for each specific subsample.

(Comment) One comment stated FDA should consider, whether in some instances, collecting fresh data from new samples of tobacco product users over time may provide better results.

(Response) Our proposed approach includes replenishment of the sample over time to address attrition from the panel. As such, the panel will include tobacco users with varying tenure lengths on the panel. We will be in a position to restrict a specific study subsample to the more recent panel members, if desired, and more generally, the panel will allow FDA to specify the composition of the sample with respect to tenure.

(Comment) One comment said FDA should consider inclusion of non-tobacco users or users of specific tobacco categories (e.g., e-cigarette users, moist smokeless tobacco users) in the sample to support comparative analyses between users and non-users or subgroup analyses.

(Response) FDA considered including non-tobacco users early in the planning process. However, the planned experimental and observational studies will examine issues specific to the tobacco-using population, especially those with lower socio-economic status. This includes the underlying demographics of users as well as their knowledge, attitudes, practices, behaviors, and reactions to various tobacco-related stimuli. Other existing data sources, including survey panels, support research with non-users. Moreover, limiting the panel to users reduces the overall public burden. Once the panel is firmly established, we may consider its expansion.

(Comment) One comment stated FDA should also consider how well the sample of 4,000 adult tobacco users will support the planned investigations.

(Response) The sample size of 4,000 was chosen after a careful review of, on the one hand, power and subclass analyses requirements, and on the other hand, the budgetary implications. After our careful review, we concluded that a sample size of 4,000 tobacco users represents a good balance, at least for the first iteration of the panel.

We should also mention that the young adult population (aged 18-25) and the low-income population (combined household income less than $30,000) will be oversampled allowing for more in-depth study of these two groups of tobacco users. We also include a screening feature that will result in oversampling of the smokeless tobacco users.

(Comment) One commenter stated that FDA suggests that the approach includes a “3-year panel commitment period”. FDA should consider developing and sharing its plan for keeping or removing panelists. For example, will FDA keep or remove a panelist if he/she decides to quit using tobacco products? Also, how will FDA monitor whether incentives are influencing a panelist's responses or behavior? These are only a few examples of issues that could arise; therefore, a thoughtful panel management plan is needed.

(Response) We agree that a detailed and well-designed panel management plan is needed to make the panel successful. The literature on panel maintenance is growing, but there is still much to be learned about optimal strategies for maintaining a strong and productive panel. Supporting Statement, Part A outlines our plans for panel management, including retention and nonresponse follow-up strategies, planned incentive experiments, monitoring of panel conditioning, and evaluation of the effects of various panel maintenance strategies on substantive responses.

Continual monitoring is planned to study these and other important aspects of the panel's health. We will also keep a close eye on individual panelists, their participation patterns, and their non-response patterns to identify potential problems requiring intervention. FDA considered removing panel members who report they have stopped using tobacco products. Because of recidivism rates, it was decided to retain all enrolled panel members regardless of changes in their tobacco use patterns. Subsampling of panelists may be implemented for specific experimental or observational studies that are intended solely for current users of one or more specific tobacco products.

(Comment) One commenter stated FDA should consider establishing mechanisms to evaluate the performance of the panel as well as the data derived from it. For example, data from the panel on measures such as current or past 30-day cigarette smoking might be compared against the most recent data from national surveys and other published reports.

(Response) We agree that benchmarking the panel sample characteristics—demographic, socioeconomic, and tobacco use—against other national data sources is extremely important. We will continuously check that our panel matches known underlying population characteristics. However, we will also monitor how the panel compares with the target population with respect to known patterns of behavior surrounding tobacco use. Differences will not necessarily suggest problems with the panel but they will stimulate further investigation and explanation.

(Comment) One commenter asked FDA to provide copies of the survey instruments for public comment.

(Response) Copies of the survey instruments used to screen and recruit panel members, as well as the first experimental or observation study (Study 1), are uploaded to the docket.

(Comment) One commenter strongly supports FDA's proposed collection of information. The commenter stated that this panel is of great utility and the proposed probability-based panel will serve as a flexible tool, giving FDA the opportunity to conduct diverse studies.

(Response) FDA agrees with this comment and believes the panel will be a valuable tool for conducting new experimental studies.

FDA estimates the burden of this collection of information as follows:

Table 1—Estimated Annual Reporting Burden ¹

Activity or type of respondent	Number of respondents	Number of responses per respondent	Total annual responses	Average burden per response	Total hours
Household Screening Respondent	29,385	0.33	9,697	0.16 (10 minutes)	1,552
Panel Member Enrollment Survey		0.33	1,320	0.25 (15 minutes)	330
Panel Member Baseline Survey		0.33	1,320	0.25 (15 minutes)	330
Panel Maintenance/Bi-annual Update Surveys	4,000	3.0	12,000	0.08 (5 minutes)	960
Experimental/Observational Studies *		2.7	10,800	0.33 (20 minutes)	3,564
Panel Replenishment Screening Respondent	10,285	0.50	5,143	0.16 (10 minutes)	823
Panel Replenishment Enrollment Survey **	2,800	0.33	924	0.25 (15 minutes)	231
Panel Replenishment Baseline Survey **	2,800	0.33	924	0.25 (15 minutes)	231
Cognitive Interview Subjects	20	0.33	7	1.0	7
Focus Group Subjects	20	0.33	7	1.5	10
Total	49,310				8,038
1 There are no capital or operating and maintenance costs or associated with this collection of information.
* Includes a total of 8 experimental or observational studies over a 3-year period for each of the 4,000 panel members who are active at the time of each study. The first study (Study 1) is included in this clearance request; the remaining studies will be funded under separate task orders but are included in this table to present an overall estimate of the burden for each participating panel member.
** Assumes 1,400 additional panel members will be recruited annually (2,800 total) as part of the panel replenishment effort.

The collection burden was estimated using data from timed-readings of each instrument, including the mail and field screeners, enrollment survey, baseline survey, panel maintenance questionnaires, and Study 1 questionnaire.

References

The following references have been placed on display in the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852, and may be seen by interested persons between 9 a.m. and 4 p.m., Monday through Friday, and are available electronically at http://www.regulations.gov.

1. Baker, R., Blumberg, S., Brick, M., Couper, M., Courtright, M., Dennis, J. M., Dillman, D., Frankel, M., Garland, P., Groves, R., Kennedy, C., Krosnick, J. and Lavrakas, P., 2010, American Association for Public Opinion Research Report on Online Panels. Public Opinion Quarterly, 74 (4), pp. 711-781.

2. Coen, T., Lorch, J. and Piekarski, L., 2005, The Effects of Survey Frequency on Panelists' Responses, Worldwide Panel Research: Developments and Progress, Amsterdam, European Society for Opinion and Marketing Research.

3. Nancarrow, C. and Catwright, T., 2007, Online Access Panels and Tracking Research, The Conditioning Issue, International Journal of Market Research, 49(5), pp. 435-447.

4. Kruse, Y., Callegaro, M., Dennis, J. M., DiSogra, C., Subias, S., Lawrence, M., and Tompson, T., 2009, Panel Conditioning and Attrition in the AP-Yahoo! News Election Panel Study, Paper presented at the American Association for Public Opinion Research 64th Annual Conference.

Dated: June 4, 2015.

Leslie Kux,

Associate Commissioner for Policy.