2020-11957. Agency Information Collection Activities; Submission to the Office of Management and Budget (OMB) for Review and Approval; Comment Request; Generic Clearance for Internet Panel Pretesting and Qualitative Survey Methods Testing

AGENCY:

U.S. Census Bureau, Commerce.

ACTION:

Notice of information collection; request for comment.

SUMMARY:

The Department of Commerce, in accordance with the Paperwork Reduction Act of 1995 (PRA), invites the general public and other Federal agencies to comment on proposed, and continuing information collections, which helps us assess the impact of our information collection requirements and minimize the public's reporting burden. The purpose of this notice is to allow for 60 days of public comment on the proposed extension of the Generic Clearance for Internet Panel Pretesting, prior to the submission of the information collection request (ICR) to OMB for approval.

DATES:

To ensure consideration, comments regarding this proposed information collection must be received on or before August 3, 2020.

ADDRESSES:

Interested persons are invited to submit written comments by mail to [e-mail]. Please reference OMB Control Number 0607-0978 in the subject line of your comments. You may also submit comments, identified by Docket Number USBC-2020-0007, to the Federal e-Rulemaking Portal: http://www.regulations.gov. All comments received are part of the public record. No comments will be posted to http://www.regulations.gov for public viewing until after the comment period has closed. Comments will generally be posted without change. All Personally Identifiable Information (for example, name and address) voluntarily submitted by the commenter may be publicly accessible. Do not submit Confidential Business Information or otherwise sensitive or protected information. You may submit attachments to electronic comments in Microsoft Word, Excel, or Adobe PDF file formats.

FOR FURTHER INFORMATION CONTACT:

Requests for additional information or specific questions related to collection activities should be directed to Jennifer Hunter Childs, U.S. Census Bureau, 4600 Silver Hill Road, Center for Behavioral Science Methods, Washington, DC 20233 or [phone].

SUPPLEMENTARY INFORMATION:

I. Abstract

The Census Bureau is committed to conducting research in a cost-efficient manner. The U.S. Census Bureau plans to request an extension of the current OMB approval to conduct a series of medium-scale internet-based tests, as a cost-efficient method of testing questions and contact strategies over the internet through different types of samples. Using internet panel pretesting, we can answer some research questions more thoroughly than in the small-scale testing, but less expensively than in the large-scale field test.

This research program will be used by the Census Bureau and survey sponsors to test alternative contact methods, including emails and text messages, improve online questionnaires and procedures, reduce respondent burden, and ultimately increase the quality of data collected in the censuses and surveys. We will use the clearance to conduct experimental pretesting of decennial and demographic census and survey questionnaires prior to fielding them as well as communications and/or marketing strategies and data dissemination tools for the Census Bureau. The primary method of identifying measurement problems with the questionnaire or survey procedure is split panel tests. This will encompass both methodological and subject matter research questions that can be tested on a medium-scale internet panel.

This research program will also be used by the Census Bureau for remote usability testing of electronic interfaces and to perform other qualitative analyses such as respondent debriefings. An advantage of using remote, medium-scale testing is that participants can test products at their convenience using their own equipment, as opposed to using Census Bureau-supplied computers. A diverse participant pool (geographically, demographically, or economically) is another advantage. Remote usability testing would use click through rates and other paradata, accuracy and satisfaction scores, and written qualitative comments to determine optimal interface designs and to obtain feedback from respondents.

The public is currently offered an opportunity to participate in this research remotely, by signing up for an online research panel. If a person opts in, the Census Bureau will occasionally email (or text, if applicable) the person an invitation to complete a survey for one of our research projects. Invited respondents will be told the topic of the survey, and how long it will take to complete it. Under this clearance, we will also conduct similar-scale and similarly designed research using other email lists to validate preliminary findings and expand the research.

II. Method of Collection

Split sample experiments. This involves testing alternative versions of questionnaires, invitations to questionnaires (e.g., emails or text messages), or websites, at least some of which have been designed to address problems identified in draft versions or versions from previous waves. The use of multiple questionnaires, invitations, or websites, randomly assigned to permit statistical comparisons, is the critical component here; data collection will be via the internet. Comparison of revised questionnaires (or invitations) against a control version, preferably, or against each other facilitates statistical evaluation of the performance of alternative versions of the questionnaire (or invitation or website).

The number of versions tested and the number of cases per version will depend on the objectives of the test. We cannot specify with certainty a minimum panel size, although we would expect that no questionnaire versions would be administered to less than fifty respondents.

Split sample tests that incorporate methodological questionnaire design experiments will have a larger maximum sample size (up to several hundred cases per panel) than other pretest methods. This will enable the detection of statistically significant differences, and facilitate methodological experiments that can extend questionnaire design knowledge more generally for use in a variety of Census Bureau data collection instruments.

Usability Interviews: This method involves getting respondent input to aid in the development of automated questionnaires and websites and associated materials. The objective is to identify problems that keep respondents from completing automated questionnaires accurately and efficiently with minimal burden, or that prevent respondents from successfully navigating websites and finding the information they seek. Remote usability testing may be conducted under this clearance, whereby a user would receive an invitation to use a website or survey, then answer targeted questions about that experience.

Qualitative Interviews: This method involves one-on-one (or sometimes group) interviews in which the respondent is typically asked questions about survey content areas, survey questions or the survey process. A number of different techniques may be involved, including cognitive interviews and focus groups. The objective is to identify problems of ambiguity or misunderstanding, or other difficulties respondents may have answering survey questions in order to improve the information ultimately collected in large scale surveys and censuses.

Data collection for this project is authorized under the authorizing legislation for the questionnaire being tested. This may be Title 13, Sections 131, 141, 161, 181, 182, 193, and 301 for Census Bureau-sponsored surveys, and Title 13 and 15 for surveys sponsored by other Federal agencies. We do not now know what other titles will be referenced, since we do not know what survey questionnaires will be pretested during the course of the clearance.

Literature on and considerations about the use of internet samples for this type of work have been thoroughly covered by a Task Force commissioned by the American Association for Public Opinion Research and are well documented there (Baker, et al., 2013).

The information collected in this program of developing and testing questionnaires will be used by staff from the Census Bureau and sponsoring agencies to evaluate and improve the quality of the data in the surveys and censuses that are ultimately conducted. Because the questionnaires being tested under this clearance are still in the process of development, the data that result from these collections are not considered official statistics of the Census Bureau or other Federal agencies. Data will be included in research reports prepared for sponsors inside and outside of the Census Bureau. The results may also be prepared for presentations related to survey methodology at professional meetings or publications in professional journals.

III. Data

OMB Control Number: 0607-0978.

Form Number(s): TBD.

Type of Review: Regular submission, Request for an Extension, without Change, of a Currently Approved Collection.

Affected Public: Individuals or households.

Estimated Number of Respondents: 67,600.

Estimated Time per Response: 15 minutes.

Estimated Total Annual Burden Hours: 16,900.

Estimated Total Annual Cost to Public: $0.

Respondent's Obligation: Voluntary.

Legal Authority: Data collection for this project is authorized under the authorizing legislation for the questionnaire being tested. This may be Title 13, Sections 131, 141, 161, 181, 182, 193, and 301 for Census Bureau-sponsored surveys, and Title 13 and 15 for surveys sponsored by other Federal agencies. We do not now know what other titles will be referenced, since we do not know what survey questionnaires will be pretested during the course of the clearance.

IV. Request for Comments

We are soliciting public comments to permit the Department/Bureau to: (a) Evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the Department, including whether the information shall have practical utility; (b) Evaluate the accuracy of our estimate of the time and cost burden for this proposed collection, including the validity of the methodology and assumptions used; (c) Evaluate ways to enhance the quality, utility, and clarity of the information to be collected; and (d) Minimize the reporting burden on those who are to respond, including the use of automated collection techniques or other forms of information technology.

Comments that you submit in response to this notice are a matter of public record. We will include or summarize each comment in our request to OMB to approve this ICR. Before including your address, phone number, email address, or other personal identifying information in your comment, you should be aware that your entire comment—including your personal identifying information—may be made publicly available at any time. While you may ask us in your comment to withhold your personal identifying information from public review, we cannot guarantee that we will be able to do so.

Sheleen Dumas,

Department PRA Clearance Officer, Office of the Chief Information Officer, Commerce Department.