2017-07951. Submission for OMB Review; Comment Request; American Community Survey Methods Panel Tests, 2017 Mail Design Test

The Department of Commerce will submit to the Office of Management and Budget (OMB) for clearance the following proposal for collection of information under the provisions of the Paperwork Reduction Act (44 U.S.C. chapter 35).

Agency: U.S. Census Bureau.

Title: American Community Survey Methods Panel Tests, 2017 Mail Design Test.

OMB Control Number: 0607-0936.

Form Number(s): ACS-1, ACS CATI, ACS CAPI, ACS Internet.

Type of Request: Non-substantive Change Request.

Number of Respondents: 288,000.

Average Hours per Response: 40 minutes.

Burden Hours: No additional burden hours are requested under this non-substantive change request.

Needs and Uses: The American Community Survey (ACS) collects detailed socioeconomic data from about 3.5 million housing units in the United States and 36,000 in Puerto Rico each year. The ACS also collects detailed socioeconomic data from about 195,000 residents living in Group Quarter (GQ) facilities. An ongoing data collection effort with an annual sample of this magnitude requires that the ACS continue research, testing, and evaluations aimed at reducing respondent burden, improving data quality, achieving survey cost efficiencies, and improving ACS questionnaire content and related data collection materials. The ACS Methods Panel is a research program that is designed to address and respond to issues and survey needs.

In the Census Bureau's continuing effort to reduce respondent burden and address concerns about the perceived intrusiveness of the ACS, the Census Bureau seeks to test three candidate changes to the current ACS mail materials. The three experimental treatments are designed to increase public awareness of the ACS through new messaging and an updated look and feel that increases respondent engagement and self-response, while softening the tone of the mandatory requirement of the survey.

The Census Bureau previously tested the impact of removing or modifying the mandatory messages from the mail materials (see Oliver, B., Risley, M., & Roberts, A. (2016). 2015 Summer Mandatory Messaging Test. Washington DC, U.S. Census Bureau. Retrieved on February 10, 2017 from https://www.census.gov/​content/​dam/​Census/​library/​working-papers/​2016/​acs/​2016_​Oliver_​01.pdf). This proposed test is aimed at building on that research and improving the results based on additional feedback the Census Bureau obtained from the National Academies' Committee on National Statistics (see Plewes, T.J. (2016). “Reducing Response Burden in the American Community Survey.” Proceedings of a Workshop conducted by the Committee on National Statistics Division of Behavioral and Social Sciences and Education).

The three experimental treatments are:

• The Softened Revised Design treatment from the 2015 Summer Mandatory Messaging Test.
• A Partial Redesign treatment that maintains the same wording as used in the Softened Revised Design treatment but includes some methodological changes: A “Why We Ask” brochure in the initial mailing, changes to the cover of the paper questionnaire, and the use of a letter instead of a postcard for the fifth mailing.
• A Full Redesign treatment that includes the same methodological changes as the Partial Redesign treatment but also modifies the wording in most of the mailings to a more personal approach with plain language.

The purpose of this test is to study the impact of these three candidate mail designs on self-response, cost, and the precision of the estimates. To field this test, the Census Bureau plans to use the ACS production sample (clearance number: 0607-0810, expires 06/30/2018). Thus, there is no increase in burden from this test since each treatment will result in the same burden estimate per interview (40 minutes). The ACS sample design consists of randomly assigning each monthly sample panel into 24 groups of approximately 12,000 addresses each. Each group, called a methods panel group, within a monthly sample is representative of the full monthly sample. Each monthly sample is a representative subsample of the entire annual sample and is representative of the sampling frame.

The Census Bureau proposes to test these mail designs as part of the ACS August 2017 panel, adhering to the same data collection protocols as production ACS. The Census Bureau proposes to use two randomly selected methods panel groups for each treatment. Hence, each treatment will have a sample size of approximately 24,000 addresses. In total, approximately 96,000 addresses will be used for the three experimental treatments and the control treatment (current production). The current production treatment will have the same mail materials as the rest of production, but will be sorted and mailed at the same time as the other treatment materials. The remaining sample will receive production materials.

The Census Bureau proposes to evaluate treatment comparisons by comparing self-response rates at various points in the mailing schedule and by comparing the final response rates. The Census Bureau proposes comparing treatments at points in the mailing schedule where the material differs by design. For each comparison, α = 0.1 and a two-tailed test will be used so that the Census Bureau can measure the impact on the evaluation measure in either direction with 80 percent power. The effective samples were calculated based on the previous year's data for the August panel. The sample size will be able to detect differences of approximately 1.25 percentage points between the self-response return rates of the control and experimental treatments. Additional metrics of interest include overall costs and response rates by subgroups.

Affected Public: Individuals or households.

Frequency: One-time test as part of the monthly American Community Survey.

Respondent's Obligation: Mandatory.

Legal Authority: Title 13, United States Code, Sections 141, 193, and 221.

This information collection request may be viewed at www.reginfo.gov. Follow the instructions to view Department of Commerce collections currently under review by OMB.

Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to [e-mail] or fax to [phone].

Sheleen Dumas,

PRA Department Lead, Office of the Chief Information Officer.