[Federal Register Volume 62, Number 90 (Friday, May 9, 1997)]
[Proposed Rules]
[Pages 25578-25582]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 97-12191]
=======================================================================
-----------------------------------------------------------------------
POSTAL RATE COMMISSION
39 CFR Part 3001
[Docket No. RM97-2; Order No. 1174]
Amendment to Rules Concerning Evidence Based on Market Research
AGENCY: Postal Rate Commission.
ACTION: Notice of proposed rulemaking.
-----------------------------------------------------------------------
[[Page 25579]]
SUMMARY: The Commission proposes to amend its rules of practice by
clarifying foundational requirements for market research evidence. The
amendment entails three substantive revisions. One requires a survey
sponsor to provide additional supporting information about technical
aspects of the market research. Another provides for participants using
statistical techniques to limit the possibility of disclosing the
identity of a survey respondent and data uniquely associated with that
respondent. The third clarifies the level of access to data files and
computer programs that is to be provided, including the stage at which
rights to replication of survey results attach. These revisions will
clarify rule 31(k)(2)'s applicability to market research, thereby
reducing the need for case-by-case determinations and minimizing the
potential for delay in issuing Commission recommendations. The
amendment also makes minor editorial improvements in rule 31(k).
DATES: Comments responding to this notice of proposed rulemaking must
be submitted on or before June 9, 1997.
ADDRESS: Comments and correspondence should be sent to Margaret P.
Crenshaw, Secretary of the Commission, 1333 H Street NW, Suite 300,
Washington, DC 20268-0001 (telephone: 202/789-6840).
FOR FURTHER INFORMATION CONTACT: Stephen L. Sharfman, Legal Advisor,
Postal Rate Commission, 1333 H Street NW., Suite 300, Washington, DC,
20268-0001, (telephone: 202/789-6820).
SUPPLEMENTARY INFORMATION: Rule 31(k) outlines foundational
requirements for studies and analyses offered as evidence to be relied
upon in Commission proceedings. See 39 CFR 3001.31(k). Submissions
within the rule's purview are subject either to the terms of rule
31(k)(1) or to similar provisions tailored to specific types of
statistical studies and computer analyses. See generally rule 31(k)(2)
and (k)(3) (39 CFR 3001.31(k)(2) and (k)(3)). These requirements call
for a description of the study plan, relevant assumptions, data
collection techniques, the facts and judgments upon which conclusions
are based, alternative courses of action considered, and certain other
supporting information and data. This material must be filed along with
the related study or be produced upon request.
Rule 31(k) generally reflects settled evidentiary principles, but
persistent questions have arisen in recent proceedings about the impact
of certain market research practices on the interpretation of
paragraphs (k)(2) and (3). The debate has centered primarily on three
major--and often interrelated--concerns. One is the interest survey
sponsors assert in providing survey participants with reasonable
assurances that their participation in the survey and the sensitive
commercial data or information provided in their responses will not be
disclosed. Another is reviewers' interests in replicating survey
results and, in certain instances, in using a preferred method to
accomplish that end. A third issue is the impact of computer-assisted
data collection (CADC) techniques on compliance with rule 31(k)'s
requirement that ``input data'' be provided. See PRC Op. MC93-1, paras.
117-122; see also PRC Op. MC95-1, Appendix C. CADC techniques, in
particular, have altered some participants' expectations about how--and
whether--rule 31(k)'s data disclosure requirements apply to certain
market research efforts.
The Commission has resolved conflicts on a case-by-case basis, but
finds that revising the rule to provide participants with additional
guidance on how market research submissions should be supported is
warranted. PRC Op. MC95-1, Appendix C at 1-2. Having had an opportunity
to review pertinent issues and concerns outside the constraints imposed
by motion practice, the Commission has made preliminary determinations
about the manner in which rule 31(k) should be revised.
Postponement of Comprehensive Revisions
While developing an amendment to address problematic aspects of
market research submissions, the Commission also considered proposing a
comprehensive reorganization of rule 31(k). Structurally, a
comprehensive review would permit redundant or overlapping requirements
to be modified or eliminated, thereby simplifying a rule that has been
the subject of several amendments. In addition, consideration could be
given to whether the numerous provisions now covered in paragraph (k)
should continue to be located within rule 31, which is an umbrella
evidentiary provision, or whether they should form an independent
provision.
Substantively, a broader focus would provide an opportunity to
address the advisability of maintaining certain formal distinctions
within the rule, such as a special set of requirements for computer-
based studies. Computer-specific provisions were added to the rule (in
subparagraph (3)) in the early 1980s, when the use of computers to
prepare or develop evidence was in its infancy and several related
evidentiary issues were unsettled. Since then, computer use has become
routine not only for submissions covered by rule 31(k)(3), but in the
preparation of nearly every filing in Commission proceedings. Thus, the
rule's underlying orientation may warrant reconsideration. At a
minimum, revisions to subparagraph (3) and conforming changes in other
provisions may need to be made. See generally Docket No. RM81-1, Notice
of Proposed Rulemaking (NPRM) at 5. Similarly, the need for
distinctions between studies that involve statistics and those that do
not could be reviewed. Moreover, a broader approach might allow issues
related to the emergence of electronic data bases, from which a number
of different studies and analyses can be developed, to be explored.
The Commission's interest in updating and reorganizing the rule is
tempered with a concern that wholesale revision might unduly delay
addressing the questions that have surfaced about market research.
Based on this consideration and an assessment that other aspects of the
rule appear to be working reasonably well, the Commission has decided
to propose only limited changes now. Accordingly, the proposed
amendment has been drafted to conform, as closely as possible, to the
existing approach and to cause minimal disruption to the current
numbering system. Structurally, this is accomplished primarily by
distinguishing market research from other sample surveys, with the
requirements specific to market research designated as rule
31(k)(2)(i). Existing rule 31(k)(2)(i), now entitled ``Sample
surveys,'' is renamed ``Other sample surveys'' and redesignated as rule
31(k)(2)(ii). Conforming numbering changes are also made to other
subparagraphs of the rule. Substantively, the Commission notes that
this amendment is not inconsistent with its recent statement, in PRC
Op. MC96-3, that the existing rules on sample surveys require certain
quantitative disclosures. See generally PRC Op. MC96-3 at 37-38. Given
its general position on the scope of the existing sample survey
requirements, the Commission is not proposing to make them more
explicit at this time.
Expanded Foundational Requirements for Market Research-Based
Submissions
Rule 31(k) now provides, in subparagraph (2)(i)(a), that a
proponent of a sample survey is to provide a clear description of the
survey design, including the definition of the universe
[[Page 25580]]
under study, the sampling frame and units, and the validity and
confidence limits that can be placed on major estimates. The rule also
provides, in subparagraph (2)(i)(b), that the survey sponsor provide an
explanation of the methods of selecting the sample and the
characteristics measured or counted.
These provisions generally provide a straightforward and
serviceable base for evaluating sample surveys. However, given the
growing importance of market research in Commission proceedings, it
appears advantageous to be more specific about the detail sponsors
should provide at the time the market research (or the submission it
supports) is filed. This also is consistent with the Commission's view,
previously expressed in connection with survey replication, that
providing descriptions of technical procedures can provide reviewers
with the ability to make an assessment, from the description itself, of
the appropriateness of various standard procedures. See PRC Op. MC95-1,
Appendix C at 7.
The Commission therefore proposes to specify, in
Sec. 3001.31(k)(2)(i)(a)(1), that the foundational requirements include
details of the sampling, observational, and data preparation designs,
with definitions of the target population, sampling frame, units of
analysis and survey variables. These requirements also include an
explanation of the methodology for the production and analysis of the
major estimates and the associated sampling errors. Proposed
Sec. 3001.31(k)(2)(i)(a)(2) requires that the proponent not only
provide measures of sampling error, but also present coverage, response
and editing rates.
In addition to these changes, which are primarily adaptations of
existing requirements for sample surveys, the Commission is also
proposing four new foundational requirements. Proposed subparagraph
(a)(3) requires a discussion of data comparability over time and with
other data sources, and the effects of benchmarking and revisions.
``Benchmarking,'' in this context, refers to establishing an acceptable
standard by which to evaluate estimates. Subparagraph (a)(4) requires
an assessment of the effects of editing and imputation and other
potential sources of error on the quality of the survey estimates.
Subparagraph (a)(5) requires identification of applicable statistical
models when model-based procedures are employed. Finally, subparagraph
(a)(6) requires an explanation of all statistical tests performed and
an appropriate set of summary statistics summarizing the results of
each test.
Confidentiality
Confidentiality issues have dominated recent motion practice, and
they have been a concern for some time. In Docket No. RM81-1, for
example, the American Bankers Association (ABA) filed comments noting
that compliance with a requirement of producing actual input data upon
request could pose difficulties because of confidentiality promises. In
evaluating ABA's position, the Commission noted that the provision in
question was not a new element of the proposal under consideration, but
had been in effect for several years without creating serious
difficulties. The Commission concluded, at that time, that it preferred
to continue its practice of addressing special needs for
confidentiality as they arose, rather than alter the general rule to
meet exceptional cases. Docket No. RM81-1, Final (Rulemaking) Notice at
11. In PRC Op. MC95-1, however, the Commission clearly signaled its
interest in ending this practice by stating that it intended to
institute a rulemaking and explore whether a widely-applicable standard
or policy statement on the confidentiality of market research data and
information could be developed.
The Commission has made a preliminary determination that the
continuing motion practice on this topic confirms the need for a
revision to its existing practice, and that statistical disclosure
limitation (SDL) methods provide a workable, objective standard. SDL
methods are techniques that limit the risk of disclosure of individual
information when statistics are disseminated in tabular or microdata
formats. These practices are not new, but have been developed and
implemented by various federal agencies over the past 25 years. See
generally Jabine, Thomas B., ``Statistical Disclosure Limitation
Practices of United States Statistical Agencies,'' Journal of Official
Statistics, Vol. 9, No. 2 (1993) at 427-454.
In conjunction with this rulemaking, the Commission is
establishing, for participants' convenience, Library Reference PRC-LR-
1, containing Statistical Policy Working Paper 22, ``Report on
Statistical Disclosure Limitation Methodology'' (May 1994), (hereafter,
Working Paper). The report was prepared by the Subcommittee on
Disclosure Limitation Methodology of the Federal Committee on
Statistical Methodology, which is associated with the Statistical
Policy Office of the Office of Information and Regulatory Affairs of
the Office of Management and Budget. The preface indicates that the
report includes a tutorial, guidelines, various recommendations, and an
annotated bibliography. The Commission notes that the report
specifically indicates that legal questions are beyond its scope.
Working Paper at 2. An excerpt on survey research from the ``Reference
Manual on Scientific Evidence,'' published by the Federal Judicial
Center in 1994, is also included in the library reference.
The specific provision the Commission proposes adding, in new rule
31(k)(2)(i)(b) is: ``Protection against disclosure of sensitive data
should be provided through the application of appropriate statistical
disclosure limitation (SDL) practices when data are produced for
secondary analysis.'' The rule indicates that SDL practices include the
following: Removal of respondent identifiers from microdata files; cell
concentration and suppression rules; and data masking through
aggregation, random noise injection, and simulation of artificial
records.
In the sense used in the rule, a microdata file consists of
individual records, each containing values of variables for a single
person, business establishment or other unit. Id. at 3. There are no
identifiers on the file, and the data may be disguised in some way to
ensure that the individual data items cannot be uniquely associated
with a particular respondent. Id. at 6. Cell concentration means that a
specific number of cases in a given cell of a data table cannot account
for a percentage of the cell total equal to or exceeding a prescribed
threshold; that is, the (n,p) cell concentration rule is violated if n
or fewer respondents account for at least p percent of the total cell
value. If this rule is violated, the cell is suppressed or collapsed
with other cells to reduce the risk of disclosure. Data masking entails
distorting data prior to its release or limiting the amount of data
released. It can involve random error (noise) to the data entries,
multiplying the data by random values from known distributions, or data
swapping. The latter refers to the practice of interchanging the values
for survey items of sample cases having similar characteristics or
values for auxiliary variables.
The proposed rule also provides that statistical disclosure is
defined as the identification of the respondent or the linking of a
respondent to sensitive data in a survey record or data file. The
revised rule also affirmatively states that under certain conditions,
the post-SDL data shall be the starting point for an evaluation on the
merits.
The Commission recognizes that its endorsement of SDL techniques as
a means of limiting disclosure
[[Page 25581]]
presupposes certain expertise on the part of both survey sponsors and
survey reviewers. However, the complexity of the surveys that have been
at the center of recent motion practice indicate that the sponsor (in
most cases, the Postal Service) would have access to the resources
needed to meet the rule's standard. To the extent that survey reviewers
might need assistance in understanding the discovery implications of
the SDL techniques, technical conferences or other forms of assistance
can be made available.
Reviewers' Access to Data, Including Replication of Survey Results
As with confidentiality, replication of survey results also has
been an issue over the course of several proceedings. In Docket No.
RM81-1, for example, the Postal Service questioned whether the word
``replicate'' in the proposed rule imposed too broad a standard. The
Commission concluded that it did not, emphasizing that the fact that
this term was not explicitly used in the final rule reflected a
technical drafting decision, rather than a substantive change in
position. The Commission said: ``We think it is clear--even without
express use of this term--that the final rule allows a participant,
upon proper request, to obtain materials that would allow replication
of the results of computer-generated presentations.'' Final
(Rulemaking) Notice at 13.
By Docket No. MC95-1 the question of what ``materials'' should be
made available had come to the fore, and the Commission noted that
newer market research techniques complicated the issue. It cautioned:
(P)articipants' insistence on the ability to trace a numerical
result from CADC market research to the primary data source by
replicating various data adjustments may often be very impractical,
and sometimes simply impossible. The task of verifying a specific
numerical result could, in itself, entail running a rather extensive
set of complex computer programs associated with the survey's data
collection, editing, coding, estimation and analysis procedures.
PRC Op. MC95-1, Appendix C at 6. In a related comment, the
Commission noted that it viewed the overall objective of the rule as
``* * * placing reviewers in a position to determine whether the data
are sufficiently accurate to satisfy the (evidentiary) standards the
Commission must apply.'' Id.
The Commission also noted that one difficulty in resolving disputes
is that several key terms in rule 31(k) are not defined. The Commission
suggested that consideration should be given to whether inclusion of a
set of definitions or guidelines for interpretation might be useful. At
this time, the Commission has decided against including definitions of
the proposed SDL techniques in the rule, but is clarifying the meaning
of ``input file.'' In addressing participants'' uncertainty over the
meaning of this term, the Commission has noted previously that an
``input file'' can be any data set that is entered into a statistical
program or package designed for a specific purpose, and that it is
therefore
* * * unlikely that the ``raw data'' and the ``input data'' for
adjustment and estimation programs would be coincidental in a
moderate-to-large survey research effort. This is because the raw
data are usually modified to some extent--even if no more than
recoded--before they are entered in a database.
PRC Op. MC95-1, Appendix C at 4. To address this situation, the
Commission proposes a new provision--in rule 31(k)(2)(i)(b)--providing
that access ``shall be sufficient to permit the replication of
electronic data processing after production of the edited data file.''
The term ``edited data file'' is defined in the rule as raw data after
appropriate coding, editing for consistency checks and application of
SDL methodology.''
Availability of Opportunity To Request Waiver
Assuming adoption of the proposed amendment, the Commission expects
participants to exercise all reasonable efforts to comply with its
terms, including the use of SDL methods. To the extent a participant
believes it cannot do so, but nevertheless seeks evidentiary status for
affected submissions, a separate rule of practice--rule 22--provides an
opportunity to seek waiver, in whole or in part, by filing a timely
motion. See 39 CFR 3001.22. Waiver is conditioned on a showing that the
interests of other participants will not be unduly prejudiced, and that
it is consistent with the public interest and the Commission's
discharge of its responsibilities. Given these conditions, the
Commission expects that a participant seeking relief from application
of the new requirements would propose, at a minimum, alternative means
of satisfying the interests sought to be protected by rule 31(k).
Limited Editorial Improvements
The Commission is also proposing limited editorial improvements in
Sec. 3001.31(k)(3) at this time. One entails the proposed deletion of a
citation to outdated software documentation standards. These standards
were current when the related text was added to rule 31(k) in the early
1980s, but are now seriously outdated and, in some instances, out of
print. The Commission has considered, but rejected, replacing these
references with more current standards, on the assumption that
participants no longer need to be provided with examples of
documentation. Thus, the proposed amendment eliminates the footnote
citation associated with the word ``standards'' in the main text of
rule 31(k)(3)(i)(e) and deletes the related footnote in its entirety.
The Commission also considered replacing the reference to ``magnetic
tape'' in rule 31(k)(3)(i)(i) with a more generic term or phrase, but
instead decided to change it to ``a compact disc.'' The Commission
invites comments on retaining the reference in the same provision to a
time-sharing service.
Comments
To assist commenters in preparing a response to this proposal, the
Commission reiterates its conscious decision to keep the focus of this
rulemaking comparatively limited. Thus, this proposal addresses the
existing sample survey provisions only in the sense of their
application to market research. Within this framework, commenters are
invited to submit comments addressing pertinent issues. In particular,
the Commission welcomes attention to the following matters:
Whether participants anticipate difficulties in employing
SDL methods and, if so, what these might be;
Whether participants are aware of any supplementary
methods or approaches that could or should be included in the rule;
Whether the general availability of an opportunity to
request waiver under rule 22 is sufficient, or whether waiver should be
further conditioned or restricted through express language in rule
31(k)(3);
Whether the definitions in this Notice of Proposed
Rulemaking provide participants with sufficient information on SDL
techniques;
Whether the Commission's assumption that a reference to
specific software standards is no longer needed is correct, or whether
the standards should be updated; and
Whether other minor editorial revisions in rule 31(k) are
necessary or desirable at this time, and can be incorporated with
minimal disruption.
List of Subjects in 39 CFR Part 3001
Administrative practice and procedures, Postal Service.
[[Page 25582]]
For the reasons set forth in the preamble, 39 CFR Part 3001 is
amended as follows:
PART 3001--RULES OF PRACTICE AND PROCEDURE
1. The authority citation for 39 CFR part 3001 continues to read as
follows:
Authority: 39 U.S.C 404(b), 3603, 3622-24, 3661, 3662.
2. 39 CFR 3001.31(k) is amended as follows:
3. Redesignate paragraph (k)(2)(i) through (iii) as (k)(2)(ii)
through (iv).
4. Amend redesignated paragraph (k)(2)(ii) by changing the title
from Sample surveys to Other sample surveys.
5. Add paragraph (k)(2)(i) to read as follows:
Sec. 3001.31 Evidence
* * * * *
(k) Introduction and reliance upon studies and analyses--(1) * * *
(2) * * *
(i) Market research. (a) The following data and information shall
be provided: (1) A clear and detailed description of the sampling,
observational, and data preparation designs, including definitions of
the target population, sampling frame, units of analysis, and survey
variables;
(2) an explanation of methodology for the production and analysis
of the major survey estimates and associated sampling errors;
(3) a presentation of response, coverage and editing rates;
(4) a discussion of data comparability over time and with other
data sources, and the effects of benchmarking and revisions;
(5) an assessment of the effects of editing and imputation and
other potential sources of error on the quality of the survey
estimates;
(6) identification of applicable statistical models, when model-
based procedures are employed;
(7) an explanation of all statistical tests performed and an
appropriate set of summary statistics summarizing the results of each
test.
(b) Upon request, access to data files and computer programs shall
be provided. Access shall be sufficient to permit replication of
results after development of the edited data file. For purposes of this
subparagraph, the phrase ``edited data file'' refers to raw data after
appropriate coding, editing for consistency checks and application of
statistical disclosure limitation methods (SDL) methods.
(c) Protection against disclosure of confidential commercial data.
(1) If the recipient of a request for data pursuant to this paragraph
asserts that compliance with the request would conflict with a
confidentiality agreement, the recipient shall be expected to employ
SDL methods to protect against the disclosure of confidential
commercial data. The SDL method(s) selected shall not interfere with
other reasonable or expected uses of the data.
(2) For purposes of this subparagraph, SDL methods include the
removal of respondent identifiers from microdata files; cell
concentration and suppression rules; and data masking through
aggregation, ``random noise'' injection, and simulation of artificial
records. Statistical disclosure means the identification of the
respondent or the linking of a respondent to sensitive data in a
tabular presentation, survey record or data file.
(3) If the results or conclusions reached after application of the
SDL method(s) differ materially from those reached prior to such
application, the post-SDL data shall be deemed controlling for purposes
of the sponsoring party's evidentiary presentation and related legal
argument.
6. Revising paragraph (k)(3)(i)(e) to read as follows:
* * * * *
(k) * * *
(3) * * *
(i) * * *
(e) For all source codes, documentation sufficiently comprehensive
and detailed to satisfy generally accepted software documentation
standards appropriate to the type of program and its intended use in
the proceeding.
7. Revise the first sentence of the concluding text after paragraph
(k)(3)(i)(i) to read as follows:
* * * * *
(k) * * *
(3) * * *
(i) * * *
(i) * * *
Paragraphs (k)(3)(i)(d) and (f) of this section shall be provided
either in the form of a compact disc or through access to a time-
sharing service, at the option of the provider. * * *
Issued by the Commission on May 2, 1997.
Margaret P. Crenshaw,
Secretary.
[FR Doc. 97-12191 Filed 5-8-97; 8:45 am]
BILLING CODE 7710-FW-P
