02-13458. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the National Aeronautics and Space Administration

AGENCY:

National Aeronautics and Space Administration (NASA).

ACTION:

Notice of guidelines and request for comments.

SUMMARY:

Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Pub. L. 106-554) directed the Office of Management and Budget (OMB) to issue government-wide information quality guidelines. OMB's final guidelines, re-published on February 22, 2002, require each Federal agency to issue Agency-specific implementing guidelines for ensuring the quality of disseminated information. The National Aeronautics and Space Administration (NASA) is seeking comments on its draft information quality guidelines. The draft sets out guidelines for ensuring the quality, objectivity, utility, and integrity of NASA's information and describes an administrative mechanism for seeking correction of information publicly disseminated by NASA.

DATES:

Written comments regarding NASA's draft information quality guidelines must be submitted on or before 30 days after date of publication in the Federal Register.

ADDRESSES:

Comments should be sent to Nancy R. Kaplan, Code AO, National Aeronautics and Space Administration, Washington, DC 20546-0001. Comments may also be e-mailed to [e-mail].

FOR FURTHER INFORMATION CONTACT:

Nancy R. Kaplan, Code AO, National Aeronautics and Space Administration, Washington, DC 20546-0001. Telephone: [phone].

SUPPLEMENTARY INFORMATION:

National Aeronautics and Space Administration Draft Guidelines for Ensuring the Quality of Information

A. Purpose

Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Pub. L. 106-554; H.R. 5658; hereafter referred to as Section 515) directed the Office of Management and Budget (OMB) to issue government-wide information quality guidelines. OMB's final guidelines, entitled “Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies,” were re-published on February 22, 2002 (67 FR 8451.) The OMB guidelines require each Federal agency to issue their own, Agency-specific, implementing guidelines for ensuring the quality of disseminated information.

This document outlines the National Aeronautics and Space Administration's (NASA's) information quality guidelines; details corresponding procedures, administrative mechanisms, and reporting requirements; and establishes NASA's responsibilities for ensuring that its information adheres to the quality guidelines. Included in this document are the procedures for affected persons to seek and obtain correction of information disseminated by NASA.

B. Background

Section 203(2)(3) of the National Aeronautics and Space Act, Public Law 85-568, as amended, chartered NASA to “provide for the widest practicable and appropriate dissemination of information concerning its activities and the results thereof.” NASA makes available a diverse wealth of information to government, industry, academia, and the public. Some examples include scientific and technical information from its world-class research and operational programs, such as reports, journal articles, data, and imagery; information concerning its current vision, mission, goals, programs, and performance, such as performance plans and reports; information regarding the missions it aspires to pursue, such as strategic plans; and educational information, such as curricula, lesson and technology plans, and educational briefs, for K-12 through post-graduate students.

Information from NASA's missions and programs is used by a number of organizations and individuals including, but not limited to, government, national, and international policymakers formulating public policy; NASA's scientists and others cooperating with NASA to pursue their important work; the media reporting on the importance of NASA's research; the educational community educating a new generation of citizens in science, math, and engineering; and members of the public learning about NASA's goals and accomplishments.

C. Policy and Procedures

C.1. Scope

These guidelines are applicable to NASA Headquarters and Centers, including Component Facilities; and to the Jet Propulsion Laboratory and other contractors where specified by contract. They prescribe policy and procedures for a wide variety of dissemination media, such as printed, electronic (including websites), and other forms of publication.

The definitions of information, dissemination, quality, and other terms used within this document were adapted from those established by OMB in its government-wide quality guidelines. Where appropriate, NASA has expanded on the OMB definitions to provide guidance that is more applicable to Agency-specific information.

The guidelines for pre-dissemination review in this document shall apply to information that NASA first disseminates on or after October 1, 2002. Other guidelines in this document shall pertain to information disseminated on or after October 1, 2002, regardless of when it was first disseminated by NASA.

C.2. Guidelines

NASA will ensure and maximize the quality, including the utility, objectivity, and integrity, of its disseminated information, except where specifically exempted. Categories of information that are exempt from these guidelines are detailed in Section C.3.

NASA's “disseminated information” includes any communication or representation of knowledge, such as facts or data, conveyed in any media or form, such as textual, numerical, graphic, cartographic, narrative, or audiovisual, whether on paper, film, or electronic media, and whether disseminated via formal publication, recording, machine-readable data, or website.

C.2.a. Basic Standard of Information Quality

This section outlines the basic standard of information quality that NASA's disseminated information must meet. NASA will treat information quality as integral to every step of its development of information, including creation, collection, maintenance, and dissemination.

A level of information quality assurance greater than the basic standard is required in those situations that involve influential scientific, financial, or statistical information. The quality standard for influential information is defined in Section C.2.b. Additionally, principles of information quality beyond the basic standard may be adopted as appropriate for specific categories of NASA's disseminated information. Section C.2.c outlines principles of information quality that may apply to certain categories of NASA's information.

The basic standard of information quality, for the purposes of these guidelines, has three components: utility, objectivity, and integrity. The guidelines sometimes refer to these terms collectively as “quality.” In ensuring the quality of its disseminated information, NASA must ensure that all of these components are sufficiently addressed.

C.2.a.1. Utility. The measure of utility refers to the extent that the information can be used for its intended purpose, by its intended audience. The following principles relate to these dimensions of information utility:

Intended Purpose

• To provide useful, relevant information, NASA will stay informed about the information needs of its stakeholders and develop new data, models, and information where appropriate.
• When currency of information is critical, NASA will ensure that relevant information is made available in a timely manner and updated as appropriate.
• NASA's information will be reviewed by content owners, at a frequency appropriate to the type of information, to ensure that it remains relevant and timely.

Intended Audience

• NASA's information dissemination process will make the Agency's information widely available and broadly accessible, as appropriate and practical for the target audience.
• NASA will ensure that its information is accessible to all potential users, including individuals with disabilities, per Federal law, statute, and Agency guidance.

C.2.a.2. Objectivity. The measure of objectivity refers to the extent that the information is accurate, clear, complete, and unbiased. The following principles relate to these dimensions of information objectivity:

Accuracy

• Information disseminated by NASA will be based on reliable, accurate data that has been validated.
• NASA's information will be proofread before release to ensure that they are free from typographical and grammatical errors.
• Where feasible and appropriate, NASA will inform users of corrections to the Agency's information resulting from discovery of errors.

Clarity

• NASA's information will be reviewed before release to ensure clarity and coherence of the material presented.

Completeness

• NASA's information will include, to the extent feasible, the proper context to ensure completeness of the material presented.
• Where feasible, data presented by NASA will have full and accurate documentation, and circumstances affecting data quality will be identified and disclosed to users.

Lack of Bias

• NASA will utilize systematic analysis and review processes to remove potential biases from its information.
• To the extent possible, NASA will ensure that information is presented without the appearance of bias.

C.2.a.3. Integrity. The measure of integrity refers to the protection of NASA's information from unauthorized access, revision, modification, corruption, falsification, and unintentional or inadvertent destruction. The following principles relate to information integrity:

• NASA employees responsible for classified information, draft materials, and otherwise sensitive information will utilize appropriate security controls and mechanisms to protect the information from improper dissemination.
• When information integrity has been compromised, NASA will take immediate steps to remedy the situation and facilitate correction of the compromised information.

A key aspect of information integrity is ensuring that NASA's computer systems remain protected from unauthorized access or other threats that could damage the information residing therein. NASA's Information Technology (IT) Security Program is the responsibility of the NASA Chief Information Officer (CIO). The roles and responsibilities of the CIO with respect to IT Security are outlined in detail in NASA Procedures and Guidelines (NPG) 2810.1, “Security of Information Technology.”

C.2.b. Quality Level for Influential Information

NASA requires a higher standard of quality for information that is considered influential. Influential scientific, financial, or statistical information is defined as NASA information that, when disseminated, will have or does have clear and substantial impact on important public policies or important private sector decisions.

Each NASA organizational director will be responsible for determining which of its disseminated information falls into this limited category. Where information is considered influential, the responsible organization shall document the safeguards and policies that are in place to ensure the quality (utility, objectivity, and integrity) of the information.

OMB requires more stringency for ensuring the quality of influential scientific, financial, or statistical information. For these categories of influential information to be considered compliant with quality guidelines, the information must be transparent and reproducible to the greatest possible extent (see C.2.b.1 and C.2.b.2 for definitions of these terms). It is important to note that applying the reproducibility standard to all influential data may not be practical or warranted; i.e., where it may be impractical or unethical to duplicate the circumstances of an experiment or investigation.

Principles related to ensuring the transparency and reproducibility of information are outlined below.

C.2.b.1. Transparency. The measure of transparency refers to the extent that information, particularly that of a scientific or statistical nature, has supporting data documented and made available.

• In disseminating information of an influential nature, NASA will specifically describe the data used, the various assumptions employed, the specific analytic methods applied, and the statistical procedures utilized.

C.2.b.2. Reproducibility. The measure of reproducibility refers to the extent that the information is capable of being substantially reproduced, subject to an acceptable degree of imprecision. In other words, independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable margin of error.

• Each NASA organization will be responsible for determining which categories of original and supporting data will be subject to the reproducibility requirement.
• NASA will make the information it disseminates and the methods used to produce this information as transparent as possible so that they can, in principle, be reproducible by qualified individuals.
• When it is not practical to apply the reproducibility standard to data or information, NASA will ensure greater transparency of the methods used to produce the data or information.

C.2.c. Principles for Specific Categories of Information

OMB's information quality guidelines encourage Federal agencies to address principles of quality for specific categories of information that they produce. NASA's experience has been that the information used in conducting the Agency's daily business falls into five categories, as documented in NPG 2810.1.

NASA will ensure the quality of information in each information category by adhering to the key principles outlined below.

C.2.c.1. Mission Information

This category consists of information that directly supports NASA's human space flight, launch operations, space vehicle operations, wind tunnel operations, training simulation vehicles, and other mission-related activities.

• NASA will use special protections to preserve its mission information from alteration or destruction, particularly where proprietary or sensitive information is involved.
• NASA will exercise special care in handling, disseminating, and ensuring the protection of information pertaining to missions involving human life.
• NASA will protect information related to individuals involved in NASA's missions, per the requirements of the Privacy Act of 1974 (5 U.S.C 552A, as amended.)

OMB's guidelines require special considerations for analysis of risks to human health, safety, and the environment. OMB directs agencies to adopt or adapt the quality standards contained in the 1996 amendments to the Safe Drinking Water Act for analysis of these types of risks. With respect to information in this category, NASA will ensure that it has analyzed and/or documented, to the extent practical:

• Each population addressed by any risk estimate and the expected risk for each population;
• Acceptable upper and lower bounds of risk;
• Uncertainties identified during the risk assessment process and how the uncertainties were or will be addressed;
• Peer review studies related to risk estimates;
• Methodologies used to reconcile inconsistencies in the scientific data.

C.2.c.2. Business and Restricted Technology Information

This category consists of information related to financial, legal, payroll, personnel, procurement, source selection, and other business and restricted technology activities. NASA is required by law to protect much of the information in this category.

• NASA will ensure that categories of information requiring protection or restricted access under law or statute (i.e., Export Administration Regulations and International Traffic in Arms regulations) are appropriately handled and protected from inappropriate dissemination.

C.2.c.3. Scientific, Engineering, and Research Information

This category consists of information that supports basic research, engineering, and technology development, but that is less protected than mission information.

OMB's guidelines give special consideration to scientific, technical, and statistical information. OMB regards information in this category that has been subject to formal, independent, external peer review as presumptively objective and therefore of higher quality. With respect to NASA's peer reviewed scientific, engineering, and research information, the following principles for ensuring information quality apply:

• NASA will ensure that peer reviews conducted by the Agency are performed in an open and rigorous manner
• Reviewers in NASA-sponsored peer reviews will be selected on the basis of technical expertise and will be requested to disclose prior technical or policy positions that may affect the issues at hand and to disclose sources of personal and institutional funding that may affect or appear to affect their technical judgment.

It is important to note that some types of scientific, engineering, and research information disseminated by NASA may be exempt from NASA's information quality guidelines. Specifically, when scientists and researchers use the “academic process” to communicate their findings, i.e., through conference presentations and papers, peer reviewed journal articles, peer reviewed summary and assessment reports, and other dissemination practices that are standard in the research community, their research data, conclusions, and results may not represent an official product or position of the Agency. If this is the case, the information disseminated should clearly indicate via a disclaimer or other means that the views expressed are the author's, and not necessarily those of NASA. More specifics about this exception are outlined in Section C.3, Exempted NASA Information.

C.2.c.4. Administrative Information

This category consists of information such as electronic or written correspondence, briefing information, program/project status documents, organizational documentation, strategic plans, and other information of an administrative or general nature.

• NASA will ensure that administrative information is reviewed regularly to ensure its continued relevance and accuracy.

C.2.c.5. Public Access Information

This category consists of information that is intended for public use, such as material related to NASA's educational programs.

• NASA will ensure that its key information is made available to the general public through the widest possible dissemination.
• NASA will carefully review references and links to external sources of information to ensure that they are business related and will not lead to an apparent conflict of interest, inappropriate endorsement, or embarrassment to the Agency.

C.3. Exempted NASA Information

The OMB information quality guidelines permit exceptions for certain types of information. These categories of information do not have to meet a minimum standard of information quality.

The biggest category of information that is exempt from this policy is information that is disseminated by but neither authored by NASA nor adopted as representing NASA's views. This category includes, but may not be limited to:

• Information communicated by scientists and researchers via the “academic process” (as defined in Section C.2.c.3);
• Information that is funded by NASA but published by a contractor, grantee, or other government organization without NASA's direction.

The following types of information dissemination are also exempted from this policy:

• Information in which distribution is limited to government employees, Agency contractors, or grantees, including intra-agency use or sharing of information;
• Responses to requests for Agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act (FACA), and other applicable laws and regulations;
• Correspondence with individuals or persons;
• Press releases;
• Public filings, subpoenas, or other adjudicative processes;
• Archival information;

C.4. Ongoing Process for Ensuring NASA's Information Quality

NASA currently has a number of policies and processes in place to ensure that information produced and disseminated by the Agency meets a basic level of quality. Much of the information that NASA issues in the Agency's name, uses to support policy, or utilizes to reach mission decisions is subject to independent, external peer review, and the remainder is generally subject to one or more levels of quality review.

The review and approval process for NASA's disseminated information will be documented as much as possible and practical. The level of documentation will be commensurate with the importance of the information.

Some of the review processes utilized by NASA are described below. These review processes are utilized at the discretion of the organizations that produce NASA's content, depending on the type of information, intended audience, and other factors relevant to the situation.

Editorial Review

Much of NASA's information is subject to editorial review by a qualified technical editor or other professional. The editorial review ensures that spelling, grammatical, and punctuation errors are discovered and corrected before an information product is disseminated.

Compliance Review

The author, technical monitor, or other NASA official responsible for an information product will ensure that, when appropriate, the information is reviewed for compliance with Federal law, statute, and NASA policy. NASA's information may be subject to limited dissemination if export control limitations, International Traffic in Arms Regulations, confidentiality considerations, proprietary or copyright concerns, or other circumstances dictate the information's protection.

Content Review

NASA's information is subject to content review to ensure its quality and integrity. The author, content owner, or other NASA official responsible for an information product ensures that content reviews are conducted before the information is disseminated. As described in NPG 2200.2, “Management of NASA Scientific and Technical Information,” scientific and technical information undergoing formal publication by NASA is subject to review before release. These reviews assess the quality of the information product in terms of readability, its communication of information, and its suitability for a particular audience.

Peer Review

The use of peer review helps NASA to ensure the quality of its information. In general, NASA evaluates program merit and priorities on the basis of peer review and advice from committees broadly representative of NASA's customers. NASA strives to form diverse, expert review panels that encompass the full range of scientific and technical expertise required.

While the general principle regarding the use of peer review applies across the Agency, there is not a uniform peer review process for all types of information, and not all of NASA's information requires peer review. Different approaches are warranted by differences in goals, customer base, etc. among the various disciplines.

Other Review Processes

NASA Enterprise, Center, Mission, Program, Project, or other organizational managers may establish and apply their own guidelines related to the quality review and dissemination of their own information. This is acceptable as long as the component organizations' guidelines do not conflict with the Agency's information quality guidelines.

D. Administrative Mechanisms

The NASA CIO will establish administrative mechanisms allowing affected persons to seek and obtain, where appropriate, timely correction of information maintained and disseminated by the Agency if the information, upon further review, does not comply with NASA's quality standards. The administrative mechanisms are intended to be flexible, appropriate for the nature of NASA's information dissemination activities, and complementary to NASA's existing information resources management and administrative practices.

For the purposes of these guidelines, affected persons are defined as persons who may benefit from or be harmed by the disseminated information. The term persons includes groups, organizations, and corporations as defined by the Paperwork Reduction Act (PRA) of 1995.

NASA will address genuine and valid needs of its users without disrupting Agency processes. NASA can reject claims made in bad faith or without justification, and can decide upon and undertake the degree of correction deemed appropriate to fit the nature and timeliness of the information involved.

D.1. Requesting Correction of Information by NASA

If an affected person believes that information disseminated by NASA does not meet the guidelines for quality (utility, objectivity, and integrity), he or she may seek correction of the information.

Requestors wishing to seek correction of information under NASA's information quality guidelines must follow the procedures outlined below. These procedures apply only to requests for the correction of information relevant to the information quality guidelines.

• Requests must be in writing, and may be submitted by regular mail, electronic mail, or fax. (Final guidelines will include explicit submission mechanisms, such as addresses)
• Requests must indicate that the correction of information is requested under NASA's information quality guidelines.
• Requests must include the requestor's name, phone number, preferred mechanism for receiving a written response from NASA (fax, e-mail, regular mail) with applicable contact information, and organizational affiliation (if any).
• Requests must clearly describe the information that the requestor believes needs correcting, and include the name of the report or information source, the location if electronic, and the date of issuance.
• Requests must indicate how the requestor is an affected person for the purposes of these guidelines (as defined in Section D, Administrative Mechanisms, and Section F, Definitions).
• Requests must state specifically what information should be corrected and what changes to the information, if any, are proposed. If possible, provide supporting evidence to document the claim.

The NASA CIO will have the responsibility for receiving suggestions for correction of information. The CIO will coordinate with such Agency officials as appropriate, including technical experts, content owners, legal counsel, and others, to determine whether or not to correct the information.

In its review, NASA will determine if the information in question does not meet the appropriate quality standards and needs to be corrected. The review of the information will be limited to that part or parts of the information that are indicated to be in error.

If NASA decides that correction of the information is warranted, NASA will correct the information in accordance with existing statutes, regulations, and procedures. The NASA CIO will inform the requester in writing of the decision and the action taken.

If NASA decides not to correct the information, the requester shall be informed promptly in writing by the CIO of the decision not to correct the information, the reason for refusal, the date of the refusal, and the opportunity for appeal.

NASA will respond to a request for correction of information within 60 calendar days of receipt of the information. NASA may extend the 60-day response period if additional time is required to review the request for correction of information. NASA will contact the requestor if an extension of response time is needed, and will indicate the reason for the delay in responding and an estimated decision date.

NASA may reject a request for information correction without taking action on it if NASA determines that:

• The requestor is not an affected person (as defined in Section F., Definitions);
• The information required to process a review is not provided in full;
• The request for correction is frivolous.

The NASA CIO will maintain file records of each request for information correction, including copies of the original request, the response from NASA, and notification to the requestor of NASA's decision and action taken.

D.2. Appeal Process

If a requestor disagrees with NASA's decision, he or she may file an appeal in writing within 30 calendar days of the decision. [Final guidelines will include explicit appeal submission mechanisms] The request for appeal will be considered by an internal review panel, convened by the CIO. The exact membership of the appeals panel will vary depending on the specifics of the information under review, but will include representatives from appropriate scientific and technical, legal, policy, and other functional areas as needed. The appeals panel will not include any personnel who were involved in the original review of the correction request.

If, after review, the appeals panel determines that the original decision should be overturned, the appeals panel will notify the CIO, who will in turn advise the requestor of NASA's decision. If applicable, NASA will then correct the information in accordance with existing statutes, regulations, and procedures. If the appeals panel determines that correction of the information is not warranted, the CIO will advise the requestor of the denial and the reason and authority for the denial.

All appeals will be processed within 30 calendar days unless NASA determines that a fair review cannot be made within this time frame. NASA will contact the requestor if an extension of response time is needed, and will indicate the reason for the delay in responding and an estimated decision date.

The NASA CIO will maintain file records of each appeal request, the response from NASA, and notification to the requestor of the appeal decision.

E. NASA Reporting Requirements

Pursuant to OMB requirements, the NASA CIO will submit an annual report on the number and nature of complaints received by the Agency regarding the accuracy of the information it disseminates. The report will contain, as appropriate, both quantitative and qualitative information about the complaints received, the resolution of the complaints, and the number of NASA staff hours that were devoted to handling requests related to the information quality guidelines. The report will also include an explanation of Agency decisions to deny or limit corrective action. The first annual report, due to OMB by January 1, 2004, will document requests received and actions taken during FY2003.

F. Definitions Based on OMB Guidance

F.1. Affected persons

Persons who may benefit from or be harmed by the disseminated information. This includes persons who are seeking to address information about themselves as well as persons who use information. “Persons” includes groups, organizations and corporations as defined by the Paperwork Reduction Act (PRA) of 1995.

F.2. Dissemination

NASA-initiated, -directed, or -sponsored distribution of information to the public. Dissemination does not include distribution limited to government employees or contractors or grantees or sharing of government information or responses to requests for records under the Freedom of Information Act, the Privacy Act, the Federal Advisory Committee Act or other similar law. This definition also does not include distribution limited to correspondence with individuals or persons, press releases, archival records, public filings, subpoenas or adjudicative processes.

F.3. Influential

When used in the context of scientific, financial, or statistical information, influential means that NASA can reasonably determine that dissemination of the information will have or does have clear and substantial impact on important public policies or important private sector decisions.

F.4. Information

Any communication such as facts or data, in any media or form, including text, numerical, graphic, cartographic, narrative, or audiovisual forms. This definition includes information that NASA disseminates from a web page, but does not include the provision of hyperlinks to information that others disseminate. This definition does not include opinions, where it is made clear that that what is being offered is someone's opinion rather than fact or NASA's views.

This includes information in any media, such as paper, electronic, web page, CD-ROM, etc.

F.5. Integrity

Integrity means the security of information (e.g., that it is protected from unauthorized access or revision so that it is not compromised through corruption or falsification)

F.6. Objectivity

Objectivity means that the information is accurate, clear, complete, and unbiased.

F.7. Quality

Quality is an encompassing term comprised of three elements: integrity, objectivity, and utility. Therefore, the terms are sometimes referred to collectively as “quality.” Integrity, objectivity, and utility are individually defined within this document.

F.8. Reproducibility

The information is capable of being substantially reproduced, subject to an acceptable degree of imprecision. With respect to analytic results, “capable of being substantially reproduced” means that independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable degree of imprecision or error.

F.9. Transparent/Transparency

Information that has transparency is clear and well documented. For scientific information, transparency refers to the extent that underlying assumptions, methodologies, and analytical processes are made available as context.

F.10. Utility

Utility means that the information can be used for its intended purpose to its intended audience.

Lee B. Holcomb,

Chief Information Officer, Office of the Administrator.