01-1782. Office of Science; Office of Science Financial Assistance Program Notice 01-11: Scientific Discovery through Advanced Computing in High Energy and Nuclear Physics Research

AGENCY:

U.S. Department of Energy (DOE).

ACTION:

Notice inviting research grant applications.

SUMMARY:

The Office of High Energy and Nuclear Physics (HENP) of the Office of Science (SC), U.S. Department of Energy (DOE), hereby announces its interest in receiving grant applications for the Department's Scientific Discovery through Advanced Computing Program (SciDAC). The goal of this program is to enable the use of terascale computers to dramatically extend our exploration of the fundamental processes of nature as well as to advance our ability to predict the behavior of a broad range of complex natural and engineered systems. This goal is to be achieved through the creation of scientific simulation codes that achieve high performance on a single node, scale to hundreds of nodes and thousands of processors, and have the potential to adapt over time and to be ported to future generations of high performance computers. Projects should address a problem of national scientific or engineering significance clearly related to the mission of DOE. They are expected to have high visibility and to present a long-term vision of how their work will fundamentally impact scientific discovery in specific areas of High Energy Physics or Nuclear Physics research.

The full text of Program Notice 01-11 is available via the Internet at the following web site address: http://www.science.doe.gov/​production/​grants/​grants.html.

DATES:

Preapplications referencing this program notice must be received by 4:30 P.M. EST, February 7, 2001. A response encouraging or discouraging the submission of a formal application will be communicated by E-mail within 14 days.

Formal applications submitted in response to this notice must be received no later than 4:30 P.M., March 15, 2001, to be accepted for merit review and consideration for award in Fiscal Year 2001.

ADDRESSES:

Preapplications referencing Program Notice 01-11 should be forwarded to: U.S. Department of Energy, Office of Science, Office of High Energy and Nuclear Physics, SC-20, 19901 Germantown Road, Germantown, Maryland 20874-1290, ATTN: Peter Rosen. Preapplications can also be submitted via E-mail at the following E-mail address: [e-mail].

Formal applications referencing Program Notice 01-11 should be forwarded to: U.S. Department of Energy, Office of Science, Grants and Contracts Division, SC-64, 19901 Germantown Road, Germantown, Maryland 20874-1290, ATTN: Program Notice 01-11. The above address must be used when submitting applications by U.S. Postal Service Express Mail, any commercial mail delivery service, or when hand-carried by the applicant. An original and seven copies of the application must be submitted.

FOR FURTHER INFORMATION CONTACT:

Dr. S. Peter Rosen, Office of High Energy and Nuclear Physics, SC-20, U.S. Department of Energy, 19901 Germantown Road, Germantown, MD 20874-1290, E-mail: [e-mail].

SUPPLEMENTARY INFORMATION:

Background: Scientific Discovery Through Advanced Computing

Advanced scientific computing will be a key contributor to scientific research in the 21st Century. Within the Office of Science (SC), scientific computing programs and facilities are already essential to progress in many areas of research critical to the nation. Major scientific challenges exist in all SC research programs that can best be addressed through advances in scientific supercomputing, e.g., designing materials with selected properties, elucidating the structure and function of proteins, understanding and controlling plasma turbulence, and designing new particle accelerators. To help ensure its missions are met, SC is bringing together advanced scientific computing and scientific research in an integrated program entitled “Scientific Discovery through Advanced Computing.”

The Opportunity and the Challenge

Extraordinary advances in computing technology in the past decade have set the stage for a major advance in scientific computing. Within the next five to ten years, computers 1,000 times faster than today's computers will become available. These advances herald a new era in scientific computing. Using such computers, it will be possible to dramatically extend our exploration of the fundamental processes of nature (e.g., the structure of matter from the most elementary particles to the building blocks of life) as well as advance our ability to predict the behavior of a broad range of complex natural and engineered systems (e.g., the earth's climate or an automobile engine).

To exploit this opportunity, these computing advances must be translated into corresponding increases in the performance of the scientific codes used to model physical, chemical, and biological systems. This is a daunting problem. Current advances in computing technology are being driven by market forces in the commercial sector, not by scientific computing. Harnessing commercial computing technology for scientific research poses problems unlike those encountered in previous supercomputers, in magnitude as well as in kind. As noted in the 1998 report ^[1] from the NSF/DOE “National Workshop on Advanced Scientific Computing” and the 1999 report ^[2] from the President's Information Technology Advisory Committee, this problem will only be solved by increased investments in computer software—in research and development of scientific simulation codes as well as on the mathematical and computing systems software that underlie these codes.

Investment Plan of the Office of Science

To meet the challenge posed by the new generation of terascale computers, SC will fund a set of coordinated investments as outlined in its long-range plan for scientific computing, Scientific Discovery through Advanced Computing,^[3] submitted to Congress on March 30, 2000. First, it will create a Scientific Computing Software Infrastructure that bridges the gap between the advanced computing technologies being developed by the computer industry and the scientific research programs sponsored by the Office of Science. Specifically, the SC effort proposes to:

• Create a new generation of Scientific Simulation Codes that take full advantage of the extraordinary computing capabilities of terascale computers.
• Create the Mathematical and Computing Systems Software to enable the Scientific Simulation Codes to effectively and efficiently use terascale computers.
• Create a Collaboratory Software Environment to enable geographically separated scientists to effectively work together as a team and to facilitate remote access to both facilities and data.

These activities are supported by a Scientific Computing Hardware Infrastructure that will be tailored to meet the needs of SC's research programs. The Hardware Infrastructure is robust, to provide the stable computing resources needed by the scientific applications; agile, to respond to innovative advances in computer technology that impact scientific computing; and flexible, to allow the most appropriate and economical resources to be used to solve each class of problems. Specifically, the SC proposes to support:

• A Flagship Computing Facility, the National Energy Research Scientific Computing Center (NERSC), to provide the robust, high-end computing resources needed by a broad range of scientific research programs.
• Topical Computing Facilities to provide computing resources tailored for specific scientific applications and to serve as the focal point for an application community as it strives to optimize its use of terascale computers.
• Experimental Computing Facilities to assess the promise of new computing technologies being developed by the computer industry for scientific applications.

Both sets of investments will create exciting opportunities for teams of researchers from laboratories and universities to create new revolutionary computing capabilities for scientific discovery.

The Benefits

The Scientific Computing Software Infrastructure, along with the upgrades to the hardware infrastructure, will enable laboratory and university researchers to solve the most challenging scientific problems faced by the Office of Science at a level of accuracy and detail never before achieved. These developments will have significant benefits to all of the government agencies that rely on high-performance scientific computing to achieve their mission goals as well as to the U.S. high-performance computing industry.

Background: Scientific Simulation in High Energy Physics and Nuclear Physics Research

The Office of High Energy and Nuclear Physics supports a program of research into the fundamental nature of matter and energy. In carrying out this mission it:

• Builds and operates large, world class charged-particle accelerator facilities for the nation and for the international scientific research community;
• Builds detectors and instruments, for accelerator and non-accelerator based experiments, designed to answer fundamental questions about the nature of matter and energy; and
• Carries out a program of scientific research based on experimental data, theoretical studies, and scientific simulation.

This solicitation is focused on proposals to accelerate progress through the use of scientific simulation codes.

Computational modeling and simulation are among the most significant developments in the practice of scientific inquiry in the 20th century. The coming advances in computing performance, if they can be realized for scientific problems, herald a new era in scientific computing. If computers capable of 100 teraflops or more become available in the next few years, it will be possible to dramatically extend our exploration of the fundamental processes of nature. It will also be possible to predict the behavior of a broad range of complex systems, such as charged-particle accelerator components, and eventually entire accelerators.

However, it is clear that the development of scientific codes that are capable of utilizing terascale computers efficiently and are adaptable, portable and re-usable is a massive undertaking that could take as long as 8-10 years to achieve its most ambitious scientific goals. This may require efforts of hundreds of person-years of work.

It is also apparent that the most appropriate, cost-effective computing resources for scientific simulations vary significantly from application to application. Therefore, much work is needed to understand the optimal configuration of computing hardware for each task and to design operating environments best able to foster significant scientific discoveries.

This solicitation is for proposals that articulate the long-term vision and potential for scientific progress through simulation, whilst laying out a concrete step-wise program of work and scientific research for the next 3 to 5 years.

The scope and complexity of the proposed projects will require close collaboration among researchers from computational and theoretical physics, computer science, and applied mathematics disciplines. Accordingly, this solicitation calls for the creation of scientific simulation teams, or collaborations, as the organizational basis for a successful application. A scientific simulation team is a multi-institutional, multi-disciplinary group of people who will:

• Create scientific simulation codes that take full advantage of terascale computers,
• Work closely with other SciDAC teams and centers to ensure that the best available mathematical algorithms and computer science methods are employed, and
• Manage the work of the team in a way that will foster good communication and decision making (see section on Collaboration and Coordination below).

Partnerships among universities, national laboratories, and industry are encouraged. Applications are being sought in the broad topical areas listed below.

Accelerator Science and Simulation

The successful development of large accelerator facilities involves enormous investments in theory, experiment and simulation. Optimizing the performance of current accelerators and the design of future accelerators will require unprecedented precision in accelerator component design and beam dynamics and control. Applicants should explain how the proposed program of work will facilitate important design decisions, increase safety and reliability, optimize performance and reduce the cost of accelerators.

The development of a comprehensive, coherent terascale simulation environment for the U.S. particle accelerator community will involve development of new computational models and codes, mathematical models, program frameworks and visualization techniques. The scientific software, while making good use of existing codes for (a) calculations for the design of complex electromagnetic components and systems and (b) beam dynamics calculations for predicting beam halo, must provide high performance on terascale computers and be capable of scaling to 100 teraflops or more. New codes will need to be developed for problems, such as electromagnetic modeling of lossy structures and wakefields, parallel static computation for electric and magnetic component design, and parallel modeling of intense beams in injectors, linear and circular machines. Models need to be developed to include a range of physical phenomena such as collisions, synchrotron radiation, and surface emissions. In order to simulate accelerator components and entire accelerators, the scientific simulation codes will need to work together to carry out simulations of complex systems involving tight coupling of beam dynamics and electromagnetics.

Collaborative work with Fusion Energy Scientists may also be useful since there are some common problems related to modeling electromagnetic fields and beam dynamics.

Theoretical Research

In the past few years, several areas of theoretical research have demonstrated the potential to further scientific knowledge by efficiently using scientific simulation codes on terascale computers to:

• Provide a major quantitative tool for simulations of quantum chromodynamics (QCD) on a lattice, which will:

(a) Provide crucial information in support of the experimental programs in high energy and nuclear physics.

(b) Make accurate determinations of a number of fundamental quantities, such as the coupling constant that determines the strength of quark-gluon interactions, and the underlying masses of the quarks.

(c) Explore the limitations, if any, of the “Standard Model” of particle interactions.

(d) Explore how quarks and gluons provide the binding and spin of the nucleon.

• Develop theoretical models of complex systems under extreme conditions, such as:

(a) Exploration of complex theoretical models of supernovae and comparison of the predictions with experimental results.

(b) Study of the behavior of supersymmetric and other quantum field theories.

Particular areas of interest include, but are not limited to:

Quantum Chromodynamics (QCD)

The development of a coherent terascale simulation environment for the study of QCD that will permit evolution of scientific codes to take advantage of 100 teraflop computers is a challenging problem. It demands a coordinated effort to provide the computer software infrastructure, the detailed scientific codes and algorithms, together with effective ways of using computing hardware now and in the future.

Simulations of Complex Nuclear Structure, Such as Found in Core-Collapse Supernovae

The development of a comprehensive model that brings together nuclear physics, particle physics, fluid dynamics, radiation transport, and general relativity is an equally challenging problem. Data from next-generation neutrino detectors, gravitational wave observatories, ground and space-based observatories, new radioactive beam facilities, and other experimental facilities will provide opportunities to evaluate and refine the many underlying physical models in the simulation.

Testbeds and Collaboratory Software Environments

Collaboratories link geographically dispersed researchers, data and tools, via high performance networks, to enable remote access to facilities, access to large datasets and shared environments. They enable geographically separated scientists to effectively work together as a team and facilitate remote access to both computing facilities and data.

As the size and complexity of high energy and nuclear physics experiments has increased so have the number and geographical dispersion of the researchers and the amount of data that must be collected, simulated and analyzed. Thus future experiments critically depend on the existence of such distributed hardware and software environments for their success. The scientific simulation applications that are the focus of this solicitation will also consist of geographically dispersed researchers, and will require high performance networks, to enable remote access to computing facilities, and multi-terabyte datasets.

Proposals for testbeds and collaborations across organizations that include network researchers, middleware developers and high energy and nuclear physicists are encouraged. However, they should be submitted in response to Notice 01-06 of the Office of Advanced Scientific Computing Research (ASCR). Copies should also be submitted to the Office of High Energy and Nuclear Physics, and joint funding can be considered.

Collaboration and Coordination

It is expected that all applications submitted in response to this notice will be for scientific simulation teams involving more than one institution. Applications from different institutions, directed at a common research activity, must include a common technical description of the overall research project. Each participating institution must have a qualified principal investigator, who is responsible for the part of the effort at that institution, and separate face pages and budget pages for each institution. The distinct scope of work proposed for each institution must be clearly specified. Any work proposed in computer science or applied mathematics should also be described separately. Applicants should include cost sharing whenever feasible. Synergistic collaborations with researchers in federal laboratories and Federally Funded Research and Development Centers (FFRDCs), including the DOE National Laboratories are encouraged, although funds will not be provided to these organizations under this particular Notice. Further information on preparation of collaborative proposals is available in the Application Guide for the Office of Science Financial Assistance Program that is available via the Internet at: http://www.science.doe.gov/​production/​grants/​Colab.html.

Preapplications

Potential applicants are strongly encouraged, but not required, to submit a brief preapplication consisting of two or three pages of narrative describing the research objectives, technical approaches and management plan. Each preapplication should include a cover sheet with the title of the project, project principal investigator, institutions involved, and their principal investigators and senior personnel. The name, telephone number, and e-mail address of each principal investigator should also be provided. In addition, brief, one-page curriculum vitae should be submitted for the principal investigators and other senior personnel involved. Preapplications will be evaluated to assess their programmatic relevance, and a response will be provided to the principal investigator within 14 days of receipt. However, notification of a successful preapplication is not an indication that an award will be made in response to a formal application.

Program Funding

Up to $2,500,000 of Fiscal Year 2001 funding will be available for grant awards in FY 2001. Additional funding for each proposed project may be available through the Office of Advanced Scientific Computing Research for closely related research in computer science and/or applied mathematics. Applications may request support for up to three years, with out-year support contingent on the availability of funds and satisfactory progress. To support multi-disciplinary, multi-institutional efforts, funding levels of up to $1.0 million per project may be requested, under this notice, for the first year of the project. Requests for increased funding levels in future years will be entertained subject to availability of funds, progress of the funded activity, and programmatic needs.

As required by the SC Grant Application Guide, applicants must submit their budgets using the Budget Page (DOE Form 4620.1) with one Budget Page for each year of requested funding. The requested funding for the proposed work in computer science and applied mathematics should be included with the other project costs on the Budget Page. However, applicants are also requested to list the proposed computer science and applied mathematics costs separately in an appendix, as the Office of Advanced Scientific Computing Research may support this part of the work (up to 20-25% of the total project cost). The Office of High Energy and Nuclear Physics expects to fund three or four successful projects, depending on the size of the awards.

Evaluation Criteria

Applications will be subjected to scientific merit review (peer review) and will be evaluated against the following criteria listed in descending order of importance as codified in 10 CFR 605.10(d) (www.science.doe.gov/​production/​grants/​605index.html):

1. Scientific and/or technical merit of the project,

2. Appropriateness of the proposed method or approach,

3. Competency of the applicant's personnel and adequacy of the proposed resources,

4. Reasonableness and appropriateness of the proposed budget.

The evaluation of applications under item 1, Scientific and Technical Merit, will pay particular attention to:

(a) the potential of the proposed project to achieve a major advance in high energy and/or nuclear physics;

(b) the potential of the proposed project to advance the state-of-the-art in computational modeling and simulation in areas pertinent to high energy and nuclear physics research;

(c) the need for extraordinary computing resources to address problems of critical scientific importance to the high energy physics or nuclear physics program and the demonstrated abilities of the applicants to exploit terascale computers;

(d) knowledge of and coupling to previous efforts in scientific simulation;

(e) the extent to which the project incorporates broad community (industry/academia/other federal programs) interaction;

(f) the extent to which the results of the project are likely to be extensible to other program or discipline areas; and

(g) the importance of the proposed project to the mission of the Office of High Energy and Nuclear Physics and its impact on overall DOE goals.

The evaluation under item 2, Appropriateness of the Proposed Method or Approach, will also consider the following elements related to appropriateness of the proposed Scientific Computing Hardware Infrastructure to be used and of the quality of planning:

(a) Viability of the plan with respect to the scale and nature of current and future Computing Hardware Infrastructure needed;

(b) clarity of the plan in detailing areas of work to be addressed by discipline scientists, computational scientists, applied mathematicians, computer scientists and computer programmers;

(c) quality of the plan for effective collaboration among participants;

(d) quality of the plan for ensuring communication with other advanced computation and simulation efforts;

(e) viability of the plan for deploying the software and for assuring long-term maintenance, support, and re-use of the scientific codes and software infrastructure developed;

(f) viability of the plan for verifying and validating the models developed, including verification using experiment results; and

(g) quality and clarity of the proposed work schedule and project deliverables.

The evaluation will include program policy factors such as the relevance of the proposed research to the terms of the announcement and the agency's programmatic needs.

Note, that external peer reviewers are selected with regard to both their scientific expertise and the absence of conflict-of-interest issues. Non-federal reviewers may be used, and submission of an application constitutes agreement that this is acceptable to the investigator(s) and the submitting institution.

General information about development and submission of applications, eligibility, limitations, evaluations and selection processes, and other policies and procedures may be found in the Application Guide for the Office of Science (SC) Financial Assistance Program and in 10 CFR part 605. Electronic access to SC's Financial Assistance Guide and required forms is made available via the Internet using the following Web site address: http://www.science.doe.gov/​production/​grants/​grants.html.

In addition, for this notice, project descriptions must be 25 pages or less, including tables and figures, but excluding attachments. The application must also contain an abstract or project summary, letters of intent from all non-funded collaborators, and short curriculum vitae of all senior personnel. On the SC grant Face Page (DOE Form 4650.2), in block 15, also provide the Principal Investigator's phone number, FAX number, and E-mail address.

The Catalog of Federal Domestic Assistance Number for this program is 81.049, and the solicitation control number is ERFAP 10 CFR Part 605.

Issued in Washington DC on January 10, 2001.

John Rodney Clark,

Associate Director of Science for Resource Management.

Footnotes

1.  This workshop was sponsored by the National Science Foundation and the Department of Energy and hosted by the National Academy of Sciences on July 30-31, 1998. Copies of the report may be obtained from: http://www.er.doe.gov/​production/​octr/​mics/​index.html

2.  Copies of the PITAC report may be obtained from http://www.cclc.gov/​ac/​report/​.

3.  Copies of the SC computing plan, Scientific Discovery through Advanced Computing, can be downloaded from the SC web site at: http://www.sc.doe.gov/​production/​octr/​index.html.