03-28875. Notice of Final Standard Infrastructure Grants Announcement

AGENCY:

Substance Abuse and Mental Health Services Administration, HHS.

ACTION:

Notice of final Infrastructure Gants announcement.

SUMMARY:

On August 21, 2003, the Substance Abuse and Mental Health Services Administration (SAMHSA) announced plans to change its approach to announcing and soliciting applications for its discretionary grant programs in Fiscal Year (FY) 2004. These changes involved the publication of four standard grant announcements that would provide the basic program design and application instructions for four types of grants—Services Grants, Infrastructure Grants, Best Practices Planning and Implementation Grants, and Service-to-Science Grants. The four announcements were made available for public review and comment for 60 days. The comments received and changes made to the standard grant announcements are described in a separate Federal Register notice. This notice provides the final text for SAMHSA's standard Infrastructure Grants announcement.

Authority: Sections 509, 516, and 520A of the Public Health Service Act.

DATES:

Use of the standard Infrastructure Grants announcement will be effective November 21, 2003. The standard Infrastructure Grants announcement must be used in conjunction with separate Notices of Funding Availability (NOFAs) that will provide application due dates and other key dates for specific SAMHSA grant funding opportunities.

ADDRESSES:

Questions about SAMHSA's standard Infrastructure Grants announcement may be directed to Cathy Friedman, M.A., Office of Policy, Planning and Budget, 5600 Fishers Lane, Room 12C-26, Rockville, Maryland, 20857. Fax: [phone]) E-mail: [e-mail].

FOR FURTHER INFORMATION CONTACT:

Cathy Friedman, M.A., Office of Policy, Planning and Budget, 5600 Fishers Lane, Room 12C-26, Rockville, Maryland, 20857. Fax: [phone]) E-mail: [e-mail]. Phone: [phone].

SUPPLEMENTARY INFORMATION:

Starting in FY 2004, SAMHSA is changing its approach to announcing and soliciting applications for its discretionary grants. SAMHSA will publish four standard grant announcements that will describe the general program design and provide application instructions for four types of grants—Services Grants, Infrastructure Grants, Best Practices Planning and Implementation Grants, and Service-to-Science Grants. The text for the final standard Infrastructure Grants announcement is provided below.

The standard Infrastructure Grants announcement will be posted on SAMHSA's web page (www.samhsa.gov) and will be available from SAMHSA's clearinghouses on an ongoing basis. The standard announcements will be used in conjunction with brief Notices of Funding Availability (NOFAs) that will announce the availability of funds for specific grant funding opportunities within each of the standard grant programs (e.g., Homeless Treatment grants, Statewide Family Network grants, HIV/AIDS and Substance Abuse Prevention Planning Grants, etc.).

Infrastructure Grants—INF 04 (Initial Announcement)

Catalogue of Federal Domestic Assistance (CFDA) No.: 93.243 (unless otherwise specified in a NOFA in the Federal Register and on www.grants.gov).

Authority: Sections 509, 516 and/or 520A of the Public Health Service Act, as amended, and subject to the availability of funds (unless otherwise specified in a NOFA in the Federal Register and on www.grants.gov).

Key Dates

Application Deadline	This Program Announcement provides general instructions and guidelines for multiple funding opportunities. Application deadlines for specific funding opportunities will be published in Notices of Funding Availability (NOFAs) in the Federal Register and on http://www.grants.gov.
Intergovernmental Review (E.O. 12372)	Letters from State Single Point of Contact (SPOC) are due no later than 60 days after application deadline.
Public Health System Impact Statement (PHSIS)/SSA Coordination	Applicants must send the PHSIS to appropriate State and local health agencies by application deadline. Comments from Single State Agency are due no later than 60 days after application deadline.

Table of Contents

I. Funding Opportunity Description

A. Introduction

B. Expectations

II. Award Information

A. Award Amount

B. Funding Mechanism

III. Eligibility Information

A. Eligible Applicants

B. Cost-Sharing

C. Other

IV. Application and Submission Information

A. Address to Request Application Package

B. Content and Form of Application Submission

C. Submission Dates and Times

D. Intergovernmental Review (E.O. 12372) Requirements

E. Funding Limitations/Restrictions

F. Other Submission Requirements

V. Application Review Information

A. Evaluation Criteria

B. Review and Selection Process

C. Award Criteria

VI. Award Administration Information

A. Award Notices

B. Administrative and National Policy Requirements

C. Reporting Requirements

VII. Agency Contacts

VIII. Other Information

A. SAMHSA Confidentiality and Participant Protection Requirements and Protection of Human Subjects Regulations

B. Intergovernmental Review (E.O. 12372) Instructions

C. Public Health System Impact Statement

Appendix A: Checklist for Application Formatting Requirements

Appendix B: Glossary

Appendix C: Logic Model Resources

I. Funding Opportunity Description

A. Introduction

The Substance Abuse and Mental Health Services Administration (SAMHSA) announces its intent to solicit applications for Infrastructure Grants. These grants will increase the capacity of mental health and/or substance abuse service systems to support effective programs and services. Applicants who seek Federal support to develop or enhance their service system infrastructure in order to support effective substance abuse and/or mental health services should apply for awards under this announcement.

SAMHSA also funds grants under three other standard grant announcements:

• Services Grants provide funding to implement substance abuse and mental health services.
• Best Practices Planning and Implementation Grants help communities and providers identify practices to effectively meet local needs, develop strategic plans for implementing/adapting those practices and pilot-test practices prior to full-scale implementation.
• Service to Science Grants document and evaluate innovative practices that address critical substance abuse and mental health service gaps but that have not yet been formally evaluated.

This announcement describes the general program design and provides application instructions for all SAMHSA Infrastructure Grants. The availability of funds for specific Infrastructure Grants will be announced in supplementary Notices of Funding Availability (NOFAs) in the Federal Register and at www.grants.gov—the Federal grant announcement web page.

Typically, funding for Infrastructure Grants will be targeted to specific populations and/or issue areas, which will be specified in the NOFAs. The NOFAs will also:

• Specify total funding available for the first year of the grants and the expected size and number of awards;
• Provide the application deadline;
• Note any specific program requirements for each funding opportunity; and
• Include any limitations or exceptions to the general provisions in this announcement (e.g., eligibility, allowable activities).

It is, therefore, critical that you consult the NOFA as well as this announcement in developing your grant application.

B. Expectations

SAMHSA's Infrastructure Grants support an array of activities to help the grantee build a solid foundation for delivering and sustaining effective substance abuse prevention and/or treatment and/or mental health services.

SAMHSA recognizes that each applicant will start from a unique point in developing infrastructure and will serve populations/communities with specific needs. Awardees may pursue diverse strategies and methods to achieve their infrastructure development and capacity expansion goals. Successful applicants will provide a coherent and detailed conceptual “roadmap” of the process by which they have assessed or intend to assess service system needs and plan/implement infrastructure development strategies that meet those needs. The plan put forward in the grant application must show the linkages among needs, the proposed infrastructure development strategy, and increased system capacity that will enhance and sustain effective programs and services.

1. Allowable Activities

SAMHSA's Infrastructure Grants will support the following types of activities:

Infrastructure Development. Infrastructure Grant funds must be used primarily to support infrastructure development, including the following types of activities:

• Needs assessment.
• Strategic planning.
• Financing/coordination of funding streams.
• Organizational/structural change (e.g., to create locus of responsibility for a specific issue/population, or to increase access to or efficiency of services).
• Development of interagency coordination mechanisms.
• Provider/network development.
• Policy development to support needed service system improvements (e.g., rate-setting activities, establishment of standards of care, development/revision of credentialing, licensure, or accreditation requirements).
• Quality improvement efforts.
• Performance measurement development.
• Workforce development (e.g., training, support for licensure, credentialing, or accreditation).
• Data infrastructure/MIS development.

Implementation Pilots (maximum 15 percent of total grant award). Depending on the scope of the project (see description of award categories below), up to 15 percent of the total grant award may be used for “implementation pilots” to test the effectiveness of the infrastructure changes on services delivery. Funds may not be used to provide direct services except in the context of an implementation pilot.

2. Data and Performance Measurement

The Government Performance and Results Act of 1993 (P.L.103-62, or “GPRA”) requires all Federal agencies to set program performance targets and report annually on the degree to which the previous year's targets were met.

Agencies are expected to evaluate their programs regularly and to use results of these evaluations to explain their successes and failures and justify requests for funding.

To meet the GPRA requirements, SAMHSA must collect performance data (i.e., “GPRA data”) from grantees. Grantees are required to report these GPRA data to SAMHSA on a timely basis.

Specifically, grantees will be required to provide data on a set of required measures, as specified in the NOFA. The data collection tools to be used for reporting the required data will be provided in the application kits distributed by SAMHSA's clearinghouses and posted on SAMHSA's website along with each NOFA. In your application, you must demonstrate your ability to collect and report on these measures, and you may be required to provide some baseline data.

The terms and conditions of the grant award also will specify the data to be submitted and the schedule for submission. Grantees will be required to adhere to these terms and conditions of award.

Applicants should be aware that SAMHSA is working to develop a set of required core performance measures for each of SAMHSA's standard grants (i.e., Services Grants, Infrastructure Grants, Best Practices Planning and Implementation Grants, and Service-to-Science Grants). As this effort proceeds, some of the data collection and reporting requirements included in SAMHSA's NOFAs may change. All grantees will be expected to comply with any changes in data collection requirements that occur during the grantee's project period.

3. Grantee Meetings

You must plan to send a minimum of two people (including the Project Director) to at least one joint grantee meeting in each year of the grant, and you must include funding for this travel in your budget. At these meetings, grantees will present the results of their projects and Federal staff will provide technical assistance. Each meeting will be 3 days. These meetings will usually be held in the Washington, DC, area, and attendance is mandatory.

4. Evaluation

Grantees must evaluate their projects, and applicants are required to describe their evaluation plans in their applications. The evaluation should be designed to provide regular feedback to the project to improve services. The evaluation must include both process and outcome components. Process and outcome evaluations must measure change relating to project goals and objectives over time compared to baseline information. Control or comparison groups are not required. You must consider your evaluation plan when preparing the project budget.

Process components should address issues such as:

• How closely did implementation match the plan?
• What types of deviation from the plan occurred?
• What led to the deviations?
• What impact did the deviations have on the intervention and evaluation?
• Who provided (program, staff) what services (modality, type, intensity, duration), to whom (individual characteristics), in what context (system, community), and at what cost (facilities, personnel, dollars)?

Outcome components should address issues such as:

• What was the effect of infrastructure development on service capacity and other system outcomes?
• What program/contextual factors were associated with outcomes?
• What individual factors were associated with outcomes?
• How durable were the effects?

If the project includes an implementation pilot involving services delivery, the evaluation should include client and system outcomes.

No more than 20% of the total grant award may be used for evaluation and data collection. The evaluation and data collection may be considered “Infrastructure” and/or “Implementation Pilots” expenditures, depending on their purpose.

II. Award Information

A. Award Amount

The NOFA will specify the expected award amount for each funding opportunity. Regardless of the amount specified in the NOFA, the actual award amount will depend on the availability of funds.

Two types of Infrastructure Grants will be made:

Category 1—Small Infrastructure Grants. The Category 1 grants will be limited in scope as specified in the NOFA. For example, allowable activities might be limited to workforce development, data infrastructure, or strategic planning. Implementation pilots are not allowed in Category 1 awards. Category 1 awards are expected to be for a period of 1-3 years in amounts ranging from $250,000-$500,000 per year.

Category 2—Comprehensive Infrastructure Grants. The scope of the Category 2 grants will be much larger. While applicants are not required to include all of the allowable activities in their proposed projects, the proposed projects must encompass multiple domains (e.g., needs assessment, strategic and financial planning, organizational/structural change, and network development). Category 2 awards may use a maximum of 15 percent of the total grant award for implementation pilots. Category 2 awards are expected to be for a period of 3-5 years in amounts ranging from $750,000-$3 million per year.

Applications with proposed budgets that exceed the allowable amount as specified in the NOFA in any year of the proposed project will be screened out and will not be reviewed. Annual continuation awards will depend on the availability of funds, grantee progress in meeting project goals and objectives, and timely submission of required data and reports.

B. Funding Mechanism

The NOFA will indicate whether awards for each funding opportunity will be made as grants or cooperative agreements (see the Glossary in Appendix B for further explanation of these funding mechanisms). For cooperative agreements, the NOFA will describe the nature of Federal involvement in project performance and specify roles and responsibilities of grantees and Federal staff.

III. Eligibility Information

A. Eligible Applicants

Eligible applicants are domestic public and private nonprofit entities. For example, State, local or tribal governments; public or private universities and colleges; community- and faith-based organizations; and tribal organizations may apply. The statutory authority for this program precludes grants to for-profit organizations. The NOFA will indicate any limitations on eligibility.

B. Cost-Sharing

Cost-sharing (see Glossary) is not required in this program, and applications will not be screened out on the basis of cost-sharing. However, you may include cash or in-kind (see Glossary) contributions in your proposal as evidence of commitment to the proposed project.

C. Other

SAMHSA applicants must comply with certain program requirements, including:

• Budgetary limitations as specified in Sections I, II, and IV-E of this document; and
• Documentation of nonprofit status as required in the PHS 5161-1.

You also must comply with any additional program requirements specified in the NOFA, such as the required signature of certain officials on the face page of the application and/or required memoranda of understanding with certain signatories.

Applications that do not comply with the eligibility and specific program requirements for the funding opportunity for which the application is submitted will be screened out and will not be reviewed.

IV. Application and Submission Information

(To ensure that you have met all submission requirements, a checklist is provided for your use in Appendix A of this document.)

A. Address To Request Application Package

You may request a complete application kit by calling one of SAMHSA's national clearinghouses:

• For substance abuse prevention or treatment grants, call the National Clearinghouse for Alcohol and Drug Information (NCADI) at 1-800-729-6686.
• For mental health grants, call the National Mental Health Information Center at 1-800-789-CMHS (2647).

You also may download the required documents from the SAMHSA Web site at www.samhsa.gov. Click on “grant opportunities.”

Additional materials available on this Web site include:

• A technical assistance manual for potential applicants;
• Standard terms and conditions for SAMHSA grants;
• Guidelines and policies that relate to SAMHSA grants (e.g., guidelines on cultural competence, consumer and family participation, and evaluation); and
• Enhanced instructions for completing the PHS 5161-1 application.

B. Content and Form of Application Submission

1. Required Documents

SAMHSA application kits include the following documents:

• PHS 5161-1 (revised July 2000)—Includes the face page, budget forms, assurances, certification, and checklist. You must use the PHS 5161-1 unless otherwise specified in the NOFA. Applications that are not submitted on the required application form will be screened out and will not be reviewed.
• Program Announcement (PA)—Includes instructions for the grant application. This document is the PA.
• Notice of Funding Availability (NOFA)—Provides specific information about availability of funds, as well as any exceptions or limitations to provisions in the PA. The NOFAs will be published in the Federal Register, as well as on the Federal grants Web site (www.grants.gov).

You must use all of the above documents in completing your application.

2. Required Application Components

To ensure equitable treatment of all applications, SAMHSA will accept only complete applications for review. In order for your application to be complete, it must include the required ten application components (Face Page, Abstract, Table of Contents, Budget Form, Project Narrative and Supporting Documentation, Appendices, Assurances, Certifications, Disclosure of Lobbying Activities, and Checklist). Applications that do not contain the required components will be screened out and will not be reviewed.

• Face Page—Use Standard Form (SF) 424, which is part of the PHS 5161-1. [Note: Beginning October 1, 2003, applicants will need to provide a Dun and Bradstreet (DUNS) number to apply for a grant or cooperative agreement from the Federal Government. SAMHSA applicants will be required to provide their DUNS number on the face page of the application. Obtaining a DUNS number is easy and there is no charge. To obtain a DUNS number, access the Dun and Bradstreet Web site at www.dunandbradstreet.com or call 1-866-705-5711. To expedite the process, let Dun and Bradstreet know that you are a public/private nonprofit organization getting ready to submit a Federal grant application.]
• Abstract—Your total abstract should not be longer than 35 lines. In the first five lines or less of your abstract, write a summary of your project that can be used, if your project is funded, in publications, reporting to Congress, or press releases.
• Table of Contents—Include page numbers for each of the major sections of your application and for each appendix.
• Budget Form—Use SF 424A, which is part of the 5161-1. Fill out Sections B, C, and E of the SF 424A.
• Project Narrative and Supporting Documentation—The Project Narrative describes your project. It consists of Sections A through D. These sections in total may not be longer than 25 pages. More detailed instructions for completing each section of the Project Narrative are provided in “Section V—Application Review Information” of this document.

The Supporting Documentation provides additional information necessary for the review of your application. This supporting documentation should be provided immediately following your Project Narrative in Sections E through H. There are no page limits for these sections, except for Section G, Biographical Sketches/Job Descriptions.

• Section E—Literature Citations. This section must contain complete citations, including titles and all authors, for any literature you cite in your application.
• Section F—Budget Justification, Existing Resources, Other Support. You must provide a narrative justification of the items included in your proposed budget, as well as a description of existing resources and other support you expect to receive for the proposed project. Be sure to show that no more than 20% of the total grant award will be used for data collection and evaluation. If you are proposing a services implementation pilot (allowed only for Category 2 applicants), show that no more than 15% of the total grant award will be used for the pilot.
• Section G—Biographical Sketches and Job Descriptions.
• Include a biographical sketch for the Project Director and other key positions. Each sketch should be 2 pages or less. If the person has not been hired, include a letter of commitment from the individual with a current biographical sketch.
• Include job descriptions for key personnel. Job descriptions should be no longer than 1 page each.
• Sample sketches and job descriptions are listed on page 22, Item 6 in the Program Narrative section of the PHS 5161-1.
• Section H—Confidentiality and SAMHSA Participant Protection/Human Subjects. Section VIII-A of this document describes requirements for the protection of the confidentiality, rights and safety of participants in SAMHSA-funded activities. This section also includes guidelines for completing this part of your application.
• Appendices 1 through 5—Use only the appendices listed below. Do not use more than 30 pages for Appendices 1, 3 and 4. There are no page limitations for Appendices 2 and 5. Do not use appendices to extend or replace any of the sections of the Project Narrative unless specifically required in the NOFA. Reviewers will not consider them if you do.
• Appendix 1: Letters of Support.
• Appendix 2: Data Collection Instruments/Interview Protocols.
• Appendix 3: Sample Consent Forms.
• Appendix 4: Letter to the SSA (if applicable; see Section VIII-C of this document).
• Appendix 5: A copy of the State or County Strategic Plan, a State or county needs assessment, or a letter from the State or county indicating that the proposed project addresses a State-or county-identified priority.
• Assurances—Non-Construction Programs. Use Standard Form 424B found in PHS 5161-1. Some applicants will be required to complete the Assurance of Compliance with SAMHSA Charitable Choice Statutes and Regulations Form SMA 170. If this assurance applies to a specific funding opportunity, it will be posted on SAMHSA's Web site with the NOFA and provided in the application kits available at SAMHSA's clearinghouse (NCADI).
• Certifications—Use the “Certifications” forms found in PHS 5161-1.
• Disclosure of Lobbying Activities—Use Standard Form LLL found in the PHS 5161-1. Federal law prohibits the use of appropriated funds for publicity or propaganda purposes, or for the preparation, distribution, or use of the information designed to support or defeat legislation pending before the Congress or State legislatures. This includes “grass roots” lobbying, which consists of appeals to members of the public suggesting that they contact their elected representatives to indicate their support for or opposition to pending legislation or to urge those representatives to vote in a particular way.
• Checklist—Use the Checklist found in PHS 5161-1. The Checklist ensures that you have obtained the proper signatures, assurances and certifications and is the last page of your application.

3. Application Formatting Requirements

Applicants also must comply with the following basic application requirements. Applications that do not comply with these requirements will be screened out and will not be reviewed.

• Text must be legible.
• Paper must be white and 8.5″ by 11.0″ in size.
• Pages must be typed single-spaced with one column per page.
• Page margins must be at least one inch.
• Type size in the Project Narrative cannot exceed an average of 15 characters per inch when measured with a ruler. (Type size in charts, tables, graphs, and footnotes will not be considered in determining compliance.)
• Photo reduction or condensation of type cannot be closer than 15 characters per inch or 6 lines per inch.
• Pages cannot have printing on both sides.
• Page limitations specified for the Project Narrative and Appendices cannot be exceeded.
• Information provided must be sufficient for review.

To facilitate review of your application, follow these additional guidelines:

• Applications should be prepared using black ink. This improves the quality of the copies of applications that are provided to reviewers.
• Do not use heavy or light-weight paper or any material that cannot be photocopied using automatic photocopying machines. Odd-sized and oversized attachments, such as posters, will not be copied or sent to reviewers. Do not send videotapes, audiotapes, or CD-ROMs.
• Pages should be numbered consecutively from beginning to end so that information can be located easily during review of the application. For example, the cover page should be labeled “page 1,” the abstract page should be “page 2,” and the table of contents page should be “page 3.” Appendices should be labeled and separated from the Project Narrative and budget section, and the pages should be numbered to continue in the sequence.

C. Submission Dates and Times

Deadlines for submission of applications for specific funding opportunities will be published in the NOFAs in the Federal Register and posted on the Federal grants Web site (www.grants.gov).

Your application must be received by the application deadline. Applications sent through postal mail and received after this date must have a proof-of-mailing date from the carrier dated at least 1 week prior to the due date. Private metered postmarks are not acceptable as proof of timely mailing.

You will be notified by postal mail that your application has been received.

Applications not received by the application deadline or not postmarked by a week prior to the application deadline will be screened out and will not be reviewed.

D. Intergovernmental Review (E.O. 12372) Requirements

Executive Order 12372, as implemented through Department of Health and Human Services (DHHS) regulation at 45 CFR part 100, sets up a system for State and local review of applications for Federal financial assistance. Instructions for this review are included in Section VIII-B of this document. Section VIII-C provides instructions for the Public Health System Impact Statement (PHSIS) and submission of comments from the Single State Agency (SSA).

E. Funding Limitations/Restrictions

Cost principles describing allowable and unallowable expenditures for Federal grantees, including SAMHSA grantees, are provided in the following documents:

• Institutions of Higher Education: OMB Circular A-21.
• State and Local Governments: OMB Circular A-87.
• Nonprofit Organizations: OMB Circular A-122.
• Appendix E Hospitals: 45 CFR Part 74.

In addition, SAMHSA Infrastructure Grant recipients must comply with the following funding restrictions:

• Infrastructure grant funds must be used for purposes supported by the program.
• If requested project funds exceed $750,000, a maximum of 15% of grant award funds may be used for implementation pilots. Direct services may be funded only in the context of an implementation pilot.
• No more than 20% of the grant award may be used for evaluation and data collection expenses. These expenses may be considered infrastructure or implementation pilot expenses, depending on the nature of the evaluation and data collection.
• Infrastructure funds may not be used to pay for the purchase or construction of any building or structure to house any part of the grant project. Applications may request up to $75,000 for renovations and alterations of existing facilities.

F. Other Submission Requirements

1. Where To Send Applications

Send applications to the following address: Substance Abuse and Mental Health Services Administration, Office of Program Services, Review Branch, 5600 Fishers Lane, Room 17-89, Rockville, Maryland, 20857.

Be sure to include the funding announcement number from the NOFA in item number 10 on the face page of the application. If you require a phone number for delivery, you may use [phone].

2. How To Send Applications

Mail an original application and 2 copies (including appendices) to the mailing address provided above. The original and copies must not be bound. Do not use staples, paper clips, or fasteners. Nothing should be attached, stapled, folded, or pasted.

You must use a recognized commercial or governmental carrier. Hand carried applications will not be accepted. Faxed or e-mailed applications will not be accepted.

V. Application Review Information

A. Evaluation Criteria

Your application will be reviewed and scored according to the quality of your response to the requirements listed below for developing the Project Narrative (Sections A-D). These sections describe what you intend to do with your project.

• In developing the Project Narrative section of your application, use these instructions, which have been tailored to this program. These are to be used instead of the “Program Narrative” instructions found in the PHS 5161-1.
• You must use the four sections/headings listed below in developing your Project Narrative. Be sure to place the required information in the correct section, or it will not be considered. Your application will be scored according to how well you address the requirements for each section.
• Reviewers will be looking for evidence of cultural competence in each section of the Project Narrative. Points will be assigned based on how well you address the cultural competence aspects of the evaluation criteria. SAMHSA's guidelines for cultural competence can be found on the SAMHSA Web site at www.samhsa.gov. Click on “Grant Opportunities.”
• The Supporting Documentation you provide in Sections E-H and Appendices 1-5 will be considered by reviewers in assessing your response, along with the material in the Project Narrative.
• The number of points after each heading below is the maximum number of points a review committee may assign to that section of your Project Narrative. Bullet statements in each section do not have points assigned to them. They are provided to invite the attention of applicants and reviewers to important areas within each section.

Section A: Statement of Need (10 points)

• Describe the target population (see Glossary) and the proposed catchment area (see Glossary), and justify the selection of both. Include the numbers to be served and demographic information. Discuss the target population's language, beliefs, norms and values, as well as socioeconomic factors that must be considered in delivering programs to this population.
• Document the need for an enhanced infrastructure to increase the capacity to implement, sustain, and improve effective substance abuse prevention and/or treatment and/or mental health services for the proposed target population in the proposed catchment area. Documentation of need may come from local data or trend analyses, State data (e.g., from State Needs Assessments), and/or national data (e.g., from SAMHSA's National Household Survey on Drug Abuse and Health or from National Center for Health Statistics/Centers for Disease Control reports). For data sources that are not well known, provide sufficient information on how the data were collected so reviewers can assess the reliability and validity of the data.
• Describe the service gaps, barriers, and other problems related to the need for infrastructure development. Describe the stakeholders (see Glossary) and resources in the target area that can help implement the needed infrastructure development.
• Non-tribal applicants must show that identified needs are consistent with priorities of the State or county that has primary responsibility for the service delivery system. Include, in Appendix 5, a copy of the State or County Strategic Plan, a State or county needs assessment, or a letter from the State or county indicating that the proposed project addresses a State- or county-identified priority. Tribal applicants must provide similar documentation relating to tribal priorities.
• Check the NOFA for any additional requirements.

Section B: Proposed Approach (35 points)

• Clearly state the purpose of the proposed project, with goals and objectives. Describe how achievement of goals will increase system capacity to support effective substance abuse and/or mental health services.
• Describe the proposed project. Provide evidence that the proposed activities meet the infrastructure needs and show how your proposed infrastructure development strategy will meet the goals and objectives.
• Provide a logic model (see Glossary) that demonstrates the linkage between the identified need, the proposed approach, and outcomes.
• If you plan to include an advisory body in your project, describe its membership, roles and functions, and frequency of meetings.
• Describe any other organizations that will participate and their roles and responsibilities. Demonstrate their commitment to the project. Include letters of commitment/coordination/support from these community organizations in Appendix 1 of the application. Identify any cash or in-kind contributions that will be made to the project.
• Describe how the proposed project will address issues of age, race/ethnicity, culture, language, sexual orientation, disability, literacy, and gender in the target population.
• Describe how members of the target population were involved in the preparation of the application, and how they will be involved in the planning, implementation, and evaluation of the project.
• Describe the potential barriers to successful conduct of the proposed project and how you will overcome them.
• Describe how your activities will improve substance abuse prevention and/or treatment and/or mental health services.
• Provide a plan to secure resources to sustain the proposed infrastructure enhancements when Federal funding ends.
• Check the NOFA for any additional requirements.

Section C: Staff, Management, and Relevant Experience (25 points)

• Provide a realistic time line for the project (chart or graph) showing key activities, milestones, and responsible staff. [Note: The time line should be part of the Project Narrative. It should not be placed in an appendix.]
• Discuss the capability and experience of the applicant organization and other participating organizations with similar projects and populations, including experience in providing culturally appropriate/competent services.
• Provide a list of staff who will participate in the project, showing the role of each and their level of effort and qualifications. Include the Project Director and other key personnel, such as the evaluator and treatment/prevention personnel.
• Describe the racial/ethnic characteristics of key staff and indicate if any are members of the target population/community. If the target population is multi-linguistic, indicate if the staffing pattern includes bilingual and bicultural individuals.
• Describe the resources available for the proposed project e.g., facilities, equipment). If an implementation pilot is proposed that includes direct services, provide evidence that services will be provided in a location that is adequate, accessible, compliant with the Americans with Disabilities Act (ADA), and amenable to the target population.
• Check the NOFA for any additional requirements.

Section D: Evaluation and Data (30 points)

• Describe the process and outcome evaluation. Include specific performance measures and target outcomes related to the goals and objectives identified for the project in Section B of your Project Narrative.
• Document your ability to collect and report on the required performance measures as specified in the NOFA, including data required by SAMHSA to meet GPRA requirements. Specify and justify any additional measures you plan to use for your grant project.
• Describe plans for data collection, management, analysis, interpretation and reporting. Describe the existing approach to the collection of data, along with any necessary modifications. Be sure to include data collection instruments/interview protocols in Appendix 2.
• Discuss the reliability and validity of evaluation methods and instruments(s) in terms of the gender/age/culture of the target population.
• Describe how collection, analysis and reporting of performance data will be integrated into the evaluation activities.
• Check the NOFA for any additional requirements.

Note:

Although the budget for the proposed project is not a review criterion, the Review Group will be asked to comment on the appropriateness of the budget after the merits of the application have been considered.

B. Review and Selection Process

SAMHSA applications are peer-reviewed according to the review criteria listed above. For those programs where the individual award is over $100,000, applications must also be reviewed by the appropriate National Advisory Council.

C. Award Criteria

Decisions to fund a grant are based on:

• The strengths and weaknesses of the application as identified by peer reviewers and, when appropriate, approved by the appropriate National Advisory Council;
• Availability of funds; and
• Equitable distribution of awards in terms of geography (including urban, rural and remote settings) and balance among target populations and program size.

VI. Award Administration Information

A. Award Notices

After your application has been reviewed, you will receive a letter from SAMHSA through postal mail that describes the general results of the review, including the score that your application received.

If you are approved for funding, you will receive an additional notice, the Notice of Grant Award, signed by SAMHSA's Grants Management Officer. The Notice of Grant Award is the sole obligating document that allows the grantee to receive Federal funding for work on the grant project. It is sent by postal mail and is addressed to the contact person listed on the face page of the application.

If you are not funded, you can re-apply if there is another receipt date for the program.

B. Administrative and National Policy Requirements

• You must comply with all terms and conditions of the grant award. SAMHSA's standard terms and conditions are available on the SAMHSA Web site (www.samhsa.gov).
• Depending on the nature of the specific funding opportunity and/or the proposed project as identified during review, additional terms and conditions may be identified in the NOFA or negotiated with the grantee prior to grant award. These may include, for example:
• Actions required to be in compliance with human subjects requirements;
• Requirements relating to additional data collection and reporting;
• Requirements relating to participation in a cross-site evaluation; or
• Requirements to address problems identified in review of the application.
• You will be held accountable for the information provided in the application relating to performance targets. SAMHSA program officials will consider your progress in meeting goals and objectives, as well as your failures and strategies for overcoming them, when making an annual recommendation to continue the grant and the amount of any continuation award. Failure to meet stated goals and objectives may result in suspension or termination of the grant award, or in reduction or withholding of continuation awards.
• In an effort to improve access to funding opportunities for applicants, SAMHSA is participating in the U.S. Department of Health and Human Services “Survey on Ensuring Equal Opportunity for Applicants.” This survey is included in the application kit for SAMHSA grants. Applicants are encouraged to complete the survey and return it, using the instructions provided on the survey form.

C. Reporting Requirements

1. Progress and Financial Reports

• Grantees must provide annual and final progress reports. The final progress report must summarize information from the annual reports, describe the accomplishments of the project, and describe next steps for implementing plans developed during the grant period.
• Grantees must provide annual and final financial status reports. These reports may be included as separate sections of annual and final progress reports or can be separate documents. Because SAMHSA is extremely interested in ensuring that infrastructure development and enhancement efforts can be sustained, your financial reports must explain plans to ensure the sustainability (see Glossary) of efforts initiated under this grant. Initial plans for sustainability should be described in year 1 of the grant. In each subsequent year, you should describe the status of the project, successes achieved and obstacles encountered in that year.
• SAMHSA will provide guidelines and requirements for these reports to grantees at the time of award and at the initial grantee orientation meeting after award. SAMHSA staff will use the information contained in the reports to determine the grantee's progress toward meeting its goals.

2. Government Performance and Results Act

The Government Performance and Results Act (GPRA) mandates accountability and performance-based management by Federal agencies. To meet the GPRA requirements, SAMHSA must collect performance data (i.e., “GPRA data”) from grantees. These requirements will be specified in the NOFA for each funding opportunity.

3. Publications

If you are funded under this grant program, you are required to notify the Government Project Officer (GPO) and SAMHSA's Publications Clearance Officer (301-443-8596) of any materials based on the SAMHSA-funded project that are accepted for publication.

In addition, SAMHSA requests that grantees:

• Provide the GPO and SAMHSA Publications Clearance Officer with advance copies of publications.
• Include acknowledgment of the SAMHSA grant program as the source of funding for the project.
• Include a disclaimer stating that the views and opinions contained in the publication do not necessarily reflect those of SAMHSA or the U.S. Department of Health and Human Services, and should not be construed as such.

SAMHSA reserves the right to issue a press release about any publication deemed by SAMHSA to contain information of program or policy significance to the substance abuse treatment/substance abuse prevention/mental health services community.

VII. Agency Contacts

The NOFAs provide contact information for questions about program issues.

For questions on grants management issues, contact: Stephen Hudak, Office of Program Services, Division of Grants Management, Substance Abuse and Mental Health Services Administration, 5600 Fishers Lane, Rockwall II 6th Floor, Rockville, MD 20857, [phone], [e-mail].

VIII. Other Information

A. SAMHSA Confidentiality and Participant Protection Requirements and Protection of Human Subjects Regulations

You must describe your procedures relating to Confidentiality, Participant Protection and the Protection of Human Subjects Regulations in Section H of your application, using the guidelines provided below. Problems with confidentiality, participant protection, and protection of human subjects identified during peer review of your application may result in the delay of funding.

Confidentiality and Participant Protection: All applicants must address each of the following elements relating to confidentiality and participant protection. You must describe how you will address these requirements.

1. Protect Clients and Staff From Potential Risks

• Identify and describe any foreseeable physical, medical, psychological, social, and legal risks or potential adverse effects as a result of the project itself or any data collection activity.
• Describe the procedures you will follow to minimize or protect participants against potential risks, including risks to confidentiality.
• Identify plans to provide guidance and assistance in the event there are adverse effects to participants.
• Where appropriate, describe alternative treatments and procedures that may be beneficial to the participants. If you choose not to use these other beneficial treatments, provide the reasons for not using them.

2. Fair Selection of Participants

• Describe the target population(s) for the proposed project. Include age, gender, and racial/ethnic background and note if the population includes homeless youth, foster children, children of substance abusers, pregnant women, or other targeted groups.
• Explain the reasons for including groups of pregnant women, children, people with mental disabilities, people in institutions, prisoners, and individuals who are likely to be particularly vulnerable to HIV/AIDS.
• Explain the reasons for including or excluding participants.
• Explain how you will recruit and select participants. Identify who will select participants.

3. Absence of Coercion

• Explain if participation in the project is voluntary or required. Identify possible reasons why participation is required, for example, court orders requiring people to participate in a program.
• If you plan to compensate participants, state how participants will be awarded incentives (e.g., money, gifts, etc.).
• State how volunteer participants will be told that they may receive services intervention even if they do not participate in or complete the data collection component of the project.

4. Data Collection

• Identify from whom you will collect data (e.g., from participants themselves, family members, teachers, others). Describe the data collection procedures and specify the sources for obtaining data (e.g., school records, interviews, psychological assessments, questionnaires, observation, or other sources). Where data are to be collected through observational techniques, questionnaires, interviews, or other direct means, describe the data collection setting.
• Identify what type of specimens (e.g., urine, blood) will be used, if any. State if the material will be used just for evaluation or if other use(s) will be made. Also, if needed, describe how the material will be monitored to ensure the safety of participants.
• Provide in Appendix 2, “Data Collection Instruments/Interview Protocols,” copies of all available data collection instruments and interview protocols that you plan to use.

5. Privacy and Confidentiality

• Explain how you will ensure privacy and confidentiality. Include who will collect data and how it will be collected.
• Describe:
• How you will use data collection instruments.
• Where data will be stored.
• Who will or will not have access to information.
• How the identity of participants will be kept private, for example, through the use of a coding system on data records, limiting access to records, or storing identifiers separately from data.

Note:

If applicable, grantees must agree to maintain the confidentiality of alcohol and drug abuse client records according to the provisions of Title 42 of the Code of Federal Regulations, Part II.

6. Adequate Consent Procedures

• List what information will be given to people who participate in the project. Include the type and purpose of their participation. Identify the data that will be collected, how the data will be used and how you will keep the data private.
• State:
• Whether or not their participation is voluntary.
• Their right to leave the project at any time without problems.
• Possible risks from participation in the project.
• Plans to protect clients from these risks.
• Explain how you will get consent for youth, the elderly, people with limited reading skills, and people who do not use English as their first language.

Note:

If the project poses potential physical, medical, psychological, legal, social or other risks, you must obtain written informed consent.

• Indicate if you will obtain informed consent from participants or assent from minors along with consent from their parents or legal guardians. Describe how the consent will be documented. For example: Will you read the consent forms? Will you ask prospective participants questions to be sure they understand the forms? Will you give them copies of what they sign?
• Include, as appropriate, sample consent forms that provide for: (1) Informed consent for participation in service intervention; (2) informed consent for participation in the data collection component of the project; and (3) informed consent for the exchange (releasing or requesting) of confidential information. The sample forms must be included in Appendix 3, “Sample Consent Forms”, of your application. If needed, give English translations.

Note:

Never imply that the participant waives or appears to waive any legal rights, may not end involvement with the project, or releases your project or its agents from liability for negligence.

• Describe if separate consents will be obtained for different stages or parts of the project. For example, will they be needed for both participant protection in treatment intervention and for the collection and use of data?
• Additionally, if other consents (e.g., consents to release information to others or gather information from others) will be used in your project, provide a description of the consents. Will individuals who do not consent to having individually identifiable data collected for evaluation purposes be allowed to participate in the project?

7. Risk/Benefit Discussion

Discuss why the risks are reasonable compared to expected benefits and importance of the knowledge from the project.

Protection of Human Subjects Regulations

Depending on the evaluation and data collection requirements of the particular funding opportunity for which you are applying or the evaluation design you propose in your application, you may have to comply with the Protection of Human Subjects Regulations (45 CFR part 46). The NOFA will indicate whether all applicants for a particular funding opportunity must comply with the Protection of Human Subject Regulations.

Applicants must be aware that even if the Protection of Human Subjects Regulations do not apply to all projects funded under a given funding opportunity, the specific evaluation design proposed by the applicant may require compliance with these regulations.

Applicants whose projects must comply with the Protection of Human Subjects Regulations must describe the process for obtaining Institutional Review Board (IRB) approval fully in their applications. While IRB approval is not required at the time of grant award, these applicants will be required, as a condition of award, to provide the documentation that an Assurance of Compliance is on file with the Office for Human Research Protections (OHRP) and that IRB approval has been received prior to enrolling any clients in the proposed project.

Additional information about Protection of Human Subjects Regulations can be obtained on the web at http://ohrp.osophs.dhhs.gov. You may also contact OHRP by e-mail ([e-mail]) or by phone [phone]).

B. Intergovernmental Review (E.O. 12372) Instructions

Executive Order 12372, as implemented through Department of Health and Human Services (DHHS) regulation at 45 CFR Part 100, sets up a system for State and local review of applications for Federal financial assistance. A current listing of State Single Points of Contact (SPOCs) is included in the application kit and can be downloaded from the Office of Management and Budget (OMB) Web site at www.whitehouse.gov/​omb/​grants/​spoc.html.

• Check the list to determine whether your State participates in this program. You do not need to do this if you are a federally recognized Indian tribal government.
• If your State participates, contact your SPOC as early as possible to alert him/her to the prospective application(s) and to receive any necessary instructions on the State's review process.
• For proposed projects serving more than one State, you are advised to contact the SPOC of each affiliated State.
• The SPOC should send any State review process recommendations to the following address within 60 days of the application deadline: Substance Abuse and Mental Health Services Administration, Office of Program Services, Review Branch, 5600 Fishers Lane, Room 17-89, Rockville, Maryland, 20857, ATTN: SPOC—Funding Announcement No. [fill in pertinent funding opportunity number from the NOFA].

C. Public Health System Impact Statement (PHSIS)

The Public Health System Impact Statement or PHSIS (Approved by OMB under control no. 0920-0428; see burden statement below) is intended to keep State and local health officials informed of proposed health services grant applications submitted by community-based, non-governmental organizations within their jurisdictions. State and local governments and Indian tribal government applicants are not subject to the following Public Health System Reporting Requirements.

Community-based, non-governmental service providers who are not transmitting their applications through the State must submit a PHSIS to the head(s) of the appropriate State and local health agencies in the area(s) to be affected no later than the pertinent receipt date for applications. This PHSIS consists of the following information:

• A copy of the face page of the application (SF 424); and
• A summary of the project, no longer than one page in length, that provides: (1) A description of the population to be served, (2) A summary of the services to be provided, and (3) a description of the coordination planned with appropriate State or local health agencies.

For SAMHSA grants, the appropriate State agencies are the Single State Agencies (SSAs) for substance abuse and mental health. A listing of the SSAs can be found on SAMHSA's Web site at www.samhsa.gov. If the proposed project falls within the jurisdiction of more than one State, you should notify all representative SSAs.

Applicants who are not the SSA must include a copy of a letter transmitting the PHSIS to the SSA in Appendix 4, “Letter to the SSA.” The letter must notify the State that, if it wishes to comment on the proposal, its comments should be sent not later than 60 days after the application deadline to: Substance Abuse and Mental Health Services Administration, Office of Program Services, Review Branch, 5600 Fishers Lane, Room 17-89, Rockville, Maryland, 20857, ATTN: SSA—Funding Announcement No. [fill in pertinent funding opportunity number from NOFA].

In addition:

• Applicants may request that the SSA send them a copy of any State comments.
• The applicant must notify the SSA within 30 days of receipt of an award.

[Public reporting burden for the Public Health System Reporting Requirement is estimated to average 10 minutes per response, including the time for copying the face page of SF 424 and the abstract and preparing the letter for mailing. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0920-0428. Send comments regarding this burden to CDC Clearance Officer, 1600 Clifton Road, MS D-24, Atlanta, GA 30333, ATTN: PRA (0920-0428).]

Appendix A—Checklist for Application Formatting Requirements

Your application must adhere to these formatting requirements. Failure to do so will result in your application being screened out and returned to you without review. In addition to these formatting requirements, there may be programmatic requirements specified in the NOFA. Please check the NOFA before preparing your application.

• Use the PHS 5161-1 application.
• The 10 application components required for SAMHSA applications must be included (i.e., Face Page, Abstract, Table of Contents, Budget Form, Project Narrative and Supporting Documentation, Appendices, Assurances, Certifications, Disclosure of Lobbying Activities, and Checklist.)
• Text must be legible.
• Paper must be white paper and 8.5″ by 11.0″ in size.
• Pages must be single-spaced with one column per page.
• Margins must be at least one inch.
• Type size in the Project Narrative cannot exceed an average of 15 characters per inch when measured with a ruler. (Type size in charts, tables, graphs, and footnotes will not be considered in determining compliance.)
• Photo reduction or condensation of type cannot be closer than 15 characters per inch or 6 lines per inch.
• Pages cannot have printing on both sides.
• Page limitations specified for the Project Narrative (25 pages) and Appendices 1, 3, and 4 (30 pages) cannot be exceeded.
• Information provided must be sufficient for review.
• Applications must be received by the application deadline. Applications received after this date must have a proof of mailing date from the carrier dated at least 1 week prior to the due date. Private metered postmarks are not acceptable as proof of timely mailing. Applications not received by the application deadline or postmarked a week prior to the application deadline will not be reviewed.
• Applications that do not comply with the following requirements and any additional program requirements specified in the NOFA, or are otherwise unresponsive to PA guidelines, will be screened out and returned to the applicant without review:
• Provisions relating to confidentiality, participant protection and the protection of human subjects specified in Section VIII-A of this document.
• Budgetary limitations as specified in Sections I, II and IV-E of this document.
• Documentation of nonprofit status as required in the PHS 5161-1.

To facilitate review of your application, follow these additional guidelines. Failure to follow these guidelines will not result in your application being screened out. However, following these guidelines will help reviewers to consider your application.

• Please use black ink and number pages consecutively from beginning to end so that information can be located easily during review of the application. The cover page should be page 1, the abstract page should be page 2, and the table of contents page should be page 3. Appendices should be labeled and separated from the Project Narrative and budget section, and the pages should be numbered to continue the sequence.
• Send the original application and two copies to the mailing address in the PA. Please do not use staples, paper clips, and fasteners. Nothing should be attached, stapled, folded, or pasted. Do not use any material that cannot be copied using automatic copying machines. Odd-sized and oversized attachments such as posters will not be copied or sent to reviewers. Do not include videotapes, audiotapes, or CD-ROMs.

Appendix B—Glossary

Best Practice: Best practices are practices that incorporate the best objective information currently available regarding effectiveness and acceptability.

Catchment Area: A catchment area is the geographic area from which the target population to be served by a program will be drawn.

Cooperative Agreement: A cooperative agreement is a form of Federal grant. Cooperative agreements are distinguished from other grants in that, under a cooperative agreement, substantial involvement is anticipated between the awarding office and the recipient during performance of the funded activity. This involvement may include collaboration, participation, or intervention in the activity. HHS awarding offices use grants or cooperative agreements (rather than contracts) when the principal purpose of the transaction is the transfer of money, property, services, or anything of value to accomplish a public purpose of support or stimulation authorized by Federal statute. The primary beneficiary under a grant or cooperative agreement is the public, as opposed to the Federal Government.

Cost-Sharing or Matching: Cost-sharing refers to the value of allowable non-Federal contributions toward the allowable costs of a Federal grant project or program. Such contributions may be cash or in-kind contributions. For SAMHSA grants, cost-sharing or matching is not required, and applications will not be screened out on the basis of cost-sharing. However, applicants often include cash or in-kind contributions in their proposals as evidence of commitment to the proposed project. This is allowed, and this information may be considered by reviewers in evaluating the quality of the application.

Fidelity: Fidelity is the degree to which a specific implementation of a program or practice resembles, adheres to, or is faithful to the evidence-based model on which it is based. Fidelity is formally assessed using rating scales of the major elements of the evidence-based model. A toolkit on how to develop and use fidelity instruments is available from the SAMHSA-funded Evaluation Technical Assistance Center at http://tecathsri.org or by calling [phone].

Grant: A grant is the funding mechanism used by the Federal Government when the principal purpose of the transaction is the transfer of money, property, services, or anything of value to accomplish a public purpose of support or stimulation authorized by Federal statute. The primary beneficiary under a grant or cooperative agreement is the public, as opposed to the Federal Government.

In-Kind Contribution: In-kind contributions toward a grant project are non-cash contributions (e.g., facilities, space, services) that are derived from non-Federal sources, such as State or sub-State non-Federal revenues, foundation grants, or contributions from other non-Federal public or private entities.

Logic Model: A logic model is a diagrammatic representation of a theoretical framework. A logic model describes the logical linkages among program resources, conditions, strategies, short-term outcomes, and long-term impact. More information on how to develop logics models and examples can be found through the resources listed in Appendix C.

Practice: A practice is any activity, or collective set of activities, intended to improve outcomes for people with or at risk for substance abuse and/or mental illness. Such activities may include direct service provision, or they may be supportive activities, such as efforts to improve access to and retention in services, organizational efficiency or effectiveness, community readiness, collaboration among stakeholder groups, education, awareness, training, or any other activity that is designed to improve outcomes for people with or at risk for substance abuse or mental illness.

Practice Support System: This term refers to contextual factors that affect practice delivery and effectiveness in the pre-adoption phase, delivery phase, and post-delivery phase, such as (a) community collaboration and consensus building, (b) training and overall readiness of those implementing the practice, and (c) sufficient ongoing supervision for those implementing the practice.

Stakeholder: A stakeholder is an individual, organization, constituent group, or other entity that has an interest in and will be affected by a proposed grant project.

Sustainability: Sustainability is the ability to continue a program or practice after SAMHSA grant funding has ended.

Target Population: The target population is the specific population of people whom a particular program or practice is designed to serve or reach.

Wraparound Service: Wraparound services are non-clinical supportive services—such as child care, vocational, educational, and transportation services—that are designed to improve the individual's access to and retention in the proposed project.

Appendix C—Logic Model Resources

Chen, W.W., Cato, B.M., & Rainford, N. (1998-9). Using a logic model to plan and evaluate a community intervention program: A case study. International Quarterly of Community Health Education, 18(4), 449-458.

Edwards, E.D., Seaman, J.R., Drews, J., & Edwards, M.E. (1995). A community approach for Native American drug and alcohol prevention programs: A logic model framework. Alcoholism Treatment Quarterly, 13(2), 43-62.

Hernandez, M. & Hodges, S. (2003). Crafting Logic Models for Systems of Care: Ideas into Action. [Making children's mental health services successful series, volume 1]. Tampa, FL: University of South Florida, The Louis de la Parte Florida Mental Health Institute, Department of Child & Family Studies. http://cfs.fmhi.usf.edu or phone [phone]

Hernandez, M. & Hodges, S. (2001). Theory-based accountability. In M. Hernandez & S. Hodges (Eds.), Developing Outcome Strategies in Children's Mental Health, pp. 21-40. Baltimore: Brookes.

Julian, D.A. (1997). Utilization of the logic model as a system level planning and evaluation device. Evaluation and Planning, 20(3), 251-257.

Julian, D.A., Jones, A., & Deyo, D. (1995). Open systems evaluation and the logic model: Program planning and evaluation tools. Evaluation and Program Planning, 18(4), 333-341.

Patton, M.Q. (1997). Utilization-Focused Evaluation (3rd Ed.), pp. 19, 22, 241. Thousand Oaks, CA: Sage.

Wholey, J.S., Hatry, H.P., Newcome, K.E. (Eds.) (1994). Handbook of Practical Program Evaluation. San Francisco, CA: Jossey-Bass Inc.

Dated: November 13, 2003.

Daryl Kade,

Director, Office of Policy, Planning and Budget, Substance Abuse and Mental Health Services Administration.