2024-00303. Agency Information Collection Activities: Proposed Collection; Comment Request

In compliance with section 3506(c)(2)(A) of the Paperwork Reduction Act of 1995 concerning opportunity for public comment on proposed collections of information, the Substance Abuse and Mental Health Services Administration (SAMHSA) will publish periodic summaries of proposed projects. To request more information on the proposed projects or to obtain a copy of the information collection plans, contact the SAMHSA Reports Clearance Officer at [e-mail].

Comments are invited on: (a) whether the proposed collections of information are necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility; (b) the accuracy of the agency's estimate of the burden of the proposed collection of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Project: 2023–2026 Advancing Wellness and Resilience in Education and Trauma Informed Services in Schools Cross-Site Evaluation—New Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Mental Health Services (CMHS) is requesting clearance for data collection associated with a national cross-site evaluation of process, outcomes, and impact for the Advancing Wellness and Resilience in Education (hereinafter referred to as Project AWARE) and Trauma-Informed Services in Schools (TISS) programs.

The purpose of the Project AWARE–TISS Cross-Site Evaluation is to better understand how each program is implemented, the extent to which they facilitate collaboration between education agencies and mental health systems, and how each program contributes to access and referral to mental health services and improved outcomes for youth.

The AWARE–TISS Cross-Site Evaluation incorporates four evaluation components to provide a robust understanding of the implementation (process), outcomes, and associated impacts of the AWARE and TISS Programs and includes program-specific components to ensure programmatic differences and commonalities are understood. With this integrated evaluation design, SAMHSA maintains the ability to evaluate and report on each program separately, while additionally benefiting from the ability to understand the overarching impact of both programs collectively.

Approval is being requested for data collection associated with a Process Evaluation and an Outcome Evaluation. Several program specific sub-studies and cross-program impact analyses will also be conducted to assess implementation and outcomes overall as well as those outcomes specific to high-need subpopulations and under-resourced communities. A behavioral health equity and cultural equity lens will be applied to each area of evaluation to ensure a culturally specific understanding of intervention implementation, outcomes, and impacts.

The Process Evaluation will contain two studies (Implementation and Sustainability Study and Systems Change Study) that examine strategies common to both programs related to program implementation facilitators and barriers, workforce development, and grantees' plans to sustain critical program components beyond their grant period. This assessment of common elements will provide a means to compare the implementation strategies that are successful across both AWARE and TISS grantees and identify successes and challenges in changing systems, policies, service provision, and school climate; increasing behavioral health equity in access and service delivery; and increasing social and emotional development and well-being in school-aged children and youth. The Process Evaluation will also address implementation of program-specific components.

For AWARE, the evaluation will document how the grantees implement the three-tiered public health model in schools and the referral pathways to increase access to mental health promotion, prevention, and intervention. The evaluation will assess the grantee collaborative efforts and grantee activities intended to increase workforce capacity to identify the signs and symptoms of mental illness and ability to refer to appropriate services promptly.

For TISS, the Process Evaluation will focus on examining what innovative strategies the grantees use to increase access to trauma informed services for school-aged youth and how the collaborative efforts of grantees and their partners develop/improve a school-based system for identification, referral, early intervention, treatment, and supportive services. Additionally, the Process Evaluation will assess the implementation of training to improve school capacity to address trauma support needs and engagement of families and communities to increase awareness of the effects of trauma on children and youth.

The Outcome Evaluation will include two studies that examine important facets of the AWARE and TISS programs: (1) identification and referral infrastructure (Identification and Referral Study); and (2) youth resiliency and outcomes (Youth Resiliency and Outcomes Study). Both studies will provide critical information about the effectiveness of the AWARE and TISS programs in establishing and enhancing school-based mental health supports for students.

Program specific sub-studies, inclusive of two TISS case studies and an AWARE Suicide Awareness and Prevention Sub-Study, will be conducted to provide more extensive contextual and implementation information related to the AWARE and TISS programs.

Finally, in addition to assessing the process and outcomes of each of the AWARE and TISS programs, we will conduct two cross-program analyses that examine the associated impacts of the both programs on the establishment and enhancement of school-based mental health supports for students (Cross-Program Impact Analysis) and the relationships of program and contextual factors with outcomes (Behavioral Health Equity Cross-Study Analysis).

The proposed multimethod approach considers allowable and required activities, variation in the partnerships and provider networks/infrastructure, program settings, populations being served, the range of program implementation plans and goals, existing data systems, and grant infrastructures that support implementation and evaluation participation. In addition, the design considers the stage of implementation of currently funded grantees to seamlessly integrate cohorts appropriately into the evaluation studies.

Fourteen primary data collection activities compose the AWARE–TISS Cross-Site Evaluation.

Instrument Descriptions

IS: The IS is a web-based survey that includes questions on protocols, policies, and structures present as part of schools' AWARE and TISS implementation processes; school/community integration; barriers and facilitators to implementation, and sustainability capacity. The survey also includes questions to gather program-specific information—for example, implementation of the pyramid model and suicide prevention policies in the case of AWARE grantees and details on trauma-informed services in the case of TISS grantees. The IS will be completed by project coordinator and program staff representatives annually. IS data will inform the Implementation and Sustainability Study, AWARE Suicide Awareness and Prevention Sub-Study and Behavioral Health Equity Cross-Study Analysis.

IKII: Supplementing IS findings, IKIIs will be conducted to obtain in-depth information about AWARE and TISS program implementation and sustainability based on the perspectives of grantee program staff and local mental health system partner staff. In each year of the 3-year data collection period, individual semi-structured interviews will be conducted with key representatives of each grantee's collaborative partnership group. Questions will focus on partnership development, coordination, and shared decision-making; perspectives on implementation including challenges, strategies, and successes; contextual, systems, or other factors that affect implementation; and approaches to planning for program sustainability. Interviews will be conducted in person during training and technical assistance (TTA) site visits or virtually when needed. IKII data will inform the Implementation and Sustainability Study, TISS Case Studies and Behavioral Health Equity Cross-Study Analysis.

YFFG–Y and YFFG–F: The YFFG–Y and YFFG–F will be used to conduct focus groups with youth (aged 14–18 or older if appropriate) who attend schools implementing the AWARE or TISS programs and/or their parents/family representatives. The moderator guides will be semi-structured and include open-ended questions to understand experiences and perspectives related to school climate, positive supports, youth or parent engagement, student resiliency and coping skills, awareness of school-based programs or resources to promote mental health literacy and meet mental health needs, mental health resource availability, and satisfaction with the program. Youth and family focus groups will be conducted annually and will include youth or parents representing a sample of AWARE and TISS grantees per year, such that all grantees will participate in the focus groups at least once during the evaluation. The YFFG–Y and YFFG–F will inform the Implementation and Sustainability Study and Behavioral Health Equity Cross-Study Analysis. Data collected through the YFFG–Y will also inform the Youth and Resiliency Outcomes Study.

CPS: CPS is a web-based survey that assesses communication, working relationships, leadership, role-expectations, resources, and partner engagement/commitment. Respondents will also be asked whether their organization currently has a formal, signed memorandum of agreement with the grantee and what changes to policy and infrastructure have been made in the past year. State and local entities, including project coordinators, school administrators, and mental health providers, identified as partners by AWARE and TISS Grantees will be considered for participation. The CPS will be administered annually and will inform the Systems Change Study.

TSF: TSF is a web-based form that will be used annually by AWARE and TISS grantees to document training and educational seminars. It will include training dates, length of time of training (in hours), topic of the training, training objectives, format of training delivery (in-person, webinar, online asynchronous, etc.), intended audience, and number of training participants. It is estimated that grantees will conduct up to 10 trainings annually for different groups ( e.g., teachers, mental health professionals, instructional support personnel). The TSF will inform the Systems Change Study and AWARE Suicide Awareness and Prevention Sub-Study.

PFF: The PFF is a web-based form that assesses perceptions of immediate and longer-term benefits in training areas that research has linked to effective implementation and practice change. The PFF will be completed annually by grantee training participants after training events to gather perception of the training experience and perceived feasibility of using the information. The PFF will inform the Systems Change Study.

APPTS and TPPTS: The APPTS and TPPTS are web-based surveys intended to be taken before and after AWARE or TISS grantee trainings across the 3-year data collection period. The APPTS will be completed by a sample of training participants per AWARE grantee annually and assesses knowledge, attitudes, and beliefs related to identifying students in need of mental health services and referring them for mental health services, mental health literacy, attitudes, beliefs (including stigma), and self-efficacy to provide support and referrals to youth experiencing mental health symptoms. The TPPTS will be completed by sample of training participants per TISS grantee annually and assesses trainee's knowledge of and self-efficacy to use trauma-informed strategies in their work. The APPTS and TPPTS will inform the Systems Change Study.

WFS: The WFS is a web-based survey that assesses barriers and facilitators to training use and the extent to which participants identified students in need of mental health services and referred them to services. The WFS will be administered to approximately 50% of AWARE and TISS training participants that also completed the APPTS or TPPTS. The WFS will be completed 3- and 12-months after training events and will be used to measure behavioral changes and longer-term impact on systems and communities. The WFS will inform the Systems Change Study.

STCSS, SSCSS, and PCSS: The STCSS, SSCSS, and PCSS are web-based surveys assessing school climate and safety among students attending grantee LEAs, parents of students, school personnel, and LEA staff. Surveys will be administered in year one and in year three of the evaluation and assess availability and utilization of referral for services (for students, parents, and school staff), trauma-informed practices (for school staff), respect for diversity (for school staff), racial climate (for students). The STCSS, SSCSS, and PCC will inform the Systems Change Study.

SIRF: The SIRF is a web-based form that gathers existing data detailing each how youth in need of mental health, substance use, or trauma-specific support services were identified because of an AWARE or TISS program, whether and to which services youth were referred, and resulting services received. Establishing identification and referral systems, including coordination with support service providers equipped to meet the needs of youth, is a core component of AWARE and TISS grant requirements. The SIRF will be completed by grantee program staff for up to 100 youth annually per grantee as part of a record review for each youth identified and referred to support services. Information about the initial identification, including the location and pathway to identification ( e.g., individual, screening tool, staff role), is obtained, along with information about referrals and support services received following identification. The form also includes deidentified demographic information about the youth receiving the identification, referral, and follow-up care. SIRF data can be extracted from case records of school-based care coordinators or mental health providers, or other existing data sources, including any school staff, support service provider, and family members who make a mental health, substance use, or trauma-related identification and referral. No personal identifiers are requested on the SIRF. SIRF data will inform the Identification and Referral Study and Behavioral Health Equity Cross-Study Analysis.

The estimated response burden to collect this information associated with the AWARE–TISS Cross-Site Evaluation is as follows annualized over the requested 3-year clearance period is presented below. Annual Burden (hours) and Total Cost ($) are rounded to the nearest whole number.

Total and Annualized Averages: Respondents, Responses, and Hours

Instrument	Type of respondent	Number of respondents	Responses per respondent	Total number of responses	Burden per response (hours)	Annual burden (hours)	Hourly wage rate ($)	Total cost ($)
IS	Project Coordinator	143	1	143	0.5	72	1  $35.52	$2,557
IS	Program Staff	15	1	15	0.5	8	2  21.71	174
IKII	Project Coordinator	94	1	94	1	94	35.52	3,339
IKII	Mental Health Provider	141	1	141	1	141	3  69.39	9,784
IKII	School Administrator	47	1	47	1	47	4  54.21	2,548
YFFG–Y	Youth	79	1	79	1.5	119	5  7.25	863
YFGG–F	Parent of Youth	79	1	79	1.5	119	7.25	863
CPS	Project Coordinator	143	1	143	0.25	36	35.52	1,279
CPS	Program Staff	47	1	47	0.25	12	21.71	261
CPS	School Administrator	47	1	47	0.25	12	54.21	651
TSF	Program Staff	47	10	470	0.15	71	21.71	1,541
PFF	Program Trainee	2,775	1	2,775	0.15	416	6  26.81	11,153
APPTS	Program Trainee	4,000	2	8,000	0.25	2,000	26.81	53,620
TPPTS	Program Trainee	750	2	1,500	0.25	375	26.81	10,054
WFS	Program Trainee	2,391	2	4,782	0.25	1,196	26.81	32,065
PCSS	Parent of Youth	282	1	282	0.4	113	7.25	819
STCSS	Youth	282	1	282	0.4	113	7.25	819
SSCSS	School Staff	282	1	282	0.5	141	7  30.20	4,258
SSCSS	School Administrator	188	1	188	0.5	94	54.21	5,096
SIRF	Program Staff	47	100	4,700	0.5	2,350	21.71	51,019
Total		11,879		24,096		7,529		192,763
1  BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates average annual salary for Community and Social Service Specialists, All Other (code 21–1099); https://www.bls.gov/​oes/​current/​oes_​nat.htm#21-0000.
2  BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates average annual salary for Community and Social Service Assistants (code 21–1093); https://www.bls.gov/​oes/​current/​oes_​nat.htm#21-0000.
3  BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates average annual salary for Healthcare Diagnosing or Treating Practitioners (code 29–1000); https://www.bls.gov/​oes/​current/​naics5_​541720.htm#29-0000.
4  BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates average annual salary for Educational Administrators, All Other (code 11–9039); https://www.bls.gov/​oes/​current/​naics5_​541720.htm#11-0000.
5   https://www.usa.gov/​minimum-wage.
6  BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates average annual salary for Community and Social Service Occupations (code 21–0000); https://www.bls.gov/​oes/​current/​oes_​nat.htm#21-0000.
7  BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates average annual salary for Educational, Guidance, and Career Counselors and Advisors (code 21–2012); https://www.bls.gov/​oes/​current/​naics5_​541720.htm#21-0000.

Send comments to Carlos Graham, SAMHSA Reports Clearance Officer at [e-mail]. Written comments should be received by March 11, 2024.

Alicia Broadus,

Public Health Advisor.