2018-27041. Submission for OMB Review; Comment Request

Title: Regional Partnership Grant National Cross-Site Evaluation and Evaluation Technical Assistance.

OMB No.: 0970-NEW.

Description: The Children's Bureau (CB) within the Administration for Children and Families of the U.S. Department of Health and Human Services seeks approval to collect information for the Regional Partnership Grants to Increase the Well-being of and to Improve Permanency Outcomes for Children Affected by Substance Abuse (known as the Regional Partnership Grants Program or “RPG”) Cross-Site Evaluation and Evaluation-Related Technical Assistance project. The Child and Family Services Improvement and Innovation Act (Pub. L. 112-34) includes a targeted grants program (section 437(f) of the Social Security Act) that directs the Secretary of Health and Human Services to reserve a specified portion of the appropriation for these Regional Partnership Grants, to be used to improve the well-being of children affected by substance abuse. Under three prior rounds of RPG, the Children's Bureau has issued 74 grants to organizations such as child welfare or substance abuse treatment providers or family court systems to develop interagency collaborations and integration of programs, activities, and services designed to increase well-being, improve permanency, and enhance the safety of children who are in an out-of-home placement or are at risk of being placed in out-of-home care as a result of a parent's or caretaker's substance abuse. In 2017, CB awarded grants to a fourth cohort of 17 grantees and in 2018 they plan to award 10 grants to a fifth cohort.

The RPG cross-site evaluation will extend our understanding of what types of programs and services grantees provided to participants, how grantees leveraged their partnerships to coordinate services for children and families, and what the outcomes were for children and families enrolled in RPG programs. First, the cross-site evaluation will describe the characteristics of participants served by RPG programs, the types of services provided to families, the dosage of each type of service received by families, and the level of participant engagement with the services provided. Second, the cross-site will assess the coordination of partners' service systems (e.g., shared participant data, joint staff training) to better understand how partners' collaborative effort affects the array of services offered to families. The cross-site evaluation will also focus more deeply on the partnership between the child welfare and substance use disorder (SUD) treatment agencies, to add to the research base about how these agencies can collaborate to address the needs of children and families affected by SUD. Finally, the evaluation will assess the outcomes of children and adults served through the RPG program.

The evaluation is being undertaken by the Children's Bureau and its contractor Mathematica Policy Research. The evaluation is being implemented by Mathematica Policy Research and its subcontractor, WRMA Inc.

The RPG Cross-Site Evaluation will include the following data collection activities:

1. Site visits and key informant interviews. The cross-site evaluation team will visit up to 21 sites to better understand the partnership and coordination between the child welfare and SUD treatment agencies. The remaining six grantees will participate in telephone interviews to gather similar information about their design and implementation. The site visits and phone interviews will focus on the RPG planning process; how and why particular services were selected; the ability of the child welfare, substance use disorder treatment, and other service systems to collaborate and support quality implementation of the RPG services; challenges experienced; and the potential for sustaining the collaborations and services after RPG funding ends.

2. Partner survey. To describe the interagency collaboration within RPG sites, grantees and their partners will participate in an online survey once during the grant period. One person from each organization knowledgeable about the RPG program will be invited to participate in the survey. The survey will collect information about communication and service coordination among partners. The survey will also collect information on characteristics of strong partnerships (e.g., data sharing agreements, colocation of staff, referral procedures, and cross-staff training).

3. Semi-annual progress reports. The semi-annual progress reports will be used to obtain updated information from grantee project directors about their program operations and partnerships, including any changes from prior periods. The CB has tailored the semi-annual progress reports to collect information on grantees' programs and other services grantees implement, the target population for the RPG program, and grantees' perceived successes and challenges to implementation.

4. Enrollment, client, and service data. To document participant characteristics and their enrollment in RPG services, all grantees will provide data on family characteristics, and enrollment of and services provided to RPG families. These data include demographic information on family members, dates of entry into and exit from RPG services, and information on RPG service dosage. These data will be submitted on an ongoing basis by staff at the grantee organizations into an information system developed by the cross-site evaluation team.

5. Outcome and impact data. To measure participant outcomes, all grantees will collect self-administered standardized instruments from RPG adults. The standardized instruments used in RPG collect information on child well-being, adult and family functioning, and adult substance use. Grantees will share the responses on these self-report instruments with the cross-site evaluation team. Grantees will also obtain administrative data on a common set of child welfare and substance use disorder treatment data elements.

In addition to conducting local evaluations and participating in the RPG Cross-Site Evaluation, the RPG grantees are legislatively required to report performance indicators aligned with their proposed program strategies and activities. A key strategy of the RPG Cross-Site Evaluation is to minimize burden on the grantees by ensuring that the cross-site evaluation, which includes all grantees in a study that collects data to report on implementation, the partnerships, and participant characteristics and outcomes, fully meets the need for performance reporting. Thus, rather than collecting separate evaluation and performance indicator data, the grantees need only participate in the cross-site evaluation. In addition, using the standardized instruments that the Children's Bureau has specified will ensure that grantees have valid and reliable data on child and family outcomes for their local evaluations. The inclusion of an impact study conducted on a subset of grantees with rigorous designs will also provide the Children's Bureau, Congress, grantees, providers, and researchers with information about the effectiveness of RPG programs.

A 60-Day Federal Register Notice was published for this study on October 10, 2018. This 30-Day Federal Register Notice covers the following data collection activities: (1) The site visits with grantees; (2) the web-based survey of grantee partners (3) the semi-annual progress reports; (4) enrollment and service data provided by grantees; and (6) outcome and impact data provided by grantees.

Respondents. Respondents include grantee staff or contractors (such as local evaluators) and partner staff. Specific types of respondents and the expected number per data collection effort are noted in the burden table below.

Annual burden estimates. The following instruments are proposed for public comment under this 30-Day Federal Register Notice. Burden for all components is annualized over three years.

RPG Cross-Site Evaluation Annualized Burden Estimates

Data collection activity	Total number of respondents	Number of responses per respondent (each year)	Average burden hours per response (in hours)	Estimated total burden hours	Total annual burden hours
Site Visit and Key Informant Data Collection
Program director in-person interview	21	.33	2	42	14
Program manager/supervisor in-person interview	21	.33	1	21	7
Partner representative interviews	63	.33	1	63	21
Frontline staff interview	42	.33	1	42	14
Program director/manager phone interview	12	.33	1	4.0	12
Partner survey	135	.33	0.42	56.3	18.8
Enrollment, client and service data
Semi-annual progress reports	27	2	16.5	2,673	891
Case enrollment data	81	43	0.25	2,612.3	870.8
Case closure	81	43	0.017	174.2	58.1
Case closure—prenatal	81	33	0.017	133.7	44.6
Service log entries	162	2,288	0.03	37,065	12,355
Outcome and impact data
Administrative Data:
Obtain access to administrative data	27	1	42.6	3,450.6	1150.2
Report administrative data	27	2	144	23,328	7,776
Standardized instruments:
Review and adopt reporting templates	27	.33	8	216	72
Data entry for standardized instruments	27	130	1.25	13,162.5	4,387.5
Review records and submit	27	2	25	4,050	1,350
Data entry for comparison study sites (22 grantees)	22	130	1.25	10,725	3,575
Estimated Total Burden Hours				97,827	32,609

Additional Information: Copies of the proposed collection may be obtained by writing to the Administration for Children and Families, Office of Planning, Research and Evaluation, 330 C Street SW, Washington, DC 20201. Attention Reports Clearance Officer. All requests should be identified by the title of the information collection. Email address: [e-mail].

OMB Comment: OMB is required to make a decision concerning the collection of information between 30 and 60 days after publication of this document in the Federal Register. Therefore, a comment is best assured of having its full effect if OMB receives it within 30 days of publication. Written comments and recommendations for the proposed information collection should be sent directly to the following: Office of Management and Budget, Paperwork Reduction Project, Email: [e-mail], Attn: Desk Officer for the Administration for Children and Families.

Robert Sargis,

Reports Clearance Officer.