2013-22774. Proposed Information Collection Activity; Comment Request

Proposed Projects:

The President signed the Child and Family Services Improvement and Innovation Act (Pub. L. 112-34) into law on September 30, 2011. This act includes a targeted grants program (section 437(f) of the Social Security Act), which directs the Secretary of Health and Human Services (HHS) to reserve a specified portion for Regional Partnership Grants, designed to improve the well-being of children affected by parental substance abuse. On September 28, 2012, CB/ACYF awarded new 5-year RPG grants to 17 partnerships in 15 states. The overall objective of the Cross-Site Evaluation and Technical Assistance project (the RPG Cross-Site Evaluation) is to plan, develop, and implement a rigorous national cross-site evaluation of the RPG Grant Program, provide legislatively-mandated performance measurement, and furnish evaluation-related technical assistance to the grantees in order to improve the quality and rigor of their local evaluations. The project will evaluate the programs and activities conducted through the RPG Grant Program.

Title: RPG National Cross-Site Evaluation and Evaluation Technical Assistance.

OMB No.: New collection.

Description: The Children's Bureau within the Administration for Children and Families of the U.S. Department of Health and Human Services seeks approval to collect information for the Regional Partnership Grants to Increase the Well-being of and to Improve Permanency Outcomes for Children Affected by Substance Abuse (known as the Regional Partnership Grants Program or “RPG”) Cross-Site Evaluation and Evaluation-Related Technical Assistance project. Under RPG, the Children's Bureau has issued 17 grants to organizations such as child welfare or substance abuse treatment providers or family court systems to develop interagency collaborations and integration of programs, activities, and services designed to increase well-being, improve permanency, and enhance the safety of children who are in an out-of-home placement or are at risk of being placed in out-of-home care as a result of a parent's or caretaker's substance abuse. The Child and Family Services Improvement and Innovation Act (Pub. L. 112-34) includes a targeted grants program (section 437(f) of the Social Security Act) that directs the Secretary of Health and Human Services to reserve a specified portion of the appropriation for these Regional Partnership Grants, to be used to improve the well-being of children affected by substance abuse. The overall objective of the Cross-Site Evaluation and Technical Assistance project (the RPG Cross-Site Evaluation) is to plan, develop, and implement a rigorous national cross-site evaluation of the RPG Grant Program, provide legislatively-mandated performance measurement, and furnish evaluation-related technical assistance to the grantees in order to improve the quality and rigor of their local evaluations. The project will evaluate the programs and activities conducted through the RPG Grant Program. The evaluation is being undertaken by the Children's Bureau and its contractor Mathematica Policy Research. The evaluation is being implemented by Mathematica Policy Research and its subcontractors, Walter R. McDonald & Associates and Synergy Enterprises.

The RPG Cross-Site Evaluation will include the following components:

1. Implementation and Partnership Study. The RPG cross-site implementation and partnership study will contribute to building the knowledge base about effective implementation strategies by examining the process of implementation in the 17 RPG projects, with a focus on factors shown in the research literature to be associated with quality implementation of evidence-based programs. This component of the study will describe the RPG projects' target populations, selected interventions and their fit with the target populations, inputs to implementation, and actual services provided (including dosage, duration, content, adherence to curricula, and participant responsiveness). It will examine the key attributes of the regional partnerships that grantees develop (for example, partnerships among child welfare and substance abuse treatment providers, social services, and the courts). It will describe the characteristics and roles of the partner organizations, the extent of coordination and collaboration, and their potential to sustain the partnerships after the grant ends. Key data collection activities of the implementation and partnership study are: (1) Conducting site visits during which researchers will interview RPG program directors, managers, supervisors, and frontline staff who work directly with families; (2) administering a survey to frontline staff involved in providing direct services to children, adults, and families; (3) asking grantees to provide information about implementation and their partnerships as part of their federally required semi-annual progress reports; (4) obtaining service use data from grantees, enrollment date and demographics of enrollees, exit date and reason, and service participation, to be entered into a web-based system developed and operated by Mathematica Policy Research and its subcontractors; and (5) administering a survey to representatives of the partner organizations.

2. Outcomes Study. The goal of the outcomes study is to describe the changes that occur in children and families who participate in the RPG programs. This study will describe participant outcomes in five domains: (1) Child well-being, (2) family functioning/stability, (3) adult recovery from substance use, (4) child permanency, and (5) child safety. Two main types of outcome data will be used—both of which are being collected by RPG grantees: (1) Administrative child welfare and adult substance abuse treatment records and (2) standardized instruments administered to the parents and/or caregivers. The Children's Bureau is requiring grantees to obtain and report specified administrative records, and to use a prescribed set of standardized instruments. Grantees will provide these data to the Cross-Site Evaluation team twice a year by uploading them to a data system developed and operated by Mathematica Policy Research and its subcontractors.

3. Impact Study. The goal of the impact study is to assess the impact of the RPG interventions on child, adult, and family outcomes by comparing outcomes for people enrolled in RPG services to those in comparison groups, such as people who do not receive RPG services or receive only a subset of the services. The impact study will use demographic and outcome data on both program (treatment) and comparison groups from a subset of grantees with appropriate local evaluation designs such as randomized controlled trials or strong quasi-experimental designs; 8 of the 17 grantees have such designs. Site-specific impacts will be estimated for these eight grantees. Aggregated impact estimates will be created by pooling impact estimates across appropriate sites to obtain a more powerful summary of the effectiveness of RPG interventions.

In addition to conducting local evaluations and participating in the RPG Cross-Site Evaluation, the RPG grantees are legislatively required to report performance indicators aligned with their proposed program strategies and activities. A key strategy of the RPG Cross-Site Evaluation is to minimize burden on the grantees by ensuring that the cross-site evaluation, which includes all grantees in a study that collects data to report on implementation, the partnerships, and participant characteristics and outcomes, fully meets the need for performance reporting. Thus, rather than collecting separate evaluation and performance indicator data, the grantees need only participate in the cross-site evaluation. In addition, using the standardized instruments that the Children's Bureau has specified will ensure that grantees have valid and reliable data on child and family outcomes for their local evaluations. The inclusion of an impact study conducted on a subset of grantees with rigorous designs will also provide the Children's Bureau, Congress, grantees, providers, and researchers with information about the effectiveness of RPG programs. This 60-Day Notice covers the following data collection activities: (1) The site visits with grantees; (2) the web-based survey of frontline staff who provide direct services to children, adults, and families, and their supervisors; (3) the semi-annual progress reports; (4) enrollment and service data provided by grantees; (5) the web-based survey of grantee partners; and (6) outcome data provided by grantees.

Respondents: Respondents include grantee staff or contractors (such as local evaluators) and partner staff. Specific types of respondents and the expected number per data collection effort are noted in the burden table below.

Annual Burden Estimates

Instrument	Number of respondents	Number of responses per respondent	Average burden hours per response	Total burden hours
Program director individual interview	17	1	1.34	22.8
Program manager/supervisor group interview	153	1	1.34	205
Program manager/supervisor individual interviews	102	1	0.67	68.3
Frontline staff individual interviews	102	1	0.67	68.3
Semi-annual progress reports	17	2	16.5	561
Case enrollment log	51	30	0.25	382.5
Service log	102	780	0.05	3978
Staff survey	340	1	0.34	115.6
Partner Survey	340	1	0.34	115.6
Administrative data	17	2	93.5	3,179
Outcome master instrument (data entry and uploading)	17	2	189	6426
Impact master instrument (data entry and uploading)	8	2	69	1104

Estimated Total Annual Burden Hours: 15,490.

In compliance with the requirements of Section 506(c)(2)(A) of the Paperwork Reduction Act of 1995, the Administration for Children and Families is soliciting public comment on the specific aspects of the information collection described above. Copies of the proposed collection of information can be obtained and comments may be forwarded by writing to the Administration for Children and Families, Office of Planning, Research and Evaluation, 370 L'Enfant Promenade SW., Washington, DC 20447, Attn: ACF Reports Clearance Officer. Email address: [e-mail]. All requests should be identified by the title of the information collection.

The Department specifically requests comments on: (a) Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility; (b) the accuracy of the agency's estimate of the burden of the proposed collection of information; (c) the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. Consideration will be given to comments and suggestions submitted within 60 days of this publication.

Robert Sargis,

Reports Clearance Officer.