2016-09749. Final Priorities, Requirements, Definitions, and Selection Criteria-Performance Partnership Pilots for Disconnected Youth

AGENCY:

Office of Career, Technical, and Adult Education, Department of Education.

ACTION:

Final priorities, requirements, definitions, and selection criteria.

SUMMARY:

The Assistant Secretary for Career, Technical, and Adult Education (Assistant Secretary) announces priorities, requirements, definitions, and selection criteria under the Performance Partnership Pilots (P3) for Disconnected Youth competition. The Assistant Secretary may use the priorities, requirements, definitions, and selection criteria for competitions for fiscal year (FY) 2015 and later years. We take this action in order to support the identification of strong and effective pilots that are likely to achieve significant improvements in educational, employment, and other key outcomes for disconnected youth.

DATES:

Effective Date: These priorities, requirements, definitions, and selection criteria are effective May 31, 2016.

FOR FURTHER INFORMATION CONTACT:

Braden Goetz, U.S. Department of Education, 400 Maryland Avenue SW., Room 11141, PCP, Washington, DC 20202. Telephone: [phone] or by email: [e-mail].

If you use a telecommunications device for the deaf (TDD) or a text telephone (TTY), call the Federal Relay Service (FRS), toll free, at 1-800-877-8339.

SUPPLEMENTARY INFORMATION:

Executive Summary

Purpose of This Regulatory Action: The Assistant Secretary announces priorities, requirements, definitions, and selection criteria under the Performance Partnership Pilots (P3) for Disconnected Youth competition. The Assistant Secretary may use the priorities, requirements, definitions, and selection criteria for competitions for fiscal year (FY) 2015 and later years. We take this action in order to support the identification of strong and effective pilots that are likely to achieve significant improvements in educational, employment, and other key outcomes for disconnected youth.

Summary of the Major Provisions of This Regulatory Action

This regulatory action announces 13 priorities, 7 application requirements, 4 program requirements, 13 definitions, and 7 selection criteria that may be used for P3 competitions for FY 2015 and later years.

Costs and Benefits: The Department of Education (Department) believes that the benefits of this regulatory action outweigh any associated costs, which we believe will be minimal. The potential costs are those resulting from statutory requirements and those we have determined as necessary for administering P3. The benefits of the priorities, requirements, definitions, and selection criteria are that they would promote the efficient and effective use of the P3 authority. Please refer to the Regulatory Impact Analysis in this notice of final priorities, requirements, definitions, and selection criteria (NFP) for a more detailed discussion of costs and benefits.

Purpose of Program: P3, first authorized by Congress for FY 2014 by the Consolidated Appropriations Act, 2014 (2014 Appropriations Act) and reauthorized for FY 2015 by the Consolidated and Further Continuing Appropriations Act, 2015 (2015 Appropriations Act) and for FY 2016 by the Consolidated Appropriations Act, 2016 (2016 Appropriations Act) (together, the Acts), authorize the Departments of Education, Labor, Health and Human Services, Housing and Urban Development,^[1] and Justice,^[2] the Corporation for National and Community Service and the Institute of Museum and Library Services (collectively, the Agencies), to enter into Performance Partnership Agreements (performance agreements) with State, local, or tribal governments to provide additional flexibility in using certain of the Agencies' discretionary funds, including competitive and formula grant funds, across multiple Federal programs. The authority enables pilot sites to test innovative, outcome-focused strategies to achieve significant improvements in educational, employment, and other key outcomes for disconnected youth using new flexibility to blend existing Federal funds and to seek waivers of associated program requirements. Section 526(a)(2), Division H of the 2014 Appropriations Act states that “ `[t]o improve outcomes for disconnected youth' means to increase the rate at which individuals between the ages of 14 and 24 (who are low-income and either homeless, in foster care, involved in the juvenile justice system, unemployed, or not enrolled in or at risk of dropping out of an educational institution) achieve success in meeting educational, employment, or other key goals.”

Program Authority: Section 524 of Division G and section 219 of Division B of the Consolidated and Further Continuing Appropriations Act, 2015 (Pub. L. 113-235) and section 219 of Division B, section 525 of Division H, and section 242 of Division L of the Consolidated Appropriations Act, 2016 (Pub. L. 114-113).

We published a notice of proposed priorities, requirements, definitions, and selection criteria (NPP) in the Federal Register on October 22, 2015 (80 FR 63975). That notice contained background information and our reasons for proposing the particular priorities, requirements, definitions, and selection criteria. In response to public comment, this notice reduces burden on applicants by removing several application requirements that had been proposed in the NPP. This NFP also revises the priority for disconnected youth who are unemployed and out-of-school (Priority 4) to limit the priority to those unemployed and out-of-school youth who face significant barriers to accessing education and employment. Additionally, this NFP revises the priorities for projects designed to improve outcomes for subpopulations of high-need disconnected youth (i.e., youth who are unemployed and out of school, youth who are English Learners (ELs), youth with a disability, homeless youth, youth in foster care, youth involved in the justice system, and youth who are immigrants or refugees) to specify that, in order to meet the priority, a project must serve the particular subpopulation identified in the priority and be likely to result in significantly better educational or employment outcomes for the subpopulation. Finally, this NFP establishes an additional priority for projects that serve disconnected youth who are pregnant or parenting and that are likely to result in significantly better educational or employment outcomes for such youth.

Public Comment: In response to our invitation in the NPP, 11 parties submitted comments on the proposed priorities, requirements, definitions, and selection criteria.

We group major issues according to subject. Generally, we do not address technical and other minor changes.

Analysis of Comments and Changes: An analysis of the comments and of any changes in the priorities, requirements, definitions, and selection criteria since publication of the NPP follows.

General

Comment: One commenter recommended that we streamline and simplify the application process to permit applicants to submit brief letters that describe their requests for waivers in lieu of a formal application that meets the requirements and addresses the selection criteria proposed in the NPP. Two commenters expressed concern about the length of the selection process that identified the FY 2014 P3 pilots; one of these commenters recommended that, going forward, pilots be selected within one month of the application deadline.

Discussion: We acknowledge the commenters' concerns about the length and structure of the application and selection processes. In fact, many of the changes from the first competition that were proposed in the NPP were intended to streamline and simplify those processes. As we note later in our discussion of the comments on the proposed application requirements, the NFP makes additional changes to the application requirements with that same goal. We believe this will make the application process clearer and easier for applicants, and also shorten the timeline for the selection process.

However, we also note the importance of a thorough review of applications and engagement with potential pilots to ensure we collect all information required to complete a performance agreement. Such a review is critical to meeting the statutory conditions on granting waivers and awarding pilots. Some of the concerns raised by commenters will be addressed as the Agencies and the field gain experience with P3 and need not necessarily be addressed through rulemaking.

Changes: None.

Priorities

Comment: One commenter expressed opposition to the proposed priorities for special populations, such as youth who are ELs, contending that they would make the application process too complicated.

Discussion: We want to clarify the purpose of the priorities for different special populations. The statutory definition of disconnected youth for P3 is broad and includes youth who are at risk of dropping out in addition to youth who fall into other categories of eligible youth, such as those who are not employed or enrolled in school. The general purpose of these priorities is to focus attention on subpopulations of disconnected youth with great needs who might otherwise not be served or to address particular challenges that communities face in reaching these populations. The priorities are intended as options for use in future P3 competitions. The Agencies may choose which, if any, of the priorities included in this NFP are appropriate for a particular P3 competition and how the priority or priorities would apply. For example, a priority may be used as an absolute priority. This means that applicants that propose projects under that priority must address it to be eligible to be selected as a pilot. A priority could also be used as a competitive preference priority. This means that applicants who propose projects addressing that priority could receive additional points for their applications.

We acknowledge the commenter's general concern that a large number of priorities may make the application process more complicated. For that reason, although we publish seven priorities for different subpopulations in this NFP, we do not intend to use all of the subpopulation priorities in a single year's competition. Instead, for each year in which we hold a competition, we would likely choose no more than a few high-need subpopulations to emphasize.

Changes: None.

Comment: A commenter recommended that the special populations described by the proposed priorities be identified as illustrative examples of populations that could be served by a P3 project, rather than set out as priorities. The commenter was concerned that some subpopulations of disconnected youth were not included among the priorities proposed in the NPP. A second commenter noted that there is a significant number of disconnected youth who meet more than one of the proposed subpopulation priorities and expressed concern that applicants would be limited to serving only the subpopulation identified in a particular priority. The commenter encouraged us to affirm that applicants could serve youth with characteristics described by multiple priorities, such as, for example, a project that proposed to serve youth who have been involved in the justice system and who also are immigrants or refugees.

Discussion: We understand the commenters' concerns and wish to emphasize that the purpose of the subpopulation priorities is to create incentives for applicants to serve disconnected youth with great needs who might otherwise not be served or who may be difficult to reach. The use of the priorities in a given competition would not bar applicants from serving other disconnected youth who are included within the statutory definition of the term. Even if we were to use one of the subpopulation priorities as an absolute priority, the effect would be to require applicants to demonstrate how they will ensure that the subpopulation receives services. However, pilots would not be required to exclusively serve that subpopulation. Applicants also could serve youth with characteristics described by multiple priorities, such as, for example, a project that proposed to serve youth who have been involved in the justice system and who also are immigrants or refugees.

Changes: None.

Comment: One commenter recommended that the number of subpopulation priorities be reduced to focus on youth with the greatest needs.

Discussion: As we explained in the NPP, all of the specific subpopulations for which we proposed priorities in the NPP have great needs. It may be a matter of opinion, perspective, or local circumstances to say which subpopulation has the greatest needs. Therefore there is ample reason to encourage P3 pilots to use innovative approaches and flexibility to overcome the challenges these subpopulations face and generate improved outcomes for these youth. For example, in proposing a priority for youth who are ELs, we pointed out that the average cohort graduation rate for ELs was only 61 percent for the 2012-13 school year, while the national average cohort graduation rate for all youth was 81 percent. Similarly, in proposing a priority for youth who are homeless, we noted that these young people experience higher rates of acute and chronic physical illness and have higher rates of mental illness and substance abuse than their peers who have stable housing. We also noted that the high mobility associated with homelessness also disrupts the education of these youth, placing them at greater risk of falling behind and dropping out of school.

We agree, however, that the priority for disconnected youth who are unemployed and out-of-school (Priority 4) should be amended to ensure that it is focused on those youth within this subpopulation who have the most significant needs. We note that a recent analysis of 2014 Current Population Survey (CPS) data found that, while youth ages 16 to 24 who were neither employed nor enrolled in school were more likely than their peers to be poor in 2014, a majority of these youth (56 percent) did not live in poverty in that same year.^[3] Consequently, we believe it is appropriate to limit the priority to those unemployed and out-of-school youth who face significant barriers to education and employment. Such barriers could include, for example, having one or more disabilities or having been in the justice system. The same analysis of 2014 CPS data found that about one-third (34 percent) of youth ages 16 to 24 who were neither employed nor enrolled in school in 2014 reported that illness or disability was a major reason why they did not work. Involvement with the justice system is another example of a significant barrier to education and employment for youth who are neither employed nor enrolled in school. Many youth involved with the justice system face significant barriers to accessing the education and training they need to achieve independence and reintegrate into the community because the education and training available to them through correctional facilities, as well as upon release, often does not meet their needs.^[4] For older youth involved with the adult criminal justice system, having a criminal record can severely limit the ability to secure employment.^[5]

Changes: We have revised Priority 4 to limit it to apply to youth who are unemployed and out-of-school and who face significant barriers to accessing education and employment.

Comment: One commenter recommended we limit the applicability of the proposed priorities for subpopulations of disconnected youth to projects that would be likely to result in significant changes in the outcomes of the particular subpopulations identified in the priorities. Another commenter expressed support for the first commenter's proposal, but also recommended that we consider either limiting the subpopulation priorities to projects that would exclusively or principally serve these subpopulations or allow applicants to focus their applications on not more than one subpopulation identified in the priorities.

Discussion: We acknowledge the first commenter's suggestion that we limit the applicability of the priorities for subpopulations of disconnected youth to projects that would be likely to result in significant changes in the outcomes of the particular subpopulations they identify. We agree with the commenter. We disagree with the second commenter's recommendation that we revise the subpopulation priorities to require that projects principally or exclusively serve the subpopulations addressed in the priority because such a requirement may result in approaches that inappropriately segregate youth with special needs from their peers and reinforce program “silos” that P3 is intended to help communities break down. However, in the event that one of these subpopulation priorities is used as a competitive preference priority, we do think it would be appropriate to consider the extent to which an applicant would serve the particular subpopulation in assessing how well an application meets the priority. An applicant that proposed to serve a small number or percentage of the subpopulation could receive fewer points than an applicant that proposed to serve a larger number or percentage of the youth identified in the priority. We also acknowledge the second commenter's suggestion that we allow applicants to focus their applications on only one of the subpopulations identified in the priorities. We can accomplish that result without additional rulemaking. Should we decide to include two or more of the subpopulation priorities in any future P3 competition, we would have the opportunity to limit applicants to selecting only one of the priorities.

Changes: We have revised the priorities for the subpopulations of high-need disconnected youth (i.e., youth who are unemployed and out of school, youth who are ELs, youth with a disability, homeless youth, youth in foster care, youth involved in the justice system, and youth who are immigrants or refugees) to specify that, in order to meet the priority, a project must both serve the particular subpopulation identified in the priority and be likely to result in significantly better educational or employment outcomes for the particular subpopulation identified in the priority. Peer reviewers will determine whether or the extent to which an applicant meets the priority based on the evidence an applicant includes in its application.

Comment: One commenter recommended that we establish a priority for urban communities with high rates of poverty and unemployment that have experienced violent protests in recent years.

Discussion: We agree with the commenter that there are numerous urban communities with high rates of poverty, unemployment, and violence that would benefit from P3. However, the 2016 Appropriations Act requires that pilots selected for FY 2015 and FY 2016 by the Agencies include “communities that have recently experienced civil unrest.” This provision makes it unnecessary to use rulemaking to ensure such communities receive priority.

Changes: None.

Comment: Two commenters recommended that we establish a priority for projects that serve disconnected youth who are parents, including, particularly, projects that implement strategies that address the needs of both the parent and the child.

Discussion: We agree with the commenters that a priority for disconnected youth who are pregnant or parenting is appropriate because these adolescents and their children are at high risk for adverse outcomes. Adolescent childbearing, for example, significantly reduces the likelihood of the mother's earning a regular high school diploma, or completing at least two years of postsecondary education by age 30.^[6] Teenage parenting also has negative consequences for fathers; they, too, are less likely to earn a high school diploma, and they complete fewer years of schooling than their non-parenting peers.^[7] We also agree that two-generation strategies—that is, strategies that simultaneously address the needs of the parent and the needs of the child—can have great merit. To preserve the freedom of applicants to innovate and the flexibility inherent to P3, however, we do not believe a priority for disconnected youth who are pregnant or parenting should specify that two-generation strategies must always be used to address the priority.

Changes: We have established a priority (now Priority 11) for pilots that are likely to result in significantly better educational or employment outcomes for disconnected youth who are pregnant or parenting.

Comment: One commenter expressed support for establishing a priority for applicants whose State government had also agreed to provide flexibility to support implementation of the project.

Discussion: We recognize that flexibility from State and local requirements can be crucial to the successful implementation of a pilot. For that reason, the NFP includes the Application Requirement (c)(1)(A), which requires that an applicant provide written assurance that it has received any and all necessary state, local, or tribal flexibility, or will receive such flexibility within 60 days of being designated a pilot. However, we decline to create a separate priority for an applicant whose State has provided flexibility. We believe that the commenter's primary concern is whether the project design can be implemented effectively and will improve outcomes for disconnected youth. We do not believe there is additional benefit to a pilot that is able to implement effectively the pilots as designed due to a State government granting additional flexibility compared to one that has that ability regardless of State flexibility.

Changes: None.

Commenter: One commenter recommended that we establish a priority for projects that would be carried out by a partnership between a State, local, or tribal government and one or more non-governmental entities with experience and expertise in providing services to the population of youth who would be served.

Discussion: We agree that non-governmental entities can play valuable roles in the design, governance, and implementation of P3 pilots, but we decline to establish the recommended priority because we wish to preserve the flexibility of State, local, and tribal governments to innovate. For an initiative like P3 that seeks to provide State, local, and tribal governments greater flexibility in how they deliver services to disconnected youth, it would be inappropriately prescriptive to specify how and with which entities a pilot must engage to deliver services. We also note that this NFP includes a selection criterion that would evaluate applicants based on the strength and capacity of the proposed pilot partnership, which can include non-governmental entities.

Changes: None.

Comment: One commenter recommended that each of the proposed priorities for subpopulations of disconnected youth be amended to include a requirement that projects provide career assessment and/or vocational evaluation services.

Discussion: We agree with the commenter that career assessment and advising may be helpful to disconnected youth in identifying and pursuing their career goals. However, amending each of the subpopulation priorities to mandate the provision of such services would be inconsistent with P3's focus on increasing the flexibility of State, local, and tribal governments to innovate and design new solutions to improve the outcomes of disconnected youth.

Changes: None.

Comment: Two commenters recommended that we establish a priority for projects that serve a Promise Zone.

Discussion: We agree with the commenters that a priority for projects that serve a Promise Zone has great merit. We note, however, that the Department already established such a priority in an NFP that was published in the Federal Register on March 27, 2014 (79 FR 17035). Because it has already been established, this priority may be used in any appropriate discretionary grant competition carried out by the Department in FY 2014 and subsequent years.

Changes: None.

Final Priority 2—Improving Outcomes for Disconnected Youth in Rural Communities.

Comment: One commenter expressed support for establishing a priority for projects that serve rural communities only.

Discussion: We acknowledge the commenter's support.

Changes: None.

Final Priority 3—Improving Outcomes for Disconnected Youth in Tribal Communities

Comment: One commenter expressed support for the proposed priority for projects that serve disconnected youth who are members of one or more State- or federally-recognized Indian tribal communities and that represent a partnership that includes one or more State- or federally-recognized Indian tribes.

Discussion: We acknowledge the commenter's support.

Changes: None.

Final Priority 5—Improving Outcomes for Youth Who are English Learners.

Comment: Two commenters expressed support for the proposed priority for projects that serve disconnected youth who are ELs.

Discussion: We acknowledge the commenters' support.

Changes: None.

Final Priority 7—Improving Outcomes for Homeless Youth.

Comment: Two commenters expressed support for the proposed priority for projects that are designed to improve outcomes for disconnected youth who are homeless youth.

Discussion: We acknowledge the commenters' support.

Changes: None.

Final Priority 10—Improving Outcomes for Youth Who are Immigrants or Refugees.

Comment: Two commenters expressed support for the proposed priority for projects that are designed to improve outcomes for disconnected youth who are immigrants or refugees.

Discussion: We acknowledge the commenters' support.

Changes: None.

Comment: Two commenters recommended that we revise the priority for immigrants or refugees to exclude individuals who have J-1 or F-1 visas.

Discussion: Individuals who are visiting the United States temporarily with a J-1 or F-1 visa are not immigrants. The J-1 and F-1 visas are nonimmigrant visas that are issued to individuals who have a permanent residence outside the U.S. and who wish to visit the U.S. on a temporary basis. J-1 visa holders participate temporarily in work-and study-based exchange visitor programs, while F-1 visa holders attend, on a full-time basis, a university or college, high school, private elementary school, seminary, conservatory, language training program, or other academic institution.

Changes: None.

Final Priority 12—Work-Based Learning Opportunities.

Comment: One commenter expressed support for the proposed priority for projects that provide disconnected youth with paid work-based learning opportunities and encouraged us to require all projects to offer paid work-based learning opportunities to the youth they serve during the summer months. Another commenter expressed concern about the proposed priority, contending that work experience opportunities may not be readily available in communities with high rates of unemployment, that not all youth may be ready to participate in a work-based learning opportunity because they have an intellectual disability, and that some projects may serve younger youth who are not old enough to work. The commenter conceded, however, that these exceptions are areas where P3 pilot may be most needed.

Discussion: Although we acknowledge the first commenter's support for the priority and agree that paid work-based learning is an important intervention for disconnected youth, we decline to require all projects to offer paid work-based learning opportunities during the summer months in order to preserve the flexibility inherent to P3. However, we do agree that it is appropriate to revise the priority to specify that an applicant must provide paid work-based learning to all of the disconnected youth it proposes to serve in order to meet the priority. We understand the second commenter's concerns about the difficulty of securing paid work-based learning opportunities in areas with high unemployment, but believe that applicants can overcome these difficulties with some creativity and determination in their project designs, including by establishing partnerships with employers and other non-governmental entities. We do not share the commenter's view that work-based learning may not be appropriate for some youth with disabilities; we believe that all youth with disabilities can participate in, and benefit from, work-based learning if they are provided the right accommodations and supports. With respect to the concern about younger youth who are not old enough to work, we note that youth must be at least 14 years of age to be included within P3's statutory definition of disconnected youth. Under regulations issued by the Department of Labor to implement the Fair Labor Standards Act, youth who are age 14 may work outside school hours in various non-manufacturing, non-mining, non-hazardous jobs under certain conditions.^[8] Moreover, we note that work-based learning opportunities can include job shadowing and internships.

Changes: We have revised the priority to specify that an applicant must provide paid work-based learning to all of the disconnected youth it proposes to serve in order to meet the priority.

Final Priority 13—Site-Specific Evaluation.

Comment: Two commenters expressed support for our proposal to consolidate what had been two priorities for site-specific evaluation, one for randomized controlled trials and another for evaluations that use a quasi-experimental design, into a single priority.

Discussion: We acknowledge the commenters' support.

Changes: None.

Comment: Two commenters expressed opposition to the proposed priority for applications that propose to conduct independent evaluations of their programs or specific components of their programs. Both commenters argued that the priority would be duplicative because a national evaluation of P3 is now underway. One of the commenters also expressed concern that projects would not implement high-quality evaluations because applicants lacked expertise in carrying out evaluations.

Discussion: We disagree with the commenters that a priority for site-specific evaluations would be duplicative of the national evaluation of P3 that is being carried out by the U.S. Department of Labor. We believe that promoting independent evaluations that focus exclusively on the implementation of a particular pilot is important because such studies are likely to yield valuable insights that might be missed by a national evaluation that examines the implementation and outcomes of all of the pilots. Moreover, we note that the national evaluation is focused on the first cohort of P3 pilots and it is not yet known to what extent the Agencies will support additional national evaluations to examine the experiences of subsequent cohorts. We do not share the commenter's concern about applicants' lack of expertise in evaluation because applicants may seek out others with this expertise to assist them in designing and carrying out an independent evaluation. Applicants that do not have expertise in evaluation or obtain it from other sources are unlikely to meet the priority because the assessment of the extent to which an applicant meets the priority will be based on, among other factors, the applicant's demonstrated expertise in planning and conducting an evaluation using a randomized controlled trial or quasi-experimental design.

Changes: None.

Comment: One commenter recommended that the priority for site-specific evaluation be amended to require the evaluation to examine the types of career assessment services provided, the outcomes of those services, how many of the assessments' recommendations were followed, and the outcomes of those recommendations.

Discussion: We decline to mandate that the evaluation examine career assessment services because not all projects may include such services.

Changes: None.

Comment: None.

Discussion: Upon further review, we identified a typographical error in the second sentence of the proposed priority for site-specific evaluation. The second sentence of this priority used the term “quasi-experimental evaluation study.” The correct term, which is defined in the Education Department General Administrative Regulations (34 CFR 77.1) is “quasi-experimental design study.”

Changes: We have changed the reference to “quasi-experimental evaluation” in the second sentence of the priority to “quasi-experimental design study.”

Application Requirements

Comment: One commenter expressed concern that proposed Application Requirement (b), Statement of Need for a Defined Target Population, was similar to one of the proposed selection criteria. The commenter encouraged us either to clarify how the two provisions differed or to delete one of them. Another commenter contended that several proposed application requirements were duplicative because they sought information that applicants must provide in responding to the proposed selection criteria. That commenter recommended that we limit the application requirements to essential information that is not addressed by the selection criteria.

Discussion: The commenters are correct that several of the proposed application requirements sought narratives that applicants would have provided in responding to the proposed selection criteria. We proposed these application requirements in an effort to ensure that applicants provide this information so that reviewers can assess it in scoring the selection criteria. However, we acknowledge the concerns of the commenters that these proposed application requirements appear duplicative and are confusing rather than helpful.

Changes: We have revised four application requirements to remove requirements for narrative text that would be assessed by one or more of the selection criteria. The revisions we made in response to these comments are:

• In Application Requirement (b), Statement of Need for a Defined Target Population, we have removed the requirement that the applicant provide a narrative description of the target population. We have retained the requirement that the applicant complete Table 1 and specify the target population(s) for the pilot, including the range of ages of youth who will be served and the number of youth who will be served over the course of the pilot. We have also retitled the requirement “Target Population.”
• In Application Requirement (d), Project Design, we have removed the requirement that the applicant submit a narrative that describes the project, the needs of the target population, the activities or changes in practice that will be implemented, why the requested flexibility is necessary to implement the pilot, how the requested flexibility will enable the applicant to implement changes in practice, and the proposed length of the pilot. We have retained the requirement that the applicant submit a logic model and, consequently, we have renamed Application Requirement (d) “Logic Model.”
• We have deleted Application Requirement (e), Work Plan and Project Management.
• In Application Requirement (g), Budget and Budget Narrative (formerly Application Requirement (h)), we have revised the requirement to refer only to the budget and to require only the completion of Table 5. We have removed the requirement to provide a narrative regarding the amount and use of start-up funds, the proposed uses of funds named in Table 5, and the amount and sources of any non-Federal funds that may be used in the pilot. In addition, Table 5 has been revised to remove the rows that asked applicants to break out, for pilots proposed for multiple years, the amount and source of Federal funds that would be used in each calendar year of the project.

Comment: One commenter urged us to require applicants to provide evidence that the parties involved in the proposed project's implementation show evidence of prior collaboration through in-kind commitments, braided funding, or shared services.

Discussion: We decline to impose the recommended requirement because it would be duplicative. The extent to which partners in the proposed project have successfully collaborated to improve outcomes for disconnected youth in the past is among the factors assessed by Selection Criterion (e)(1). Additionally, the recommended requirement is inappropriately prescriptive. To be effective, collaboration need not always involve in-kind commitments, braided funding, or shared services.

Changes: None.

Comment: One commenter commended us for giving applicants some flexibility in selecting the indicators and outcome measures that would be used to evaluate their projects, but suggested that we establish a small, common set of outcome measures that all pilots would use. The commenter recommended that we make placement and retention in school and/or placement and retention in employment required outcome measures for all pilots.

Discussion: As the commenter acknowledged, both the interventions implemented and the populations served can be diverse across P3 projects, making it difficult to identify appropriate indicators and outcome measures that should apply to all projects. We do see merit, however, in having a menu of indicators and outcome measures from which applicants may choose so that similar projects use common indicators and outcome measures, facilitating comparisons in performance across the P3 pilots.

Changes: We have added a menu of indicators and outcome measures to redesignated Application Requirement (f). Applicants may choose from this menu, or propose alternative indicators and outcome measures if they describe why those are more appropriate for their proposed projects. Applicants may propose additional measures and indicators that are not included among the options we identify, so long as they select at least one indicator and one outcome measure in the domain of education and at least one indicator and outcome measure in the domain of employment. Applicants may also propose additional measures and indicators outside of the education and employment domains such as well-being, including health, housing, recidivism, or other outcomes and are encouraged to do so where such outcomes are central to the proposed pilot.

Comments: None.

Discussion: One of the outcome measures we proposed in Application Requirement (f) was “community college completion.” Upon further review, we determined that it would be more appropriate and inclusive to refer more generally to college completion so that pilots would have the option of measuring and setting targets for the completion of degree and certificate programs offered by four-year colleges and universities, as well as those offered by community colleges.

Changes: We substituted the phrase “college completion” for “community college completion” in Application Requirement (f).

Comment: None.

Discussion: Upon further review, we noted that the text of Application Requirement (b) did not conform to the headings in Table 1 in two instances. First, the text of Application Requirement (b) instructed applicants to include the “range of ages of youth” while the heading for column 2 in Table 1 was “age range.” Second, the text of Application Requirement (b) instructed applicants to provide the “number of youth who will be served annually,” while the header for column 3 in Table 1 was “Estimated Number of Youth Served Over the Course of the Pilot.”

Changes: We revised the text of Application Requirement (b) so that it conforms to the headings of Table 1. We have substituted “age range” for “range of ages of youth” and “estimated number of youth served over the course of the pilot” for “number of youth who will be served annually.”

Comments: None.

Discussion: In the NPP, the note accompanying Table 2 in Application Requirement (c)(1) (Federal requests for flexibility, including waivers) instructed applicants to indicate in the column for the name of grantee whether the grantee was a State, local, or tribal government. Upon further review, we determined that this note also should include a reference to non-governmental entities, if applicable. This change is appropriate because, while only State, local, or tribal governments may submit a P3 application, they may request waivers on behalf of non-governmental entities that are their partners in order to implement their pilots.

Changes: We have added “or non-governmental entity” to the note accompanying Table 2 in Application Requirement (c)(1).

Comments: None.

Discussion: Upon further review, we noted that Table 2 in Application Requirement (c)(1) (Federal requests for flexibility, including waivers) was titled “Requested Waivers.” However, the requirement refers more generally to requests for flexibility, including waivers.

Changes: As a result, we have retitled Table 2 “Requested Flexibility.”

Comments: None.

Discussion: Upon further review, we concluded that, for clarity, Table 5 in Application Requirement (g) should include a column that requests the name of the grantee that is the recipient of the specified funds and that the reference to “applicant or its partners” should be changed to “the grantee.” These changes are important because the recipient of funds may not always be the applicant.

Changes: We have added to Table 5 in Application Requirement (g) a column that requests the name of the grantee that is the recipient of the specified funds and changed to reference to “applicants and its partners” to the “grantee.”

Comment: None.

Discussion: Upon further review, we noted that the text of Application Requirement (g)(1)(A) was incomplete because it did not specify the content of the fifth and sixth columns in the accompanying Table 5.

Changes: We revised Application Requirement (g)(1)(A) to specify the information to be provided in these columns: the Federal fiscal year of the award (column 5) and whether the grant has already been awarded (column 6).

Comment: One commenter recommended that we impose a program requirement that would mandate that intake personnel or case workers involved in a P3 project seek to obtain a youth's school records, if feasible, to avoid spending unnecessary time and resources on assessing the youth's academic skills.

Discussion: We agree that projects should seek school records where feasible so that time and money are not wasted on unnecessary reassessments of youth's skills. However, we decline to mandate this practice because it would be inappropriately prescriptive for an initiative like P3 that seeks to increase State, local, and tribal flexibility to innovate. We also wish to avoid establishing detailed procedural requirements or other mandates for how projects must be carried out so that we can focus on assessing P3 projects on the basis of the outcomes they achieve for youth, rather than how they deliver services to youth.

Changes: None.

Comment: One commenter recommended that we require P3 projects to assess the career interests, aptitudes and goals of participants, as well as compel projects to offer work-based career assessment strategies as one option for such assessments.

Discussion: We agree that assessing the career interests, aptitudes, and goals of youth is worthwhile, but we decline to impose the mandate recommended by the commenter so that we can preserve the freedom of State, local, and tribal governments to innovate. We do not believe it is appropriate to compel applicants to provide particular types of services and interventions.

Changes: None.

Definitions:

Comment: Two commenters expressed support for our proposal to base the definition of English learner on the definition of “English language learner” found in section 203 of the Workforce Innovation and Opportunity Act (WIOA) (29 U.S.C. 3272(7)). However, one of these commenters noted that the WIOA section 203 definition requires English language learners to be “eligible individuals,” which is defined by WIOA section 203(4) as individuals who are at least 16 years of age. This commenter urged us to affirm that the P3 definition of “English learner” includes youth as young as age 14.

Discussion: We acknowledge the support for the definition. The second commenter is correct that an individual must be 16 years of age to meet the WIOA section 203 definition of “English language learner.” For this reason, we did not cross-reference the WIOA section 203 definition in our proposed definition of the term “English learner” for P3, choosing instead to adapt the definition so that it would be suitable for P3 and include youth as young as age 14.

Changes: None.

Comment: Two commenters expressed support for our proposal to define the term “homeless youth” using the definition found in section 725(2) of the McKinney-Vento Education for Homeless Children and Youth Act of 2001 (42 U.S.C. 11434a(2)).

Discussion: We acknowledge the commenters' support.

Changes: None.

Comment: None.

Discussion: Upon further review, we determined that the proposed definition of “braided funding” required revision. The definition we had originally proposed had indicated that braiding funds does not require a waiver. While this is true, it is possible that a waiver might facilitate a pilot's ability to braid funds, such as by aligning the eligibility requirements of two programs.

Changes: We have amended the definition of “braided funding” to clarify that waivers may be used to support more effective or efficient braiding of funds.

Comment: None.

Discussion: Upon further review, we determined that the proposed definition of “waiver” required revision. The definition we had originally proposed had indicated that a waiver provides relief from specific statutory, regulatory, or administrative requirements. In some instances, however, a waiver might waive a specific requirement in part, rather than eliminate it altogether. For example, a waiver could enable a pilot to increase the eligibility requirements of a program from 18 to 21 years old.

Changes: We have amended the definition of waiver to indicate that a waiver may waive specific statutory, regulatory, or administrative requirements in whole or in part.

Comment: None.

Discussion: Upon further review, we identified a typographical error in the first sentence of the proposed definition of “evidence-based intervention.” The first sentence of this definition used the term “quasi-experimental studies.” The correct term, which is defined in the Education Department General Administrative Regulations (34 CFR 77.1) is “quasi-experimental design studies.”

Changes: We have changed the reference to “quasi-experimental studies” in the first sentence of the definition to “quasi-experimental design studies.”

Comment: None.

Discussion: The NPP included a proposed definition for the term “evidence-based intervention,” which was used in proposed Selection Criterion (c)(2). Since the publication of the NPP, the Every Student Succeeds Act (ESSA) (Pub. Law 114-95) was enacted into law. This Act, which authorizes most of the Department's elementary and secondary education programs, uses extensively the terms “evidence-based” and “evidence-based intervention.” However, ESSA defines the term “evidence-based” differently than we had proposed to define the term “evidence-based intervention” in the NPP.

Change: To prevent confusion with the ESSA definition of the term “evidence-based,” we have changed the term “evidence-based intervention” in the Definitions section and in Selection Criterion (c)(2) to “intervention based on evidence.”

Selection Criteria:

Comment: Two commenters expressed concern that disaggregated outcome data are not readily available for some ELs and youth who are immigrants or refugees, including, particularly, outcome data by nativity and ethnicity for Asian Americans and Pacific Islanders (AAPIs). The commenters were concerned that applications that proposed to serve these populations would not score well under Selection Criterion (a), Need for Project, as a result.

Discussion: We acknowledge the commenters' concerns about the limited availability of data on AAPIs that is disaggregated by nativity and ethnicity, but we note that, in part due to the efforts of the White House Initiative on Asian Americans and Pacific Islanders, such data are becoming increasingly available. For example, the U.S. Department of Labor's Bureau of Labor Statistics now disaggregates Current Population Survey estimates on labor force participation, employment, and unemployment for seven Asian groups. However, we recognize that there may still be instances where disaggregated data are difficult to obtain.

Changes: We have added a sentence to the note accompanying Selection Criterion (a) clarifying that applicants may also refer to disaggregated data available through research, studies, or other sources that describe similarly situated populations as the one the applicant is targeting with its pilot.

Comment: None.

Discussion: Upon further review, we determined that it was necessary to clarify that Selection Criterion (a) does not require applicants to submit the needs assessment to which the criterion refers. Applicants need only present data from a needs assessment that was conducted or updated in the past three years; the needs assessment itself does not need to be provided.

Changes: We have added a note to accompany Selection Criterion (a) that indicates that applicants are not required to submit the needs assessment but that they should identify when the needs assessment was conducted or updated.

Comment: None.

Discussion: Upon further review, we determined that it was necessary to replace the term “a waiver” in Selection Criterion (b)(1) with the broader term “flexibility” in order to make the text of the criterion consistent with its title.

Changes: We have replaced the word “waiver” in Selection Criterion (b)(1) with the word “flexibility.”

Comment: None.

Discussion: Upon further review, we determined that, for clarity, it was necessary to include in Selection Criterion (b)(1) and (2) cross-references to Table 2 because this is where an applicant identifies the requirements for which it is seeking flexibility.

Changes: We have revised Selection Criterion (b) (1) and (2) to include cross-references to Table 2.

Comment: None.

Discussion: As proposed in the NPP, Application Requirement (b) would have required that the needs assessment used to identify the needs of the target population to have been conducted or updated within the past three years. As discussed above, in response to public comment, we have removed some of the requirements from proposed Application Requirement (b) because much of the information it sought also must be provided to respond to Selection Criterion (a), Need for Project.

Changes: Because it is important that applicants provide recent data on the needs of the population(s) they propose to serve, we have revised Selection Criterion (a) to specify that the data provided in response to this selection criterion must be from a needs assessment conducted or updated within the past three years.

Comments: None.

Discussion: In the NPP, Selection Criterion (c) referred to the “Statement of Need section” and “Need for Flexibility section.” Upon further review, we determined that it was not clear that these were cross-references to the applicant's responses to Selection Criteria (a) and (b), respectively.

Changes: We have revised Selection Criterion (c) to clarify that it refers to the applicant's responses to Selection Criteria (a) and (b).

Comment: None.

Discussion: Upon further review, we determined that it was necessary to revise Selection Criterion (c)(2) to clarify its meaning. As we had originally proposed it, this subcriterion was confusing with regard to the meaning of “evidence” and “base.” Further, we determined that the subcriterion's reference to “relevant evidence” was unclear.

Changes: We have revised the subcriterion to eliminate the use of the word “base” as both a noun and a verb so that it now assesses “[t]he strength of the evidence supporting the pilot design, and whether the applicant proposes the effective use of interventions based on evidence and evidence-informed interventions (as defined in this notice).” We also revised the subcriterion to clarify that evidence is relevant if it informed the applicant's design.

Comment: None.

Discussion: Upon further review, we determined that, for clarity, it was necessary to revise Selection Criterion (f) (2) and (3) to indicate that the information evaluated by these two subcriteria appears in Table 4.

Changes: We have revised Selection Criterion (f) (2) and (3) to include cross-references to Table 4.

Comment: None.

Discussion: Upon further review, we found that Selection Criterion (g), Budget and Budget Narrative, may be confusing to applicants because the budget to which it refers is not clearly specified. The criterion could refer to the start-up grant funds requested by the applicant, the Federal funds that would be blended or braided in the proposed pilot, the non-Federal funds contributed by the applicant, or all of these sources of funds.

Changes: We have revised Selection Criterion (g) to indicate that its scope includes all of the funds that will be used by a pilot, including the start-up grant funds, blended and braided funds, and any non-Federal resources contributed by the applicant.

Final Priorities

Priority 1—Improving Outcomes for Disconnected Youth.

To meet this priority, an applicant must propose a pilot that is designed to improve outcomes for disconnected youth.

Priority 2—Improving Outcomes for Disconnected Youth in Rural Communities.

To meet this priority, an applicant must propose a pilot that is designed to improve outcomes for disconnected youth in one or more rural communities (as defined in this notice) only.

Priority 3—Improving Outcomes for Disconnected Youth in Tribal Communities.

To meet this priority, an applicant must (1) propose a pilot that is designed to improve outcomes for disconnected youth who are members of one or more State- or federally-recognized Indian tribal communities; and (2) represent a partnership that includes one or more State- or federally-recognized Indian tribes.

Priority 4—Improving Outcomes for Youth Who Are Unemployed and Out of School.

To meet this priority, an applicant must propose a pilot that—

(1) will serve disconnected youth who are neither employed nor enrolled in education and who face significant barriers to accessing education and employment; and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 5—Improving Outcomes for Youth Who are English Learners.

To meet this priority, an applicant must propose a pilot that—

(1) will serve disconnected youth who are English learners (as defined in this notice); and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 6—Improving Outcomes for Youth with a Disability.

To meet this priority, an applicant must propose a pilot that—

(1) will serve disconnected youth who are individuals with a disability (as defined in this notice); and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 7—Improving Outcomes for Homeless Youth.

To meet this priority, an applicant must propose a pilot that—

(1) will serve disconnected youth who are homeless youth (as defined in this notice); and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 8—Improving Outcomes for Youth in Foster Care.

To meet this priority, an applicant must propose a pilot that—-

(1) will serve disconnected youth who are or have ever been in foster care; and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 9—Improving Outcomes for Youth Involved in the Justice System.

To meet this priority, an applicant must propose a pilot that—

(1) will serve disconnected youth who are involved in the justice system; and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 10—Improving Outcomes for Youth Who are Immigrants or Refugees.

To meet this priority, an applicant must propose a pilot that—

(1) will serve disconnected youth who are immigrants or refugees; and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 11—Improving Outcomes for Youth Who are Pregnant or Parenting.

To meet this priority, an applicant must propose a pilot that—-

(1) will serve disconnected youth who are pregnant or parenting; and

(2) is likely to result in significantly better educational or employment outcomes for such youth.

Priority 12—Work-Based Learning Opportunities.

To meet this priority, an applicant must propose a pilot that will provide all of the disconnected youth it proposes to serve with paid work-based learning opportunities, such as opportunities during the summer, which are integrated with academic and technical instruction.

Priority 13—Site-Specific Evaluation.

To meet this priority, an applicant must propose to conduct an independent evaluation of the impacts on disconnected youth of its overall program or specific components of its program that is a randomized controlled trial or a quasi-experimental design study. The extent to which an applicant meets this priority will be based on the clarity and feasibility of the applicant's proposed evaluation design, the appropriateness of the design to best capture key pilot outcomes, the prospective contribution of the evaluation to the knowledge base about serving disconnected youth (including the rigor of the design and the validity and generalizability of the findings), and the applicant's demonstrated expertise in planning and conducting a randomized controlled trial or quasi-experimental design study.

In order to meet this priority, an applicant also must include the following two documents as separate attachments to its application:

1. A Summary Evaluation Plan that describes how the pilot or a component of the pilot (such as a discrete service-delivery strategy) will be rigorously evaluated. The evaluation plan may not exceed eight pages. The plan must include the following:

• A brief description of the research question(s) proposed for study and an explanation of its/their relevance, including how the proposed evaluation will build on the research evidence base for the project as described in the application and how the evaluation findings will be used to improve program implementation;
• A description of the randomized controlled trial or quasi-experimental design study methodology, including the key outcome measures, the process for forming a comparison or control group, a justification for the target sample size and strategy for achieving it, and the approach to data collection (and sources) that minimizes both cost and potential attrition;
• A proposed evaluation timeline, including dates for submission of required interim and final reports;
• A description of how, to the extent feasible and consistent with applicable Federal, State, local, and tribal privacy requirements, evaluation data will be made available to other, third‐party researchers after the project ends; and
• A plan for selecting and procuring the services of a qualified independent evaluator (as defined in this notice) prior to enrolling participants (or a description of how one was selected if agreements have already been reached). The applicant must describe how it will ensure that the qualified independent evaluator has the capacity and expertise to conduct the evaluation, including estimating the effort for the qualified independent evaluator. This estimate must include the time, expertise, and analysis needed to successfully complete the proposed evaluation.

2. A supplementary Evaluation Budget Narrative, which is separate from the overall application budget narrative and provides a description of the costs associated with funding the proposed program evaluation component, and an explanation of its funding source—i.e., blended funding, start-up funding, State, local, or tribal government funding, or other funding (such as philanthropic). The budget must include a breakout of costs by evaluation activity (such as data collection and participant follow-up), and the applicant must describe a strategy for refining the budget after the services of an evaluator have been procured. The applicant must include travel costs for the qualified independent evaluator to attend at least one in-person conference in Washington, DC during the period of evaluation. All costs included in this supplementary budget narrative must be reasonable and appropriate to the project timeline and deliverables.

The Agencies will review the Summary Evaluation Plans and Evaluation Budget Narratives and provide feedback to applicants that are determined to have met the priority and that are selected as pilots. After award, these pilots must submit to the lead Federal agency a detailed evaluation plan of no more than 30 pages that relies heavily on the expertise of a qualified independent evaluator. The detailed evaluation plan must address the Agencies' feedback and expand on the Summary Evaluation Plan.

[Approved by the Office of Management and Budget under control number 1830-0575]

Types of Priorities

When inviting applications for a competition using one or more priorities, we designate the type of each priority as absolute, competitive preference, or invitational through a notice in the Federal Register. The effect of each type of priority follows:

Absolute priority: Under an absolute priority, we consider only applications that meet the priority (34 CFR 75.105(c)(3)).

Competitive preference priority: Under a competitive preference priority, we give competitive preference to an application by (1) awarding additional points, depending on the extent to which the application meets the priority (34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets the priority over an application of comparable merit that does not meet the priority (34 CFR 75.105(c)(2)(ii)).

Invitational priority: Under an invitational priority we are particularly interested in applications that meet the priority. However, we do not give an application that meets the priority a preference over other applications (34 CFR 75.105(c)(1)).

Requirements

Application Requirements

The Assistant Secretary announces the following application requirements for this program. We may apply one or more of these requirements in any year in which this program is in effect.

(a) Executive Summary. The applicant must provide an executive summary that briefly describes the proposed pilot, the flexibilities being sought, and the interventions or systems changes that would be implemented by the applicant and its partners to improve outcomes for disconnected youth.

(b) Target Population. The applicant must complete Table 1, specifying the target population(s) for the pilot, including the age range of youth who will be served and the estimated number of youth who will be served over the course of the pilot.

Table 1—Target Population

Target population	Age range	Estimated number of youth served over the course of the pilot

(c) Flexibility, including waivers:

1. Federal requests for flexibility, including waivers. For each program to be included in a pilot, the applicant must complete Table 2, Requested Flexibility. The applicant must identify two or more discretionary Federal programs that will be included in the pilot, at least one of which must be administered (in whole or in part) by a State, local, or tribal government.^[9] In table 2, the applicant must identify one or more program requirements that would inhibit implementation of the pilot and request that the requirement(s) be waived in whole or in part. Examples of potential waiver requests and other requests for flexibility include, but are not limited to: blending of funds and changes to align eligibility requirements, allowable uses of funds, and performance reporting.

Table 2—Requested Flexibility

Program name	Federal agency	Program requirements to be waived in whole or in part	Statutory or regulatory citation	Name of program grantee	Blending funds? (Yes/No)

Note:

Please note in “Name of Program Grantee” if the grantee is a State, local, or tribal government, or non-governmental entity.

2. Non-Federal flexibility, including waivers. The applicant must provide written assurance that:

A. The State, local, or tribal government(s) with authority to grant any needed non-Federal flexibility, including waivers, has approved or will approve such flexibility within 60 days of an applicant's designation as a pilot finalist; ^[10] or

B. Non-Federal flexibility, including waivers, is not needed in order to successfully implement the pilot.

(d) Logic Model. The applicant must provide a graphic depiction (not longer than one page) of the pilot's logic model that illustrates the underlying theory of how the pilot's strategy will produce intended outcomes.

(e) Partnership Capacity and Management. The applicant must—

1. Identify the proposed partners, including any and all State, local, and tribal entities and non-governmental organizations that would be involved in implementation of the pilot, and describe their roles in the pilot's implementation using Table 3. Partnerships that cross programs and funding sources but are under the jurisdiction of a single agency or entity must identify the different sub-organizational units involved.

2. Provide a memorandum of understanding or letter of commitment signed by the executive leader or other accountable senior representative of each partner that describes each proposed partner's commitment, including its contribution of financial or in-kind resources (if any).

Table 3—Pilot Partners

Partner	Type of organization (state agency, local agency, community-based organization, business)	Description of partner's role in the pilot

Note:

Any grantees mentioned in Table 2 that are not the lead applicant must be included in Table 3.

(f) Data and Performance Management Capacity. The applicant must propose outcome measures and interim indicators to gauge pilot performance using Table 4. At least one outcome measure must be in the domain of education, and at least one outcome measure must be in the domain of employment. Applicants may specify additional employment and education outcome measures, as well as outcome measures in other domains of well-being, such as criminal justice, physical and mental health, and housing. Regardless of the outcome domain, applicants must identify at least one interim indicator for each proposed outcome measure. Applicants may apply one interim indicator to multiple outcome measures, if appropriate.

Examples of outcome measures and interim indicators follow. Applicants may choose from this menu or may propose alternative indicators and outcome measures if they describe why their alternatives are more appropriate for their proposed projects.

Education Domain

Outcome measure	Interim indicator
High school diploma or equivalency attainment	• High school enrollment. • Reduction in chronic absenteeism.
	• Grade promotion.
	• Performance on standardized assessments.
	• Grade Point Average.
	• Credit accumulation.
College completion	• Enrollment.
	• Course attendance.
	• Credit accumulation.
	• Retention.

Employment Domain

Outcome measure	Interim indicator
Sustained Employment	• Unsubsidized employment at time periods after exit from the program.
	• Median earnings at time periods after exit from the program.

The specific outcome measures and interim indicators the applicant uses should be grounded in its logic model, and informed by applicable program results or research, as appropriate. Applicants must also indicate the source of the data, the proposed frequency of collection, and the methodology used to collect the data.

Table 4—Outcome Measures and Interim Indicators

Domain	Outcome measure	Interim indicator(s)
Education:
	Data Source:	Data Source:
	Frequency of Collection:	Frequency of Collection:
	Methodology:	Methodology:
Employment:
	Data Source:	Data Source:
	Frequency of Collection:	Frequency of Collection:
	Methodology:	Methodology:
Other:
	Data Source:	Data Source:
	Frequency of Collection:	Frequency of Collection:
	Methodology:	Methodology:

(g) Budget and Budget Narrative.

1. The applicant must complete Table 5 to provide the following budget information:

A. For each Federal program, the grantee, the amount of funds to be blended or braided (as defined in this notice), the percentage of total program funding received by the grantee that the amount to be blended or braided represents, the Federal fiscal year of the award, and whether the grant has already been awarded; and

B. The total amount of funds from all Federal programs that would be blended or braided under the pilot.

Table 5—Federal Funds

Program name	Grantee	Amount of funds to be blended	Blended funds as a percentage of grantee's total award	Federal fiscal year of award	Grant already awarded? (Y/N)
TOTAL BLENDED
Program name	Grantee	Amount of funds to be braided	Braided funds as a percentage of grantee's total award	Federal fiscal year of award	Grant already awarded? (Y/N)
TOTAL BRAIDED

Note:

Applicants may propose to expand the number of Federal programs supporting pilot activities using future funding beyond FY 2016, which may be included in pilots if Congress extends the P3 authority.

[Approved by the Office of Management and Budget under control number 1830-0575]

Program Requirements

The Assistant Secretary announces the following program requirements for this program. We may apply one or more of these requirements in any year in which this program is in effect.

(a) National evaluation. In addition to any site-specific evaluations that pilots may undertake, the Agencies may initiate a national P3 evaluation of the pilots selected in Round 2, as well as those selected in subsequent rounds.^[11] Each P3 pilot must participate fully in any federally sponsored P3 evaluation activity, including the national evaluation of P3, which will consist of the analysis of participant characteristics and outcomes, an implementation analysis at all sites, and rigorous impact evaluations of promising interventions in selected sites. The applicant must acknowledge in writing its understanding of these requirements by submitting the form provided in Appendix A, “Evaluation Commitment Form,” as an attachment to its application.

[Approved by the Office of Management and Budget under control number 1830-0575]

(b) Community of practice. All P3 pilots must participate in a community of practice (as defined in this notice) that includes an annual in-person meeting of pilot sites (paid with grant funding that must be reflected in the pilot budget submitted) and virtual peer-to-peer learning activities. This commitment involves each pilot site working with the lead Federal agency on a plan for supporting its technical assistance needs, which can include learning activities supported by foundations or other non-Federal organizations as well as activities financed with Federal funds for the pilot.

(c) Consent. P3 pilots must secure necessary consent from parents, guardians, students, or youth program participants to access data for their pilots and any evaluations, in accordance with applicable Federal, State, local, and tribal laws. Applicants must explain how they propose to ensure compliance with Federal, State, local, and tribal privacy laws and regulations as pilot partners share data to support effective coordination of services and link data to track outcome measures and interim indicators at the individual level to perform, where applicable, a low-cost, high-quality evaluation.^[12]

(d) Performance agreement. Each P3 pilot, along with other non-Federal government entities involved in the partnership, must enter into a performance agreement that will include, at a minimum, the following (as required by section 526(c)(2) of Division H of the 2014 Appropriations Act):

1. The length of the agreement;

2. The Federal programs and federally-funded services that are involved in the pilot;

3. The Federal discretionary funds that are being used in the pilot;

4. The non‐Federal funds that are involved in the pilot, by source (which may include private funds as well as governmental funds) and by amount;

5. The State, local, or tribal programs that are involved in the pilot;

6. The populations to be served by the pilot;

7. The cost‐effective Federal oversight procedures that will be used for the purpose of maintaining the necessary level of accountability for the use of the Federal discretionary funds;

8. The cost‐effective State, local, or tribal oversight procedures that will be used for the purpose of maintaining the necessary level of accountability for the use of the Federal discretionary funds;

9. The outcome (or outcomes) that the pilot is designed to achieve;

10. The appropriate, reliable, and objective outcome‐measurement methodology that will be used to determine whether the pilot is achieving, and has achieved, specified outcomes;

11. The statutory, regulatory, or administrative requirements related to Federal mandatory programs that are barriers to achieving improved outcomes of the pilot; and

12. Criteria for determining when a pilot is not achieving the specified outcomes that it is designed to achieve and subsequent steps, including:

i. The consequences that will result; and

ii. The corrective actions that will be taken in order to increase the likelihood that the pilot will achieve such specified outcomes.

Definitions

The Assistant Secretary announces the following definitions for this program. We may apply one or more of these definitions in any year in which this program is in effect.

Blended funding is a funding and resource allocation strategy that uses multiple existing funding streams to support a single initiative or strategy. Blended funding merges two or more funding streams, or portions of multiple funding streams, to produce greater efficiency and/or effectiveness. Funds from each individual stream lose their award-specific identity, and the blended funds together become subject to a single set of reporting and other requirements, consistent with the underlying purposes of the programs for which the funds were appropriated.

Braided funding is a funding and resource allocation strategy in which entities use existing funding streams to support unified initiatives in as flexible and integrated a manner as possible while still tracking and maintaining separate accountability for each funding stream. One or more entities may coordinate several funding sources, but each individual funding stream maintains its award-specific identity. Whereas blending funds typically requires one or more waivers of associated program requirements, braiding does not. However, waivers may be used to support more effective or efficient braiding of funds.

Community of practice means a group of pilots that agrees to interact regularly to solve persistent problems or improve practice in an area that is important to them and the success of their projects.

English learner means an individual who has limited ability in reading, writing, speaking, or comprehending the English language, and—

(A) Whose native language is a language other than English; or

(B) Who lives in a family or community environment where a language other than English is the dominant language.

Evidence-informed interventions bring together the best available research, professional expertise, and input from youth and families to identify and deliver services that have promise to achieve positive outcomes for youth, families, and communities.

Homeless youth has the same meaning as “homeless children and youths” in section 725(2) of the McKinney-Vento Education for Homeless Children and Youth Act of 2001 (42 U.S.C. 11434a(2)).

Individual with a disability means an individual with any disability as defined in section 3 of the Americans with Disabilities Act of 1990 (42 U.S.C. 12102).

An interim indicator is a marker of achievement that demonstrates progress toward an outcome and is measured at least annually.

Interventions based on evidence are approaches to prevention or treatment that are validated by documented scientific evidence from randomized controlled trials, or quasi-experimental design studies or correlational studies, and that show positive effects (for randomized controlled trials and quasi-experimental design studies) or favorable associations (for correlational studies) on the primary targeted outcomes for populations or settings similar to those of the proposed pilot. The best evidence to support an applicant's proposed reform(s) and target population will be based on one or more randomized controlled trials. The next best evidence will be studies using a quasi-experimental design. Correlational analysis may also be used as evidence to support an applicant's proposed reforms.

Outcomes are the intended results of a program, or intervention. They are what applicants expect their projects to achieve. An outcome can be measured at the participant level (for example, changes in employment retention or earnings of disconnected youth) or at the system level (for example, improved efficiency in program operations or administration).

A qualified independent evaluator is an individual who coordinates with the grantee and the lead Federal agency for the pilot, but works independently on the evaluation and has the capacity to carry out the evaluation, including, but not limited to: Prior experience conducting evaluations of similar design (for example, for randomized controlled trials, the evaluator will have successfully conducted a randomized controlled trial in the past); positive past performance on evaluations of a similar design, as evidenced by past performance reviews submitted from past clients directly to the awardee; lead staff with prior experience carrying out a similar evaluation; lead staff with minimum credential (such as a Ph.D. plus three years of experience conducting evaluations of a similar nature, or a Master's degree plus seven years of experience conducting evaluations of a similar nature); and adequate staff time to work on the evaluation.

A rural community is a community that is served only by one or more local educational agencies (LEAs) that are currently eligible under the Department of Education's Small, Rural School Achievement (SRSA) program or the Rural and Low-Income School (RLIS) program authorized under the Elementary and Secondary Education Act of 1965 (ESEA), as amended, or includes only schools designated by the National Center for Education Statistics (NCES) with a locale code of 42 or 43.

A waiver provides flexibility in the form of relief, in whole or in part, from specific statutory, regulatory, or administrative requirements that have hindered the ability of a State, locality, or tribe to organize its programs and systems or provide services in ways that best meet the needs of its target populations. Under P3, waivers provide flexibility in exchange for a pilot's commitment to improve programmatic outcomes for disconnected youth consistent with underlying statutory authorities and purposes.

Selection Criteria

The Assistant Secretary announces the following selection criteria for evaluating an application under this program. We may apply one or more of these criteria in any year in which this program is in effect. In the notice inviting applications, the application package, or both we will announce the maximum possible points assigned to each criterion.

(a) Need for Project. In determining the need for the proposed project, we will consider the magnitude of the need of the target population, as evidenced by the applicant's analysis of data, including data from a comprehensive needs assessment conducted or updated in the past three years, using representative data on youth in the jurisdiction(s) proposing the pilot, that demonstrates how the target population lags behind other groups in achieving positive outcomes and the specific risk factors for this population.

Note:

Applicants are encouraged to disaggregate these data according to relevant demographic factors such as race, ethnicity, gender, age, disability status, involvement in systems such as foster care or juvenile justice, status as pregnant or parenting, and other key factors selected by the applicant. If disaggregated data specific to the local population are not available, applicants may refer to disaggregated data available through research, studies, or other sources that describe similarly situated populations as the one the applicant is targeting with its pilot.

Note:

Applicants do not need to include a copy of the needs assessment but should identify when it was conducted or updated.

(b) Need for Requested Flexibility, Including Blending of Funds and Other Waivers. In determining the need for the requested flexibility, including blending of funds and other waivers, we will consider:

1. The strength and clarity of the applicant's justification that each of the specified Federal requirements identified in Table 2 for which the applicant is seeking flexibility hinders implementation of the proposed pilot; and

2. The strength and quality of the applicant's justification of how each request for flexibility identified in Table 2 (i.e., blending funds and waivers) will increase efficiency or access to services and produce significantly better outcomes for the target population(s).

(c) Project Design. In determining the strength of the project design, we will consider:

1. The strength and logic of the proposed project design in addressing the gaps and the disparities identified in the response to Selection Criterion (a) (Need for Project) and the barriers identified in the response to Selection Criterion (b) (Need for Requested Flexibility, Including Blending of Funds and Other Waivers). This includes the clarity of the applicant's plan and how the plan differs from current practices. Scoring will account for the strength of both the applicant's narrative and the logic model;

Note:

The applicant's narrative should describe how the proposed project will use and coordinate resources, including building on participation in any complementary Federal initiatives or efforts.

2. The strength of the evidence supporting the pilot design and whether the applicant proposes the effective use of intervention based on evidence and evidence-informed interventions (as defined in this notice) as documented by citations to the relevant evidence that informed the applicant's design;

Note:

Applicants should cite the studies on interventions and system reforms that informed their pilot design and explain the relevance of the cited evidence to the proposed project in terms of subject matter and evaluation evidence. Applicants proposing reforms on which there are not yet evaluations (such as innovations that have not been formally tested or tested only on a small scale) should document how evidence or practice knowledge informed the proposed pilot design.

3. The strength of the applicant's evidence that the project design, including any protections and safeguards that will be established, ensures that the consequences or impacts of the changes from current practices in serving youth through the proposed funding streams:

A. Will not result in denying or restricting the eligibility of individuals for services that (in whole or in part) are otherwise funded by these programs; and

B. Based on the best available information, will not otherwise adversely affect vulnerable populations that are the recipients of those services.

(d) Work Plan and Project Management. In determining the strength of the work plan and project management, we will consider the strength and completeness of the work plan and project management approach and their likelihood of achieving the objectives of the proposed project on time and within budget, based on—

1. Clearly defined and appropriate responsibilities, timelines, and milestones for accomplishing project tasks;

2. The qualifications of project personnel to ensure proper management of all project activities;

3. How any existing or anticipated barriers to implementation will be overcome.

Note:

If the program manager or other key personnel are already on staff, the applicant should provide this person's resume or curriculum vitae.

Note:

Evaluation activities may be included in the timelines provided as part of the work plan.

(e) Partnership Capacity. In determining the strength and capacity of the proposed pilot partnership, we will consider the following factors—

1. How well the applicant demonstrates that it has an effective governance structure in which partners that are necessary to implement the pilot successfully are represented and have the necessary authority, resources, expertise, and incentives to achieve the pilot's goals and resolve unforeseen issues, including by demonstrating the extent to which, and how, participating partners have successfully collaborated to improve outcomes for disconnected youth in the past;

2. How well the applicant demonstrates that its proposal was designed with substantive input from all relevant stakeholders, including disconnected youth and other community partners.

Note:

Where the project design includes job training strategies, the extent of employer input and engagement in the identification of skills and competencies needed by employers, the development of the curriculum, and the offering of work-based learning opportunities, including pre-apprenticeship and registered apprenticeship, will be considered.

(f) Data and Performance Management Capacity. In determining the strength of the applicant's data and performance management capacity, we will consider the following factors—

1. The applicant's capacity to collect, analyze, and use data for decision-making, learning, continuous improvement, and accountability, and the strength of the applicant's plan to bridge any gaps in its ability to do so. This capacity includes the extent to which the applicant and partner organizations have tracked and shared data about program participants, services, and outcomes, including the execution of data-sharing agreements that comport with Federal, State, and other privacy laws and requirements, and will continue to do so;

2. How well the proposed outcome measures, interim indicators, and measurement methodologies specified in Table 4 of the application appropriately and sufficiently gauge results achieved for the target population under the pilot; and

3. How well the data sources specified in Table 4 of the application can be appropriately accessed and used to reliably measure the proposed outcome measures and interim indicators.

(g) Budget and Budget Narrative. In determining the adequacy of the resources that will be committed to support the project, we will consider the appropriateness of expenses within the budget with regards to cost and to implementing the pilot successfully. We will consider the entirety of funds the applicant will use to support its pilot including start-up grant funds, blended and braided funds included in Table 5, and non-Federal funds, including in-kind contributions.

This notice does not preclude us from proposing additional priorities, requirements, definitions, or selection criteria, subject to meeting applicable rulemaking requirements.

Note:

This notice does not solicit applications. In any year in which we choose to use one or more of these priorities, requirements, definitions, or selection criteria, we invite applications through a notice in the Federal Register.

Executive Orders 12866 and 13563

Regulatory Impact Analysis

Under Executive Order 12866, the Secretary must determine whether this regulatory action is “significant” and, therefore, subject to the requirements of the Executive order and subject to review by the Office of Management and Budget (OMB). Section 3(f) of Executive Order 12866 defines a “significant regulatory action” as an action likely to result in a rule that may—

(1) Have an annual effect on the economy of $100 million or more, or adversely affect a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or tribal governments or communities in a material way (also referred to as an “economically significant” rule);

(2) Create serious inconsistency or otherwise interfere with an action taken or planned by another agency;

(3) Materially alter the budgetary impacts of entitlement grants, user fees, or loan programs or the rights and obligations of recipients thereof; or

(4) Raise novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles stated in the Executive order.

This final regulatory action is not a significant regulatory action subject to review by OMB under section 3(f) of Executive Order 12866.

We have also reviewed this final regulatory action under Executive Order 13563, which supplements and explicitly reaffirms the principles, structures, and definitions governing regulatory review established in Executive Order 12866. To the extent permitted by law, Executive Order 13563 requires that an agency—

(1) Propose or adopt regulations only upon a reasoned determination that their benefits justify their costs (recognizing that some benefits and costs are difficult to quantify);

(2) Tailor its regulations to impose the least burden on society, consistent with obtaining regulatory objectives and taking into account—among other things and to the extent practicable—the costs of cumulative regulations;

(3) In choosing among alternative regulatory approaches, select those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity);

(4) To the extent feasible, specify performance objectives, rather than the behavior or manner of compliance a regulated entity must adopt; and

(5) Identify and assess available alternatives to direct regulation, including economic incentives—such as user fees or marketable permits—to encourage the desired behavior, or provide information that enables the public to make choices.

Executive Order 13563 also requires an agency “to use the best available techniques to quantify anticipated present and future benefits and costs as accurately as possible.” The Office of Information and Regulatory Affairs of OMB has emphasized that these techniques may include “identifying changing future compliance costs that might result from technological innovation or anticipated behavioral changes.”

We are issuing these final priorities, requirements, definitions, and selection criteria only on a reasoned determination that their benefits justify their costs. In choosing among alternative regulatory approaches, we selected those approaches that maximize net benefits. Based on the analysis that follows, the Department believes that this regulatory action is consistent with the principles in Executive Order 13563.

We also have determined that this regulatory action does not unduly interfere with State, local, and tribal governments in the exercise of their governmental functions.

In accordance with both Executive orders, the Department has assessed the potential costs and benefits, both quantitative and qualitative, of this regulatory action. The potential costs are those resulting from statutory requirements and those we have determined as necessary for administering the Department's programs and activities.

In this regulatory impact analysis we discuss the need for regulatory action, the potential costs and benefits, net budget impacts, assumptions, limitations, and data sources, as well as regulatory alternatives we considered. The potential costs of the final priorities requirements, definitions, and selection criteria are the costs associated with preparing an application. We estimate that each applicant would spend approximately 80 hours of staff time to address the final priorities, requirements, definitions, and selection criteria, prepare the application, and obtain necessary clearances. The total number of hours for all applicants will vary based on the number of applications. Based on the number of applications the Department received in response to the November 2014 notice inviting applications, we expect to receive approximately 55 applications. The total number of hours for all expected applicants is an estimated 4,400 hours. We estimate the total cost per hour of the staff who carry out this work to be $44.66 per hour, the mean hourly compensation cost for State and local government workers in September 2015. The total estimated cost for all applicants would be $196,504.

The potential benefits of the final priorities requirements, definitions, and selection criteria are that they would promote the efficient and effective use of the P3 authority. Implementation of these priorities, requirements, definitions, and selection criteria will help the Agencies identify pilots that will: (1) Serve disconnected youth with significant needs; (2) carry out effective reforms and interventions; and (3) be managed by strong partnerships with the capacity to collect, analyze, and use data for decision-making, learning, continuous improvement, and accountability.

Paperwork Reduction Act of 1995: The Paperwork Reduction Act of 1995 (PRA) does not require you to respond to a collection of information unless it displays a valid OMB control number. We display the valid OMB control number assigned to the collections of information in this NFP at the end of the affected priorities and requirements.

Priority 13 (Site-Specific Evaluation), Application Requirements (a) through (g), and Program Requirement (a) (National evaluation) contain information collection requirements. Under PRA, the Department has submitted a copy of these sections to OMB, as well as the related Information Collection Request (ICR) (the application package), for its review and approval. In accordance with the PRA, the OMB Control number associated with these collections of information and the related ICR is OMB Control number 1830-0575. OMB approval of these collections of information and the related ICR is expected at the time of publication of the NFP.

Intergovernmental Review: This program is subject to Executive Order 12372 and the regulations in 34 CFR part 79. One of the objectives of the Executive order is to foster an intergovernmental partnership and a strengthened federalism. The Executive order relies on processes developed by State and local governments for coordination and review of proposed Federal financial assistance.

This document provides early notification of our specific plans and actions for this program.

Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., Braille, large print, audiotape, or compact disc) on request to the contact person listed under FOR FURTHER INFORMATION CONTACT.

Electronic Access to This Document: The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available via the Federal Digital System at: www.gpo.gov/​fdsys. At this site you can view this document, as well as all other documents of this Department published in the Federal Register, in text or Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at the site.

You may also access documents of the Department published in the Federal Register by using the article search feature at: www.federalregister.gov. Specifically, through the advanced search feature at this site, you can limit your search to documents published by the Department.

Johan E. Uvin,

Deputy Assistant Secretary, Delegated the Duties of Assistant Secretary for Career, Technical, and Adult Education.

Appendix A: Evaluation Commitment Form

An authorized executive of the lead applicant and all other partners, including State, local, tribal, and non-governmental organizations that would be involved in the pilot's implementation, must sign this form and submit it as an attachment to the grant application. The form is not considered in the recommended application page limit.

Commitment To Participate in Required Evaluation Activities

As the lead applicant or a partner proposing to implement a Performance Partnership Pilot through a Federal grant, I/we agree to carry out the following activities, which are considered evaluation requirements applicable to all pilots:

Facilitate Data Collection: I/we understand that the award of this grant requires me/us to facilitate the collection and/or transmission of data for evaluation and performance monitoring purposes to the lead Federal agency and/or its national evaluator in accordance with applicable Federal, State, and local, and tribal laws, including privacy laws.

The type of data that will be collected includes, but is not limited to, the following:

• Demographic information, including participants' gender, race, age, school status, and employment status;
• Information on the services that participants receive; and
• Outcome measures and interim outcome indicators, linked at the individual level, which will be used to measure the effects of the pilots.

The lead Federal agency will provide more details to grantees on the data items required for performance and evaluation after grants have been awarded.

Participate in Evaluation: I/we understand that participation and full cooperation in the national evaluation of the Performance Partnership Pilot is a condition of this grant award. I/we understand that the national evaluation will include an implementation systems analysis and, for certain sites as appropriate, may also include an impact evaluation. My/our participation will include facilitating site visits and interviews; collaborating in study procedures, including random assignment, if necessary; and transmitting data that are needed for the evaluation of participants in the study sample, including those who may be in a control group.

Participate in Random Assignment: I/we agree that if our Performance Partnership Pilot or certain activities in the Pilot is selected for an impact evaluation as part of the national evaluation, it may be necessary to select participants for admission to Performance Partnership Pilot by a random lottery, using procedures established by the evaluator.

Secure Consent: I/we agree to include a consent form for, as appropriate, parents/guardians and students/participants in the application or enrollment packet for all youth in organizations implementing the Performance Partnership Pilot consistent with any Federal, State, local, and tribal laws that apply. The parental/participant consent forms will be collected prior to the acceptance of participants into Performance Partnership Pilot and before sharing data with the evaluator for the purpose of evaluating the Performance Partnership Pilot.

SIGNATURES

Lead Applicant

Print Name

Signature

Organization

Date

Partner

Print Name

Signature

Organization

Date

Partner

Print Name

Signature

Organization

Date

Partner

Print Name

Signature

Organization

Date

Partner

Print Name

Signature

Organization

Date

Partner

Print Name

Signature

Organization

Date

[Approved by the Office of Management and Budget under control number 1830-0575]

Footnotes

1.  The Department of Housing and Urban Development was first authorized to enter into performance agreements by the 2016 Appropriations Act.

2.  The Department of Justice was first authorized to enter into performance agreements by the 2015 Appropriations Act.

3.  Fernandes, A.L. (2015). Disconnected Youth: A Look at 16 to 24 Year Olds Who Are Not Working or in School. Congressional Research Service Report No. R40535. Retrieved from http://www.fas.org/​sgp/​crs/​misc/​R40535.pdf.

4.  See, for example, Juvenile Justice Students Face Barriers to High School Graduation and Job Training (2010). Report No. 10-55. Tallahassee, FL: Office of Program Policy Analysis and Government Accountability, the Florida Legislature, Retrieved from: www.oppaga.state.fl.us/​MonitorDocs/​Reports/​pdf/​1055rpt.pdf.

5.  See, for example, Pager, D.P. and Western, B. (2009). Investigating Prisoner Reentry: The Impact of Conviction Status on the Employment Prospects of Young Men: Final Report to the National Institute of Justice. Document No.: 228584. Retrieved from: www.ncjrs.gov/​pdffiles1/​nij/​grants/​228584.pdf.

6.  Hoffman, S.D. (2008). Updated Estimates of the Consequences of Teen Childbearing for Mothers. In: Hoffman, S.D., and Maynard, R.A., eds. Kids Having Kids: Economic and Social Consequences of Teen Pregnancy. Washington, DC: Urban Institute Press: 74-92.

7.  Fletcher, J.M. and Wolfe, B.L. (2012). The effects of teenage fatherhood on young adult outcomes. Economic Inquiry, 50 (1), 182-201.

8.  See 29 CFR part 570—Child Labor Regulations, Orders, and Statements of Interpretation.

9.  Local governments that are requesting waivers of requirements in State-administered programs are strongly encouraged to consult with the State agencies that administer the programs in preparing their applications.

10.  This includes, for example, for local governments, instances in which a waiver must be agreed upon by a State. It also includes instances in which waivers may only be requested by the State on the local government's behalf, such as waivers of the performance accountability requirements for local areas established in Title I of the Workforce Innovation and Opportunity Act.

11.  The initiation of any federally sponsored national P3 evaluation is dependent upon the availability of sufficient funds and resources.

12.  To the extent feasible and consistent with applicable privacy requirements, grantees must also ensure the data from their evaluations are made available to third‐party researchers.