Promoting Readiness of Minors in Supplemental Security Income (PROMISE): ASPIRE Process Analysis Report

Publisher: Washington, DC: Mathematica Policy Research
Dec 11, 2018
Authors
Mary Anne Anderson, Gina Livermore, AnnaMaria McCutcheon, Todd Honeycutt, Karen Katz, Joseph Mastrianni, and Jacqueline Kauff

Key Findings:

  • The Utah State Office of Rehabilitation (USOR) led ASPIRE. Each of the six consortium state had (1) a “lead agency” with which USOR contracted to implement the program statewide and (2) its own ASPIRE site coordinator who managed all aspects of program service delivery in that state. The lead agencies each had agreements with other agencies in their respective states to provide guidance and support to the program at the state level. ASPIRE intended to use case management to connect youth and their families to four ASPIRE core interventions, which were typically provided by subcontractors located in each state that had provided the same or similar services before participating in ASPIRE: (1) benefits counseling, (2) financial education, (3) training and information for parents and guardians, and (4) self-determination training. Additionally, case managers were responsible for connecting youth to career exploration activities and work-based learning experiences, as well as educational services.
  • ASPIRE enrolled 2,051 youth in the evaluation of the program, 1,033 of whom were assigned to the treatment group. Three years into program operations, ASPIRE had engaged 86 percent of treatment group youth as participants in the program, but was lagging in its goal to deliver intensive case management to them. By design, case managers were supposed to meet with participating families in person for at least 30 minutes once per month. In practice, case managers met face to face with families in just under half of all months between intake and October 2017, on average. Most case management contacts were under 20 minutes in duration and occurred by telephone. The program had met one of its two goals for providing career exploration and work opportunities to youth; ASPIRE anticipated that least 30 percent of youth age 16 and older would have a paid work experience by the third year of program operations (31 percent actually did) and that 95 percent of youth would engage in career exploration activities during each year of enrollment (51 percent actually did). About one-third of participating families had received parent training and information services. About one-quarter of participating families had received financial education. The program also intended to deliver self-determination training to 95 percent of participating youth within their first year of enrollment and benefits counseling to 80 percent of participating families with youth for whom employment or age 18 was imminent. By the end of the third year of program operations, about half of youth and families had met these benchmarks. Progress toward service delivery benchmarks was consistently lower in Arizona, the state with the largest share of enrollees.
  • The lead agencies in the consortium states formally partnered with 36 organizations to deliver ASPIRE services in addition to those organizations that served on the states’ advisory committees. The findings of a network analysis of three states—North Dakota, Colorado, and Utah—indicated that both the administrators and frontline staff of ASPIRE partner organizations increased the amount of contact and the number and types of collaborations with their fellow ASPIRE partners as program implementation progressed. ASPIRE held semiannual multiday trainings for program staff from all of the states to foster networking across the states and peer-to-peer sharing of best practices. In addition, ASPIRE leadership within USOR provided oversight, support, and training. It was ultimately the responsibility of each state’s lead agency and site coordinator, however, to ensure that case managers could navigate their unique state systems and that the program delivered services consistent with its model. The process analysis suggests that the states’ success in these areas varied.
  • Youth in the evaluation’s control group were not able to access case management or any specific intervention services through ASPIRE. Because ASPIRE leveraged existing programs and providers for most of its services (with the exception of youth self-determination training), however, in principle, control group youth had access to many of the same services to which case managers referred treatment group youth. Changes that occurred as a result of the Workforce Innovation and Opportunity Act may have increased opportunities for control group youth to receive such services. In some of the consortium states, career exploration activities and self-determination training became more readily available to youth, and new partnerships between vocational rehabilitation agencies and schools in some states may have increased the likelihood that youth would be connected to these services. By October 2017, the consortium states varied in how much information they were providing to families about new or enhanced services, and how far they had progressed in planning and implementing them. For states further along, the contrast between the experiences of treatment and control group youth may have been diluted, having implications for the ability of the evaluation to detect program impacts.

PROMISE—Promoting Readiness of Minors in Supplemental Security Income (SSI)—was a joint initiative of the U.S. Department of Education (ED), the Social Security Administration (SSA), the U.S. Department of Health and Human Services, and the U.S. Department of Labor to fund and evaluate programs to promote positive changes in the lives of youth who were receiving SSI and their families. Under cooperative agreements with ED, six entities across 11 states enrolled SSI youth ages 14 through 16 and implemented demonstration programs intended to (1) provide educational, vocational, and other services to youth and their families and (2) make better use of existing resources by improving service coordination among state and local agencies. Under contract to SSA, Mathematica Policy Research is evaluating how the programs were implemented and operated, their impacts on SSI payments and education and employment outcomes for youth and their families (using an experimental design under which we randomly assigned youth to treatment or control groups), and their cost-effectiveness.

In this report, we present findings from the process analysis of the first three years of the implementation and operation of the Achieving Success by Promoting Readiness for Education and Employment (ASPIRE) program, which operated in Arizona, Colorado, Montana, North Dakota, South Dakota, and Utah. The findings are based on data collected through October 2017 via site visits to ASPIRE, telephone interviews with and social network surveys of program administrators and staff, and the management information system that the program’s staff used to record their efforts. The report describes how ASPIRE engaged with youth, the services the program provided to them and their families, and the collaborations the program fostered to support its efforts. It also highlights information about the experiences of control group youth that could have implications for the evaluation’s impact analysis.

Project

Evaluation of the Promoting the Readiness of Minors in Supplemental Security Income PROMISE Grants

Funders

Social Security Administration

Time Frame

2013-2022

Senior Staff

AnnaMaria McCutcheon
Read More

Mary Anne Anderson
Read More

Gina Livermore
Read More

Jacqueline Kauff
Read More

Todd Honeycutt
Read More