Evaluation

Introducing NEW Research on the SBCT Approach

A new evaluation study from the American Institutes for Research (AIR) provides further evidence that the Safe Babies Court Team™ (SBCT) improves outcomes for very young children in the child welfare system. The researchers examined retrospective data from three SBCT sites (Little Rock, Arkansas; Tulsa, Oklahoma; and Des Moines, Iowa), comparing children whose cases were assigned to an SBCT by the court system to cases who experienced traditional dependency court and child welfare processes. The study found that children in cases adjudicated by an “SBCT judge” –that is, a judge who participated in trainings provided by ZERO TO THREE on the science of early childhood development, the impacts of trauma, and judicial practices that support family engagement – achieved permanency four months sooner than cases who experienced traditional dependency court. While there was no statistically significant difference in the likelihood of recurrence between cases seen by an SBCT judge and a traditional dependency court judge, cases that experienced the full SBCT approach (i.e., where all core components of the approach were in place) were 5 times less likely to experience a recurrence of abuse or neglect. This study did not examine specific permanency outcomes but other research has shown high rates of reunification for families participating in SBCTs, even for parents with significant histories of trauma and serious adversity.

The study also examined maltreatment outcomes for children whose families participated in the full SBCT approach and children whose cases were seen by an SBCT judge. In comparing maltreatment recurrence for these two groups, the researchers found that enhanced judicial practice alone is not sufficient for preventing maltreatment recurrence.

This finding underscores the importance of SBCT’s two-generation, holistic approach in ensuring that parents receive timely, meaningful supports that build their protective capacity and prevent maltreatment. It also underscores how critical other SBCT core components are for preventing recurrence and promoting other positive outcomes for children and families, including:

• The role of the Community Coordinator in building and strengthening linkages to services and helping families to navigate needed services;
• The intensive work of the Family Team in engaging and empowering parents and engaging in collaborative, proactive problem-solving that results in expedited receipt of needed services;
• The collective impact work of the Active Community Team in bringing stakeholders together to reduce racial disparities, address gaps in system coordination and the availability of services, and drive system improvements through new practices and policies in the court and child welfare system.

The promising findings in the AIR study adds to the growing evidence that the SBCT approach results in positive outcomes for children. In the next year, we anticipate that the evidence base will grow further with the results of a quasi-experimental multisite evaluation study that is currently underway and which will capture the prevention orientation of the HRSA/MCHB-funded Infant-Toddler Court Program.

Please read here for an accompanying FAQ for judges on the SBCT approach.

The ITCP evaluation uses a mixed-methods design consisting of three components:

• Continuous quality improvement (CQI) that includes monthly meetings with each evaluation site to review performance indicators and assess their progress by comparing data over time;
• A process evaluation that examines the sites’ adoption of core components and fidelity to the Safe Babies Court Team approach; and
• Outcome evaluation using a quasi-experimental design with a comparison group from the National Survey of Child and Adolescent Well-Being (NSCAW), the only nationally representative sample of children involved with the child welfare system. The comparison group was created by using propensity score matching to select a subsample of infants and toddlers with a maltreatment investigation and a placement history similar to the children in ITCP sites. Parents and caregivers complete at baseline and before the case is closed an audio-computer assisted self-interviewing that uses selected instruments used by NSCAW related to child and parent well-being outcomes.

Evaluation Questions Driving the ITCP

Process Evaluation:
Stakeholder and System Collaboration 

• What characterizes the collaboration among stakeholders?
• What supports the development and sustainability of stakeholder relationships?
• What factors are associated with sustaining the ITCP during periods of transition (e.g. loss of judge or Community Coordinator)?

Child Permanency 

• What intervention inputs drive reunification?
• What intervention inputs drive other positive permanency outcomes?

Outcome Evaluation:
Child Safety

• Are children involved with ITCP sites less likely to have substantiated or indicated maltreatment re-reports compared to a similar group of children from NSCAW?

Child Placements

• Are children involved with ITCP sites more likely to have two or fewer placements compared to a similar group of children from NSCAW?

Child Permanency 

• Are children involved with ITCP sites more likely to reach permanency within the first 12 months compared to a similar group of children from NSCAW?

Child Well-Being

• Socio-emotional: Are children involved with ITCP sites less likely to have emotional/behavioral problems compared to a similar group of children from NSCAW?
• Physical Health: Are children involved with ITCP sites more likely to have better (excellent or very good) health compared to a similar group of children from NSCAW?

Access to Services

• Are children involved with ITCP sites more likely to have access to needed services (e.g. well-being checkups, screening, early intervention) compared to a similar group of children from NSCAW?
• Are parents involved with ITCP sites more likely to have access to needed services (e.g. trauma-informed mental health, substance abuse, parenting) compared to a similar group of parents from NSCAW?
• Click here to read the Executive Summary of the QIC-ITCT 2017 Final Evaluation Report.
• Click here to read the full QIC-ITCT 2017 Final Evaluation Report.
• Click here to read an overview of the evaluation of the QIC-CT Project.
• Click here to read an Executive Summary of The Baseline Evaluation Report of the Quality Improvement Center for Research-Based Infant-Toddler Court Teams released in August 2016.

Additional Evaluation Resources

The data briefs, one-pagers and infographics here tell the story of our work with infants, toddlers and their families over the past few years. These were completed by our independent evaluator, RTI International, using data from all sites involved in the project. These include the following:

• Infographic: Making a Difference in the Lives of Families: The Safe Babies Court Team™ Approach

Quality Improvement Center for Infant-Toddler Court Teams. (2018). Making a Difference in the Lives of Families: The Safe Babies Court Team™ Approach.

• Adverse Childhood Experiences, Family Risk Factors, and Child Permanency Outcomes of Very Young Children Involved in Safe Babies Court Team™ Sites
• Using Data to Drive Decision Making and Promote Continuous Quality Improvement
• Helping Children Affected by Adult Opioid Use
• Working with Families in the Child Welfare System Affected by Incarceration
• When Less is Better: Reducing Out-of-Home Care Placements for Young Maltreated Children
• Services Support for Young Children and Families in the Child Welfare System
• ACEs Permanency Webinar

Past Evaluations of the Safe Babies Court TeamTM Approach

James Bell Associates Evaluation of the Court Teams for Maltreated Infants and Toddlers
The first Safe Babies Court Team evaluation, completed in 2009 by James Bell Associates, looked at evidence of system change, knowledge among Court Teams stakeholders regarding the impact of maltreatment on early development, and short-term outcomes for infants and toddlers monitored by the Safe Babies Court Teams. This process and outcome evaluation examined three questions using site visits, interviews, focus groups, court observations, and document review. The questions were:

1. To what extent is there evidence that systems change is underway at each program site through implementation of the Court Teams model;
2. What is the state of knowledge among Court Team stakeholders regarding the impact of abuse and neglect on early development and the needs of maltreated infants and toddlers who come through the courts; and
3. What short-term outcomes result for infants and toddlers served by the Court Teams?

Fostering a Permanent Home: A Mixed Methods Evaluation of the ZERO TO THREE Court Teams for Maltreated Infants and Toddlers Initiative
The second Safe Babies Court Team evaluation, completed by Kimberly McCombs-Thornton, PhD of The University of North Carolina at Chapel Hill in 2011, looked at the effect of Safe Babies Court Teams on time to permanency and how children exit the foster care system. It also examined how program components or client characteristics affected time to permanency. Dr. McCombs-Thornton utilized two populations: children served by the initial four Safe Babies Court Teams, and a nationally representative sample of young child welfare participants found in the National Survey of Child and Adolescent Well-Being. Using these populations, Dr. McCombs-Thornton answered three questions:

1. Does the Court Team have an effect on time to permanency;
2. Does the Court Team influence how children exit the foster care system; and
3. Do program components or client characteristics contribute to time to permanency

Investing in our Most Vulnerable: A Cost Analysis of the ZERO TO THREE Safe Babies Court Teams Initiative
This evaluation looks at the effect of the expedited permanency outcome on the cost of Safe Babies Court Team implementation. Economics for the Public Good calculated an average direct cost of $10,000 per child.  These costs are similar to or substantially lower than those found in other early childhood interventions. Short-term savings generated by their earlier exits from foster care are estimated at an average of $7,300 per child.  In other words, the Court Teams’ reduced costs of foster care placements alone cover two-thirds of the average costs per child. This study also showed that children involved with ZTT Court Teams access more services than the comparison group.  In particular, Court Teams children were significantly more likely to receive a developmental screening (92% v. 25%), health care visit (94% v. 76%), and dental visit (29% v. 18%).  The study also demonstrated ZTT Court Teams’ ability to leverage substantial in-kind resources: for every grant dollar, ZTT was able to generate another dollar of in-kind support.