Project Overview

Brodsky Research and Consulting is evaluating CCCAP stimulus-funded policies: Enrollment-Based Payment, Increased Absence Payment and Reimbursement Rate, Reduced Copayment, and Expanded Income Eligibility. These policies aim to increase financial stability for providers and support family access to affordable child care.

This evaluation focuses on Step 1, Step 2, and Step 3 of the Steps to Building Evidence, using mixed methods to explore policy implementation and pre-post outcomes analysis. Data sources include family and provider CCCAP participation data connected through LINC. Additional data include CCCAP administrator and family interviews, surveys with providers, and analysis of county CCCAP plans.

Interim results show success in increasing infant and toddler CCCAP enrollment. These policies are key to maintaining CCCAP agreements with providers, keeping providers financially stable, and ensuring continued access for families. These policies should be included in recruiting materials with providers or families. Final results will be reported in September 2024.

Project Partner

The Colorado Lab selected Brodsky Research and Consulting (BRC) to evaluate stimulus-funded CCCAP policies. The BRC team of early childhood policy researchers has developed evidence-informed reimbursement rates for CCCAP and Universal Preschool. The BRC team is well positioned to ensure this evaluation is aligned with these broader efforts.

Project Resources

• CCCAP Provider Policy Evidence-Building Brief
• CCCAP Family Policy Evidence-Building Brief
• CCCAP Policy Theories of Change

Project Overview

Shultz Patel Evaluation is evaluating Employer-Based Child Care (EBCC) and Emerging & Expanding (E&E) Grants. Both grants support infrastructure and capital improvement projects to expand availability of licensed child care throughout Colorado.

This evaluation focuses on Step 2 and Step 3 of the Steps to Building Evidence, using mixed methods to explore grant implementation and early outcomes. Key data sources include administrative data, surveys with providers and Early Childhood Councils (ECCs), and interviews with participating employers, grantee providers, ECCs, and CDEC program leads.

Early results show success in partnering with ECCs for outreach and technical assistance partners on capital improvement projects. Despite implementation challenges for smaller providers and employers, the grants are filling an important need for child care infrastructure, especially in child care deserts. Final results will be reported in September 2024.

Project Partner

The Colorado Lab selected Schultz Patel Evaluation (SPE) to evaluate these two grants. The SPE team has experience conducting evaluations in early childhood, workforce development, and criminal justice. These include formative evaluations of new grant programs similar to stimulus-funded activities.

Project Resources

• EBCC and E&E Evidence-Building Brief
• EBCC and E&E Logic Model

Project Overview

Marzano Research is evaluating the Family Child Care Home (FCCH) and Availability and Outreach of Child Care (AOC) Navigator programs and Staffed Family Child Care Network Pilot (Network) implemented within Early Childhood Councils (ECCs). Navigators support providers through the licensing process and reporting their availability of care. The Network supports FCCH providers to build in-depth business knowledge.

This evaluation focuses on Step 1 and Step 2 of the Steps to Building Evidence, using descriptive analysis and mixed methods to examine implementation and participant experiences. Data sources include discovery with CDEC program leads, surveys and focus groups with providers and Navigators, and administrative data.

Interim results show variation in Navigator caseloads and engagement with providers. ECCs are a valuable implementation partner between CDEC and providers. FCCH providers need more support to participate in the Network. Final results will be reported in September 2024.

Project Partner

The Colorado Lab selected Marzano Research to evaluate these stimulus-funded activities. The team has expertise in early childhood program evaluation and supported implementation of the Preschool Development Grant Plan. The Marzano team is coordinating with the Early Childhood Council Leadership Alliance in their aligned evaluation efforts for the Navigator activities.

Project Resources

• Navigator Evidence-Building Brief
• Network Evidence-Building Brief
• Navigation Services Theories of Change
• Network Implementation Model

Project Overview

Marzano Research is evaluating the Substitute Fund Pilot. The Substitute Fund provides access to substitute coverage for providers and is implemented by an intermediary organization that supports Substitute Placement Agencies (SPAs). The Substitute Fund Pilot was launched in August 2023.

This evaluation focuses on Step 1 and Step 2 of the Steps to Building Evidence, using descriptive analysis and mixed methods to examine implementation and participant experiences. Data sources include discovery with CDEC program leads, surveys and focus groups with providers, SPAs, and substitutes, and administrative data.

Early results show that center-based and Family Child Care Home (FCCH) providers have different needs and preferences for using substitutes. The Pilot should explore how to reach more underserved areas and FCCH providers. Final results will be reported in September 2024.

Project Partner

The Colorado Lab selected Marzano Research to evaluate this stimulus-funded strategy. The team has extensive experience in early childhood program evaluation and supported implementation of the Preschool Development Grant Plan.

Project Resources

• Substitute Fund Evidence-Building Brief
• Substitute Fund Theory of Change
• Substitute Fund Implementation Model

Project Overview

MDRC is evaluating the Colorado Child Care Assistance Plan (CCCAP) Teacher Wage Increase Pilot. This Pilot aims to bring teacher pay closer to a livable wage.

This evaluation focuses on Step 1, Step 2, and Step 5 of the Steps to Building Evidence, using mixed methods and a randomized controlled trial to examine outcomes such as teacher retention and economic well-being. The study uses data sources connected through LINC: CDEC’s Professional Development Information System and Quality Rating and Improvement System, the Colorado Department of Labor and Employment’s unemployment insurance data, and the Colorado Department of Human Services Office of Economic Security’s Supplemental Nutrition Assistance Program data. Other data sources include baseline and follow-up surveys with teachers and directors, interviews with implementation partners, and cost workbooks.

Interim results indicate high teacher participation rates in the pilot and high recruitment and retention at participating provider sites. Final results will be reported in September 2024.

Project Partner

The Colorado Lab selected MDRC to evaluate this stimulus-funded strategy. The evaluation is also being funded by the Office of Planning, Research, and Evaluation, within the Administration for Children and Families, in the U.S. Department of Health and Human Services.

Project Resources

• Wage Increase Evidence-Building Brief

Project Overview

The Butler Institute is evaluating workforce preparation strategies, including the Recruitment & Retention Scholarship Program, Free Early Care and Education (ECE) Coursework, Apprenticeship Program, and Child Development Associate (CDA) Support Specialist Program. These strategies provide alternative pathways for higher education students to enter the ECE field by providing paid competency-based training, apprenticeships, and certificate completion support.

This evaluation focuses on Step 1, Step 2, and Step 3 of the Steps to Building Evidence, using mixed methods to explore program implementation and short-term outcomes. Key data sources include interviews with CDEC program leads, implementation partners, and employing providers; surveys with current and aspiring teachers; and administrative data.

Interim results show strong participation in the workforce preparation strategies. These strategies support entry to the field and provide promising strategies for increasing recruitment and retention. Final results will be reported in September 2024.

Project Partner

The Colorado Lab selected the Butler Institute for Families at the University of Denver to evaluate these stimulus strategies. The Butler Institute is the evaluation partner for the Preschool Development Grant. Their expertise in early childhood programs and systems is critical for aligning this evaluation with broader system efforts.

Project Resources

• Apprenticeship Evidence-Building Brief
• CDA Evidence-Building Brief
• Coursework and Scholarship Evidence-Building Brief
• Workforce Preparation Theory of Change

Project Overview

The Hub provides strategic consultation to CDEC for the Universal Preschool Program’s continuous improvement and evaluation strategy. This includes development of a multi-phased evaluation approach, theory of change, and implementation model. The Hub leveraged these foundational documents to support CDEC with the development of Wildly Important Goals (WIGs) and measures related to the Universal Preschool Program and Requests for Proposals to identify vendors for process evaluation and two continuous improvement efforts. The Lab will continue to support CDEC’s Universal Preschool Program team and coordinate the various evaluation and continuous improvement efforts through its Universal Preschool Evidence-Building Hub.

Project Partner

The Hub engaged Dr. Diana Schaack as a subject matter expert to support all aspects of Universal Preschool Program evaluation planning efforts. The Hub also worked with SRI International to conduct document review and stakeholder interviews to produce an initial theory of change and implementation model, along with the OMNI Institute to develop a user-friendly version of the theory of change and provide communications support.

Project Resources

• Universal Preschool Program Theory of Change

Project Overview

The Colorado Lab has provided strategic consultation to CDEC’s Workforce Division with data and analytic needs since 2018. Early work leveraged the Linked Information Network of Colorado (LINC) to create a sustainable and continuously updated early care and education (ECE) workforce dashboard. As a continuation of this work, the Colorado Lab supported a longitudinal study of the ECE workforce, their career trajectories, wages, and preparation pathways. 

The study is performing descriptive analysis to explore processes and outcomes to improve workforce development efforts. This study uses data from CDEC’s Professional Development Information System, the Colorado Department of Human Services Office of Economic Security’s Supplemental Nutrition Assistance Program, and the Colorado Child Care Assistance Program connected through LINC. 

Ongoing work includes additional workforce data analyses, improvements to the ECE workforce dashboard, and regular briefs and presentations to CDEC leadership related to key learning goals.

Project Partner

The Colorado Lab partnered with Chapin Hall at the University of Chicago to conduct the longitudinal study and ongoing workforce data analysis. Chapin Hall combines rigorous research to generate evidence and supports implementation of tested methods in the field. Their expert researchers have experience in data analytics and knowledge in early childhood programs and education pathways. 

Project Resources

• Colorado Early Care and Education Workforce Data Dashboard

Project Overview

The Hub provides strategic consultation to CDEC with embedding the Evidence-Base Decision-Making (EBDM) approach into agency strategic planning, performance management, and budgeting. The Hub supported CDEC’s strategic planning process by co-developing organizational theories of change for CDEC’s strategic goals. These theories of change will guide program management processes and support identification of shared outcomes and measures across divisions.

The Hub also supported CDEC in developing and completing an evidence portfolio process to inform immediate decisions related to continued funding for stimulus strategies, then the full set of CDEC budget and policy decisions. After action reviews informed modification of the evidence portfolio tool for utility beyond stimulus strategies and broader application along all Steps to Building Evidence. 

Finally, the Hub supported alignment of CDEC’s EBDM activities to the full state budgeting process. This included publishing an EBDM glossary to align definitions and reduce misunderstanding and conflict, particularly between branches of government. The Hub also trained performance leads from across state agencies on EBDM, the evidence portfolio process, and the organizational theories of change.

Project Partner

n/a

Project Resources

• CDEC Organizational Theories of Change
• Evidence Portfolios for State Agencies
• Evidence Portfolio Tool 
• EBDM Glossary

Project Overview

The Hub partnered with The Ohio State University’s Crane Center for Early Childhood Research and Policy (Crane Center) to 1) develop a cost model to estimate the costs for early care and education (ECE) providers to become licensed and advance quality levels in Colorado and 2) lay the foundation for an economic analysis of the benefits of child care. 

This descriptive analysis work includes background interviews with CDEC staff and stakeholders; focus groups with ECE providers; administrative data analysis from Colorado’s Quality Rating and Improvement System; landscape interviews with other states to determine alternative approaches for estimating the economic benefits of child care; and a feasibility analysis to determine which approach is best suited for Colorado.

This work concludes in fall 2024 and will result in a cost of care calculator and report, economic benefits of child care report (including a literature review and findings from the landscape and feasibility analyses), and presentations to CDEC leadership.

Project Partner

The Hub engaged the Crane Center to conduct this analysis. The Crane Center team’s researchers are experts in rigorous methodology and translating knowledge into real-world ideas and actions to enhance children’s learning, development, and well-being. 

Project Overview

To further demonstrate the impact of stimulus funding and to inform future decisions, the Hub is developing an external Early Childhood Stimulus Dashboard. The dashboard will provide a comprehensive view of stimulus-funded strategies across the state; the reach of these funds to subgroups of providers, workforce, and families; and identify gaps in uptake or access to funding in early childhood deserts. The dashboard will also serve as a starting point for other early childhood data projects, such as CDEC’s Early Childhood Information Data System (ECIDS). 

Data sources for the dashboard include early care and education (ECE) provider-, workforce-, and family-level data from the Linked Information Network of Colorado (LINC) and stimulus program administrative data. The external Early Childhood Stimulus Dashboard will launch in October 2024.

Project Partner

The Colorado Lab engaged Resultant to develop the dashboard. The Resultant team is conducting stakeholder interviews to inform the content and look of the dashboard, developing wireframes and technical documentation, conducting user testing, and producing the final populated dashboard. 

Project Overview

The Hub is coordinating with evaluation teams funded outside of the Hub to align deliverables and package evaluation findings for stimulus-funded strategies for CDEC. 

Some of these “aligned” evaluations have the same timeline as stimulus funding. Evaluation teams have separate contracts outside of the Hub, and the Hub provides supplemental funding and support for teams to produce core deliverables for synthesis to CDEC. For example, Marzano Research received external funding to evaluate CIRCLE Grants. The Hub is now supporting Marzano to produce deliverables aligned with other stimulus evaluation teams.

Other aligned evaluations pre-date and are funded outside of the Hub. These existing programs received stimulus funding, and evaluation may continue after stimulus funding ends. The Hub provides supplemental funding and support for evaluation teams to produce annual briefs to package progress and findings to date. For example, CDEC partnered with the Colorado Lab to lead evaluations for the Early Childhood Mental Health Consultation and Imagination Library of Colorado. Similarly, CDEC engaged the Center for Inclusive Design and Engineering (CIDE) at the University of Colorado Denver to evaluate Project Include. The Hub is supporting both aligned evaluation teams to produce annual briefs for CDEC.

Project Partner

The Hub is partnering with Marzano Research for their evaluation of the CIRCLE program. The Hub is also partnering with the Center for Inclusive Design and Engineering for their evaluation of Project Include. Finally, the Hub is leveraging the Colorado Lab’s own internal research teams for evaluation of Early Childhood Mental Health Consultation and Imagination Library of Colorado.

Project Resources

• CIRCLE Evidence-Building Brief
• Early Literacy: Imagination Library of Colorado Evidence-Building Brief
• Early Childhood Mental Health Consultation Evidence-Building Brief
• Access to Inclusive Care: Project Include Evidence-Building Brief