Evaluating an early education literacy curriculum

School districts depend on evidence from research evaluations to help them select curriculum that best supports student learning and achievement. Independent third-party evaluations of curriculum are crucial in helping districts to make these decisions, which is where we come in.

Challenge

McGraw Hill wanted to know how to determine the effectiveness of its World of Wonders early childhood literacy program in helping children ages 3-5 develop foundational literacy skills needed to succeed in kindergarten and beyond. How could an independent study ensure that findings are relevant across diverse student subpopulations?

Drawing on our extensive expertise in early childhood education, program research, and evaluation, our team led a two-year study to determine the extent to which the World of Wonders program supports the early literacy achievement of students.

We worked with McGraw Hill and a school district in central Oklahoma to obtain preschool literacy assessment data during their first two years of implementing the World of Wonders curriculum. Using kindergarten literacy assessments and demographic data as a nonequivalent control group posttest-only design, we compared students in the district who had attended district pre-K and experienced World of Wonders to those who had not.

Our study used a large sample size of diverse students, including beginning-of-year and end-of-year data on 306 prekindergarten (Pre-K) students in the first year of implementation. The study also used beginning-of-year data from 742 kindergarten students who had attended district Pre-K (and, therefore, were exposed to World of Wonders) and 413 kindergarten students who did not attend district Pre-K.

We used school-level demographic characteristics as a proxy for student-level Pre-K demographics. We made statistical adjustments to the data prior to analysis to account for differences in performance that may have been attributed to the particular school that students attended.

Our study provides Every Student Succeeds Act (ESSA) Tier III promising evidence, with qualifications, that World of Wonders supports the early literacy achievement of students. ESSA is the 2015 revision of the federal law regulating PreK-12 education in the U.S., which strongly recommends that schools and districts implement interventions with documented evidence of effectiveness (i.e., evidence-based interventions). Districts consistently seek evidence of effectiveness when introducing curriculum into their schools that best supports achievement of students.

Students attending district Pre-K were 1.7 times more likely to be at Level 2 on the kindergarten early literacy assessment (two levels higher than Level A) than students not attending Pre-K. Students not attending district Pre-K were 1.9 times more likely to be at Level A (lowest level) than students attending district Pre-K.

We also saw statistically significant improvements from beginning-of-year to end-of-year on each of the Early Literacy Quick Assessment subscales within both implementation years of the study. Among students who entered kindergarten in the district, those students who had attended district Pre-K implementing World of Wonders scored higher on average on the beginning-of-year Direct Reading Assessment (DRA2), even after accounting for student demographics.