Transcript
1
SMALL MIRACLES IN TULSA: THE EFFECTS OF UNIVERSAL PRE-K ON COGNITIVE DEVELOPMENT
William T. Gormley, Jr. Public Policy Institute Georgetown University Washington DC 20007
[email protected]
Paper Presented at the National Conference of the Early Childhood Research Collaborative, Sponsored by the Federal Reserve Bank of Minneapolis and the University of Minnesota, Minneapolis, Minn., December 7, 2007
2 ACKNOWLEDGMENTS The research reported here, from 2003 test score data, was funded by the Foundation for Child Development, the National Institute for Early Education Research (NIEER), and the Pew Charitable Trusts. The author would like to thank them for their generous support. He would also like to thank his collaborators, Ted Gayer and Deborah Phillips, and his wonderful research assistants at the Center for Research on Children in the U.S. (CROCUS). The author alone is responsible for the contents of this report.
3 SMALL MIRACLES IN TULSA: THE EFFECTS OF UNIVERSAL PRE-K ON COGNITIVE DEVELOPMENT
Parents and public officials are increasingly concerned about the school readiness of young children. In recent years, state governments have boosted their support for preK programs. Several states, including Oklahoma, have opted for universal pre-K, making pre-K available, on a voluntary basis, to all four-year-olds. At the same time, the federal government, through the No Child Left Behind Act, has imposed new testing requirements on public schools, to determine whether students as a whole and particular subgroups of students are making good academic progress. These trends have heightened interest in the effectiveness of pre-K programs. The Oklahoma pre-K program is of special interest because it enrolls a higher percentage of four-year-olds than any pre-K program in the U.S (Barnett et al. 2006). It is also of particular interest because it is based in the public schools and because it places strong emphasis on high quality: all lead teachers must have a college degree and must be early-childhood certified; to recruit and retain outstanding individuals, they are paid at the same rate as other public school teachers. Many studies have demonstrated that considerable benefits flow from a highquality targeted pre-K program. But can a large-scale universal pre-K program also produce substantial benefits by enhancing the school readiness of young children? Do all children benefit from such a program? Do some children benefit more? And how large are the impacts of such a program, in absolute or relative terms? We set out to answer these questions in Tulsa, Oklahoma, the largest school district in Oklahoma. In addition to its size, Tulsa has the advantage of a racially and
4 ethnically diverse school population, which facilitates estimates of subgroup impact. 1 With the cooperation of the Tulsa Public Schools (TPS), we were able to test students in August 2003, using a nationally normed test. This study, whose results have been reported elsewhere (Gormley et al. 2005), built upon an earlier study, in which we utilized a homegrown testing instrument (Gormley and Gayer 2005).
Methodology
Unlike previous researchers, who often used cross-sectional data or a pre-test, post-test research design, we decided early on that a better research strategy was to use a regression discontinuity design that takes advantage of a September 1 birthday requirement for enrollment in pre-K. If a student turned four years old on or before September 1, 2002, then he or she was eligible to enroll in Oklahoma’s pre-K program in 02-03. If a student turned four years old anytime between September 2, 2002 and August 31, 2003, then he or she had to wait until the 03-04 school year to enroll. By taking advantage of this cutoff policy, which was strictly enforced, we can construct a comparison group that closely resembles our treatment group – TPS kindergarten students who participated in the TPS pre-K program in 02-03. Our comparison group consists of TPS pre-K program entrants in 03-04. The principal strength of this research design is that both sets of students have parents who affirmatively chose to place them in the TPS pre-K program. This helps to ensure that the students are alike in their talent and motivation – intangibles that are extremely difficult to measure.
5 Of course, the two groups do differ in one key respect – students in the treatment group are, on average, exactly one year older than students in the control group. This discrepancy, however, is very easy to deal with through the use of a statistical control for the child’s age (as measured by the date of birth). We also find it useful to control for other demographic variables (gender, race/ethnicity, school lunch status, and mother’s education), even though these controls (other than age) should, in theory, be unnecessary if the regression discontinuity design works properly.
Data
In August 2003, we administered three subtests of the Woodcock Johnson Achievement Test. The testers were TPS kindergarten and pre-K teachers, who administered the tests just prior to the commencement of classes (i.e., a genuine pre-test). We trained them to administer these tests, with help from Barbara Wendling, an independent consultant from Dallas, Texas, who is an expert on the Woodcock Johnson Test. All tests were administered one-on-one, while the child’s parent (or guardian) sat in a nearby room. While waiting, the parent (or guardian) completed a survey that provided valuable information to us, including the mother’s education. The three subtests selected were the Letter-Word Identification Test, which measures pre-reading skills; the Spelling Test, which measures pre-writing skills; and the Applied Problems Test, which measures pre-math skills. These particular subtests were selected because they are thought to be especially appropriate for relatively young children.
6 Our aim was to test as many kindergarten students and pre-K students as possible, at the same point in time. Ultimately, we were successful in testing 85.0 percent of all kindergarten students (3,149/3,727) and 84.5 percent of all pre-K students (1,567/8,843). In general, the tested children closely resembled the universe of children from which they were drawn, though there were some discrepancies. 2 Once the testing and coding was completed, we compared those kindergarten students who were in TPS pre-K the previous year (our treatment group) with those students who were about to begin TPS pre-K (our control group). With only two exceptions, the treatment group students closely resembled the control group students in their demographic characteristics. 3
Findings
As Figure 1 indicates, the overall effects of the TPS pre-K program are substantial. For children as a whole, we see: a 52 percent gain in the Letter-Word ID test score; a 27 percent gain in the Spelling test score; and a 21 percent gain in the Applied Problems test score. That is the average change in each test score attributable to the TPS pre-K program, above and beyond the gains that naturally occur as the child ages one year, and after controlling for various demographic variables, including age. Because TPS has a diverse student population, we can break these findings down by race and ethnicity. As Figure 2 indicates, all racial and ethnic groups benefit from the TPS pre-K program. Hispanic students and black students experience statistically significant gains for all three tests. Native American and white students experience
7 statistically significant gains for two of three tests. Gains for Hispanic students are especially impressive. Specifically, Hispanic students experience a 79 percent gain in Letter-Word ID, a 39 percent gain in Spelling, and a 54 percent gain in Applied Problems, above and beyond the gains that occur as the child ages one year and after controlling for demographic differences. Because Oklahoma’s four-year-old students are eligible for pre-K, irrespective of income, we can also break our results down by socio-economic status, as measured by eligibility for a free or reduced price lunch. As Figure 3 indicates, all socio-economic groups benefit from the TPS pre-K program. Students eligible for a free lunch (the poorest students) experience statistically significant gains for all three tests, while students eligible for a reduced-price lunch and students who must pay for a full-price lunch experience statistically significant gains for two of three tests. In general, more disadvantaged students (free and reduced price lunch) benefit more from TPS pre-K than more advantaged students (full price lunch). Evaluators sometimes measure effect sizes by comparing the test score gain coefficient to the standard deviation for the control group. This helps to facilitate comparisons across studies by creating a common metric. As Figure 4 indicates, effect sizes for the Tulsa pre-K program are quite substantial: 0.79 of a standard deviation for Letter-Word Identification; 0.64 of a standard deviation for Spelling; and 0.38 of a standard deviation for Applied Problems. By way of comparison, these effect sizes exceed those reported for other pre-K programs (Magnuson et al. 2007) and for highquality child care programs (Peisner-Feinberg et al. 2001; NICHD and Duncan 2003). They come close to the effect sizes reported for such legendary early childhood
8 demonstration programs as the Perry Preschool Project (Ramey et al. 1985) and the Abecedarian Project (Ramey et al. 2000). For a hypothetical child who just made the pre-K eligibility cutoff by one day (born on September 1, 1998) and another hypothetical child who just missed the pre-K eligibility cutoff by one day (born on September 2, 1998), it is possible to convert raw test scores into age-equivalent test scores. As Figure 5 reveals, the child exposed to TPS pre-K is substantially better off. Whereas the child not yet exposed to TPS pre-K falls below national norms for a five-year old (more precisely, a child who is exactly five years old) for all three tests, the child exposed to TPS pre-K exceeds national norms in Letter-Word Identification and equals national norms in Spelling. Expressed a bit differently, TPS pre-K yields test score gains of approximately seven months for LetterWord ID, six months for Spelling, and four months for Applied Problems.
Discussion
Over the past few years, Tulsa has become a showcase for early childhood education. Oklahoma is the nation’s leader in providing state-funded pre-K to approximately 70 percent of all four-year-olds, and it is unique in offering universal preK through the public schools, as opposed to a mixed-services delivery system. The Tulsa Public Schools pre-K program has attracted attention in part because of the results reported here but also because we do not normally think of a relatively poor, relatively conservative state as leading the nation in an important social policy realm. When
9 Appalachian State beats the University of Michigan, that is news. When Tulsa outshines the rest of the nation in early childhood education, that is also news. But what exactly can we learn from these results? And what remains to be investigated? First, a well-designed universal pre-K program can produce impressive improvements in school readiness. Thanks to the TPS pre-K program, most incoming kindergarten students in the Tulsa Public Schools have better pre-reading skills, better pre-writing skills, and better pre-math skills. They are readier to learn than they
otherwise would be. In short, the pre-K program has given Tulsa children a valuable boost in cognitive development as they begin their school years. Second, a well-designed universal pre-K program can benefit children from diverse racial and ethnic backgrounds and from diverse social strata. The TPS pre-K program benefits white, black, Hispanic and Native American children. It benefits poor children (eligible for a free lunch), near-poor children (eligible for a reduced price lunch), and non-poor children (ineligible for any school lunch subsidy). Although some critics have contended that only working poor children benefit from a universal pre-K program (Fuller 2007), there is no basis for that allegation. As we have seen, children ineligible for a school lunch subsidy benefited greatly from the Tulsa pre-K program. In 2003 the reduced price lunch income cutoff point (as specified by the U.S. Department of Agriculture) was approximately $35,000, just short of the median income of families with children in Oklahoma – approximately $39,000. Thus children who were ineligible for a school lunch subsidy came from families with incomes above or close to the state’s median income for families with children. By any reasonable definition, these children are middle-class.
10 Third, a well-designed universal pre-K program produces benefits that compare quite favorably to those produced by other human investment initiatives, including class reduction programs and job training programs. Overall students benefit about 0.15
standard deviations from assignment to a smaller class (Schanzenbach 2006) – a much smaller benefit than those reported here. Job training programs also seem to be less efficacious than well-designed early childhood education programs (Heckman 2000). Fourth, a well-designed universal pre-K program need not impose major burdens on taxpayers. In Oklahoma, the state’s contribution to the TPS pre-K program is
approximately $4,000 for a full-day slot, even less for a half-day slot. To be sure, other resources augment these expenditures, including in-kind contributions from the local school system (physical plant, maintenance, etc.) and Title I contributions from the federal government (used to help fund full-day programs that serve disadvantaged children). Nevertheless, the overall costs to taxpayers are relatively modest. There are several questions that this study does not answer. It says nothing about the long-term effectiveness of the TPS pre-K program. And there are reasons to be concerned about this. A number of studies have shown that “fade out” can occur over time, if public schools do not work vigorously to sustain short-term gains experienced by students who have attended a Head Start program (McKey et al. 1985: III-20; Currie and Thomas 1995; Lee and Loeb 1995). On the other hand, other studies have shown that fade out is not inevitable (Currie and Thomas 1999; Garces and Currie 2002; Ludwig and Miller 2007). The key question, in Tulsa and elsewhere, is whether public schools (and, for that matter, other schools) make adjustments in their curriculum (especially K-3) that take into account the dramatic changes in school readiness that are taking place. If
11 schools adjust their teaching styles and tempos accordingly, then short-term cognitive gains may persist over time. If they do not, then short-term cognitive gains are likely to evaporate. Our study also cannot directly answer the question: how has the Tulsa Public Schools managed to produce such wonderful short-term results? Is it because all the teachers have a B.A. degree and are early childhood certified? That question is hotly disputed. Is it because all the pre-K programs (except for authorized collaboratives) are located within the public schools? That is also unclear. However, based on recent data, gathered in the spring of 2006, we can report that TPS pre-K classrooms differ significantly from other school-based pre-K programs in 11 states (Phillips, Gormley and Lowenstein 2007). Specifically, TPS pre-K programs offer higher levels of “instructional support” (as measured by the Classroom Assessment Scoring System) than other school-based programs. They also devote considerably more time to pre-reading, pre-writing, and pre-math skills than other school-based programs. Some critics have charged that this comes at the expense of attention to socio-emotional development or to higher-order critical thinking skills (Fuller 2007). But that allegation appears to be unfounded. If we look at attention to higher-order critical thinking skills, as measured by a “concept development” score, that is higher in TPS pre-K programs than in other school-based pre-K programs. If we look at levels of emotional support in the classroom, they are neither better nor worse in TPS pre-K programs than they are in other school-based pre-K programs. 4 In short, Tulsa teachers seem to be educating four-yearolds in a developmentally appropriate way.
12
Conclusion The Tulsa Public Schools pre-K program, which became universal in 1998 as a result of state legislation, has been remarkably effective in enhancing the school readiness skills of young children. Kindergarten children who participated in the statefunded pre-K program have stronger pre-reading skills, stronger pre-writing skills, and stronger pre-math skills than would otherwise be the case. Children from diverse racial and ethnic backgrounds and from diverse socio-economic backgrounds benefit from the program. Although Hispanics benefit the most from the program, blacks, whites, and Native Americans also benefit considerably. Although disadvantaged children benefit the most from the program, middle-class children also benefit considerably. Now that the Oklahoma pre-K program is firmly established, the challenge for the Tulsa Public Schools and other school districts will be to build on the gains of four-yearolds by adapting elementary education teaching practices to the new skills that most kindergarten students now possess. The challenge for other states that seek to realize similar gains through a universal pre-K program will be to determine which elements of Tulsa pre-K experience need to be replicated in order to achieve similar learning gains.
13
Figure 1: Overall Effects of Tulsa Pre-K Program
0.6 0.52 0.5
0.4 Test Score Gains
0.3
0.27 0.21
0.2
0.1
0 Letter-Word Identification Spelling Applied Problems
14
Figure 2: Effects of Tulsa Pre-K Program by Race/Ethnicity of Student
0.9 0.8 0.7 0.6 Test Score Gains 0.53 0.5 0.4 0.3 0.2
n.s. 0.39
0.79
0.54 0.49
0.52
0.35 0.29 0.21
n.s. 0.2
0.26
0.1 0 Black Hispanic Native American
0.06
White
Race/Ethnicity of Student Letter-Word Identification Spelling Applied Problems
n.s. = not significant
15
Figure 3: Effects of Tulsa Pre-K Program by Free Lunch Program Status of Student
0.8 0.74 0.7
0.6 Test Score Gains
0.56
0.5 0.41 0.4 0.31 0.3 0.28 0.37
0.2
n.s.
0.17
n.s.
0.11 0.1
0.09
0 Free Lunch Reduced-Price Lunch Full-Price Lunch
Lunch Program Status of Student Letter-Word Identification Spelling Applied Problems
n.s. = not significant
16
Figure 4: Test Score Gains Attributable to Tulsa Pre-K
0.9 0.79
0.8 Gains as Fraction of Standard Deviation for Control Group
0.7 0.64 0.6
0.5 0.38
0.4
0.3
0.2
0.1
0 Letter-Word Identification Spelling Applied Problems
17
Figure 5: Age-Equivalent Test Scores for Children Exposed to Tulsa Pre-K
5Y 2M 5Y 0M Age-Equivalent Test Scores
National Norm
4Y 9M
4Y 7M 4Y 6M 4Y 5M
Letter-Word Identification Pre-K
Spelling
Applied Problems
No Pre-K
Note: Test scores are expressed in years and months.
18 REFERENCES Barnett, W. Steven, Jason Hustedt, Laura Hawkinson, and Kenneth Robin. “The State of Preschool 2006.” New Brunswick, N.J.: National Institute for Early Education Research, Rutgers University, 2006. Currie, Janet and Duncan Thomas. 1995. “Does Head Start Make a Difference?” American Economic Review 85 (June), pp. 341-64. Currie, Janet and Duncan Thomas. 1999. “Does Head Start Help Hispanic Children?” Journal of Public Economics 74 (November), pp. 235-62. Fuller, Bruce. 2007. Standardized Childhood: The Political and Cultural Struggle Over Early Education. Stanford: Stanford University Press. Garces, Elena, Duncan Thomas, and Janet Currie. 2002. “Longer Term Effects of Head Start,” American Economic Review 92 (September), pp. 999-1012. Gormley, William, Ted Gayer, Deborah Phillips, and Brittany Dawson. 2005. “The Effects of Universal Pre-K on Cognitive Development,” Developmental Psychology 41 (November), pp. 872-84. Gormley, William and Ted Gayer. 2005. “Promoting School Readiness in Oklahoma: An Evaluation of Tulsa’s Pre-K Program,” Journal of Human Resources 40 (Summer), pp. 533-59. Heckman, James. 2000. “Policies to Foster Human Capital,” Research in Economics 54, pp. 3-56. Lee, Valerie and Susanna Loeb. 1995. “Where Do Head Start Attendees End Up? One Reason Why Preschool Effects Fade Out,” Educational Evaluation and Policy Analysis 17 (Spring), pp. 62-82. Ludwig, Jens and Douglas Miller. 2007. “Does Head Start Improve Children’s Life Chances? Evidence from a Regression Discontinuity Design,” Quarterly Journal of Economics 122, pp. 159-208. Magnuson, Katherine, Christopher Ruhm, and Jane Waldfogel. 2007. “Does PreKindergarten Improve School Preparation and Performance?” Economics of Education Review 78, pp. 115-57. McKey, Ruth, Larry Condelli, Harriet Ganson, Barbara Barrett, Catherine McConkey, and Margaret Plantz. 1985. “The Impact of Head Start on Children, Families and Communities.” Final Report of the Head Start Evaluation, Synthesis and Utilization Project.” June. Washington, D.C.: CSR, Inc.
19 National Institute of Child Health and Human Development and Greg Duncan. 2003. “Modeling the Impacts of Child Care Quality on Children’s Preschool Cognitive Development,” Child Development 74, pp. 1454-75. Peisner-Feinberg, Ellen, Margaret Burchinal, Richard Clifford, Mary Culkin, Carolee Howes, Sharon Lynn Kagan, and Noreen Yazejian. 2001. “The Relation of Preschool Child-Care Quality to Children’s Cognitive and Social Developmental Trajectories Through Second Grade,” Child Development 72 (September/October), pp. 1534-53. Phillips, Deborah, William Gormley, and Amy Lowenstein. 2007. “Classroom Quality and Time Allocation in Tulsa’s Early Education Programs.” Paper presented at the Society for Research in Child Development Meeting. Boston, Mass.: March 30. Ramey, Craig, Francis Campbell, Margaret Burchinal, Martie Skinner, David Gardner, and Sharon Ramey. 2000. “Persistent Effects of Early Childhood Education on HighRisk Children and Their Mothers,” Applied Developmental Science 4, pp. 2-14. Ramey, Craig, Donna Bryant and Tanya Suarez. 1985. “Preschool Compensatory Education and the Modifiability of Intelligence: A Critical Review.” In D.K. Detterman, ed., Current Topics in Human Intelligence: Vol. 1. Research Methodology. Norwood, NJ: Ablex Publishing, pp. 247-96. Schanzenbach, Diane Whitmore. 2006. “What Have Researchers Learned from Project Star?” Harris School Working Paper Series 06.06. Chicago: Harris School of Public Policy, August.
In the fall of 2003, the pre-K cohort in TPS was 36 percent white, 36 percent black, 18 percent Hispanic, 9 percent Native American, and 1 percent Asian. 2 The tested pre-K children were somewhat more likely to be black than the universe of pre-K children, and the tested kindergarten children were somewhat less likely to be Hispanic than the universe of kindergarten children. The tested kindergarten children also differed somewhat from the universe of kindergarten children in their eligibility for free lunch (Gormley et al. 2005: 874). 3 Students in the comparison group were somewhat less likely to be Hispanic and somewhat less likely to have a mother with no high school degree than students in the treatment group (Gormley et al. 2005: 877). 4 Of four emotional support dimensions in the Classroom Assessment Scoring System, Tulsa pre-K classrooms score higher than other school-based pre-K classrooms in one, lower in another. There are no statistically significant differences for the other two.
1