The US Institute for Education Sciences released an important report about the effects of Reading First program at the end of April. In the report, “Reading First Impact Study: Interim Report,” Beth C. Gamse and colleagues describe the methods and findings of a study mandated by law to examine the effects of the RF program on instruction in classrooms and outcomes for children attending those schools where it it is implemented.
For those of us who think RF methods represent an improvement over garden-variety reading instruction, the results are disappointing. Although teachers were found to be devoting more time to phonemic awareness, phonics, vocabulary, fluency, and comprehension, students were not experiencing significant improvements in their reading outcomes.
Here’s the executive summary from the report.
This report presents preliminary findings from the Reading First Impact Study, a congressionally mandated evaluation of the federal government’s $1.0 billion-per-year initiative to help all children read at or above grade level by the end of third grade. The No Child Left Behind Act of 2001 (P.L. 107-110) established Reading First (Title I, Part B, Subpart 1) and mandated its evaluation. This evaluation is being conducted by Abt Associates and MDRC with RMC Research, Rosenblum-Brigham Associates, Westat, Computer Technology Services, DataStar, Field Marketing Incorporated, and Westover Consulting under the oversight of the U.S. Department of Education, Institute of Education Sciences (IES).
The present report is the first of two; it examines the impact of Reading First funding in 2004-05 and 2005-06 in 17 school districts across 12 states and one statewide program (18 sites). The report examines program impacts on students’ reading comprehension and teachers’ use of scientifically based reading instruction. Key findings are that:
• On average, across the 18 participating sites, estimated impacts on student reading comprehension test scores were not statistically significant.
• On average, Reading First increased instructional time spent on the five essential components of reading instruction promoted by the program (phonemic awareness, phonics, vocabulary, fluency, and comprehension).
• Average impacts on reading comprehension and classroom instruction did not change systematically over time as sites gained experience with Reading First.
• Study sites that received their Reading First grants later in the federal funding process (between January and August 2004) experienced positive and statistically significant impacts both on the time first and second grade teachers spent on the five essential components of reading instruction and on first and second grade reading comprehension. Time spent on the five essential components was not assessed for third grade, and impacts on third grade reading comprehension were not statistically significant. In contrast, there were no statistically significant impacts on either time spent on the five components of reading instruction or on reading comprehension scores at any grade level among study sites that received their Reading First grants earlier in the federal funding process (between April and December 2003).The study’s final report, which is due early 2009, will provide an additional year of follow-up data, and will examine whether the magnitude of impacts on the use of scientifically based reading instruction is associated with improvements in reading comprehension.
Download the entire report or parts of it from the list of alternatives on this page.
In a story headlined “Reading Program Is Called Ineffective,” Sam Dillon of the New York Times reported about the interim report about the impact of the Reading First program on reading instruction and achievement in the primary grades. His lead was, “President Bush’s $1 billion a year initiative to teach reading to low-income children has not helped improve their reading comprehension, according to a Department of Education report released on Thursday.” Mr. Dillon incorporated multiple political concerns about RF in his coverage of the report, quoting Senator Edward Kennedy, recounting an investigation of relations between officials and publishers, and noting the resignation of the program’s original leader. Link to the story (registration may be required).
Obligatory reminder: I am a member of the Reading First federal advisory committee. I am not, however, speaking for the committee, my fellow panelists, nor the US Department of Education here.
Update (1:25 pm 2 May 2008): Kathleen Kennedy Manzo published an article about the report on Education Week. David Glenn reported about it for the Chronicle of Higher Education.
Sphere: Related Content
I read the same results and note that in Canada, state-mandated tests give similar results for comprehension among schools that use a direct instruction phonics approach, although the students are reading at or above grade level in other assessments.
The fact that young children decode is promising for later comphrension. Also if there isn’t “significant diffences” from those schools not under the plan, how do the researchers actually know what those other schools are really doing? Perhaps as a result of generally more attention paid to literacy initiatives ALL schools are doing more by using effective teaching practice, hence little relative differences. The study did show that there was improvement after all! Teachers who support effective practice should not be disheartened.
Do your readers know about the Clackmannanshire, Scotland UK study by Johnson & Watson (2005)? This seven-year longitudinal study compared three approaches: Synthetic phonics(British terminology for a direct instruction sequential systematic phonics program before reading text), analytic phonics (balanced literacy–or teaching phonics unconnected to text), and a control group.
They found that teaching hundreds of low income (K or Grade 1) children “synthetic” phonics (20 minutes a day for 16 weeks) put them miles ahead of their higher-income peers even after seven years in all areas including comprehension! In fact, the researchers noticed that the gains were so significant they felt it unethical not to teach the rest of the control groups using synthetic phonics.
It can be accessed here:
http://www.scotland.gov.uk/Publications/2005/02/20688/52464
Frankly I don’t think the Reading First programs were getting it right even though they were supposed to be using the recommendations of the National Reading Panel. Some of those recommendations still border on what in Canada we call “balanced literacy” –whole language with phonics on the side.
educ8m (great license plate; are you associated with http://educ8m.com/ ?),
I agree with much of your analysis. Even though I’m not sure that the students in the “experimental group” in the RF impact analysis had lower outcomes than those in “comparison group,” if there were differences in other aspects of reading (e.g., accurate decoding at a normal speech rate), then those students in the experimental group should have a longer-term advantage.
Thanks for the reminder about Johnson and Watson’s study. For those who do not have the complete reference, here ’tis:
Johnston, R.S, & Watson, J. (2005) The effects of synthetic phonics teaching on reading and spelling attainment, a seven year longitudinal study. Scottish Executive Education Department. (PDF available here.)
What programs were used in RF schools? I know they only had to be “based” on scientific research. There is no silver bullet, but if we want to make any meaningful improvement on reading achievement, I think we need to follow England’s lead and implement synthetic phonics…
To John L. No I’m not associated with that blog.
From what I read on Dr. Lyon’s interview on this topic, there was no “control” group. This was in no way a scientific study. I still cannot find out exactly what particular pedagogy, programs, reading material, etc. was used in Reading First schools.