Teach Effectively!

Over on Reading Rockets, friend-of-TE Joanne has a quick reminder of the importance of monitoring student progress as a part of response-to-instruction efforts and pointers to helpful resources. Read the post here.

Hmmmm….friend-of-TE: FoTE?

Over on Britannica Blog, Dan Willingham has a new post entitled “What Makes a Good Fourth-Grade Reader? Knowledge.” Professor Willingham asks, “What makes for effective reading instruction?” and then answers, “A new study indicates that an important contributor is integrating material from other subjects into reading instruction.”

He’s talking about a recently released study by Wai Ming Cheung and colleagues from the University of Hong Kong. They examined predictors of reading literacy among fourth graders and found that “the most powerful predictor [of high outcomes] was the use of materials from other subjects as reading resources.”

This finding is consistent with points made elsewhere as well as here on TE: It’s not sufficient to teach decoding and abstract strategies. Kids need to read stuff! That means they need real content, and certainly one of the best sources of that content would be what they’re learning in other courses. It’s relevant, probably pitched at their level, etc.

Reading literacy of fourth-grade students in Hong Kong showed a remarkable improvement from 2001 to 2006 as shown by international PIRLS studies. This study identified various aspects of the teacher factor contributing to the significant improvement among students. A total of 4,712 students and 144 teachers from 144 schools were randomly selected using probability proportional-to-size technique to receive the Reading Assessment Test and complete the Teacher’s Questionnaire, respectively. A number of items pertaining to teachers’ instructional strategies and activities, opportunities for students to read various types of materials, practices on assessment, and professional preparation and perception, were found to be significantly correlated with the outcome of students’ reading literacy. Stepwise regression procedure revealed four significant predictors for students’ overall reading achievement. The most powerful predictor was the use of materials from other subjects as reading resources. Suggestions to improve quality of teaching of reading and further studies are made.

Cheung, W. M, Tse, S. K., Lam, J. W. I., & Loh, E. K. Y. (2009). Progress in international reading literacy study 2006 (PIRLS): Pedagogical correlates of fourth-grade students in Hong Kong. Journal of Research in Reading, 32, 293-308.

Link to Professor Willingham’s blog entry. Link to the abstract for the study by Professor Cheung.

Under the auspices of the Best Evidence Encyclopedia, Bob Slavin and colleagues Cynthia Lake, Susan Davis, and Nancy Madden released an analysis of the research literature on methods for teaching students who are struggling to learn to read, “Effective Programs for Struggling Readers: A Best-Evidence Synthesis.” In the synthesis they report the results of their examination of nearly 100 studies that used randomized or well-matched control groups, lasted for at least 12 weeks, and employed trustworthy measures of outcomes. The results of their review, which include both effect sizes and narrative descriptions of the studies, provide valuable insight into effective methods for remediating reading problems.

Key Findings

Overall, 96 experimental-control comparisons met the inclusion criteria, of which 38 used random assignment to treatments. Effect sizes (experimental-control differences as a proportion of a standard deviation) were averaged across studies, weighting by sample size.

One-to-One Tutoring by Teachers: ES=+0.38 in 19 studies
• Reading Recovery: ES=+0.23 in 8 studies
• Other programs: ES=+0.60 in 11 studies

One-to-One Tutoring by Paraprofessionals and Volunteers: ES=+0.24 in 18 studies
• Paraprofessionals: ES=+0.38 in 11 studies
• Volunteers: ES=+0.16 in 7 studies

Small Group Tutorials: ES=+0.38 in 11 studies

Classroom Instructional Process Approaches (low achievers): ES=+0.56 in 16 studies
• Cooperative Learning: ES=+0.58 in 8 studies

Classroom Instructional Process Programs with Tutoring (Success for All, low achievers): ES=+0.55 in 9 studies

Instructional Technology (low achievers): ES=+0.09 in 14 studies

Salvin, R. E., Lake, C., Davis, S., & Madden, N. A. (2009). Effective programs for stuggling readings: A best-evidence synthsis. Best Evidence Encyclopedia: http://www.bestevidence.org/reading/strug/strug_read.htm

In Child Development Jo Ann Farver and colleagues reported that young children who speak Spanish can learn English early literacy skills better when they receive instruction in English. That finding’s not particularly surprising, but there’s more: There’s a comparison of English-only and “transitional” methods. Children who received instruction in English-only or Spanish with transition to English (both using the Literacy Express Preschool Curriculum) had higher pre-literacy outcomes than peers who had been randomly assigned to receive the High/Scope Curriculum.
Continue reading ‘English intervention improves Spanish-speakers’ early literacy outcomes’

Among others, Ed Week reported about findings from a recent study showing the beneficial impact of having adults provide reading tutoring for young children. Under the headline “Volunteer Tutors Found to Help Poor Readers,” Catherine Gewertz wrote “A program that uses older volunteers as tutors has significantly improved the reading skills of students in the early grades, according to a study released today [10 April 2009].”

The study is an evaluation conducted by Nancy Morrow-Howell and colleagues of Washington University in St. Louis (MO, US) and in collaboration with Mathematica Policy Research. In brief, the study compared the reading outcomes (and other measures, e.g., teachers’ endorsement of the program) of 825 1st, 2nd, and 3rd graders on a suite of school measures (including decoding, comprehension, vocabulary, and teacher assessments). About half of the students received tutoring about once per week for a year. The data revealed that the tutored students made statistically greater gains on some measures than those found for the students in the control group.

The study deserves both accolades and scrutiny. Although it has multiple strengths (e.g., students were assigned randomly; pre-tests showed equivalent levels of competence; numbers were fairly large), there are problems, too. Not the least of these is that the report depends upon gain scores. Because the design fits classical experimental procedures, wouldn’t it be appropriate simply to examine the outcomes after the one year of tutoring? Also, what was the intervention?

One way to think about benefits for students is to examine effect sizes. Notably, in the document by Murrow-Howell and colleagues, the reported effect sizes are based on the gain scores and they were actually pretty small (0.13 to 0.17); to get these effect sizes, they—understandably—used only students who received at least 35 sessions of tutoring. Using the data in Table 3 (p. 12) and comparing the means for the experimental and control groups at posttest (a la a classical experiment), I get the even-smaller effect sizes shown in the table at the right. (For the technically inclinded, I calculated simple d using the control SD; the authors used Hedges’ G.)

Tutoring has well-documented benefits, but small-group instruction is equally effective and clearly more efficient (Elbaum et al., 2000; Journal of Educational Psychology). So, tutoring might not be a bad thing, but could the Experience Corps get more bang for their proverbial $$ if they had tutors take groups of, say, three? What is more, we don’t really know what happened to the students in the control condition. Did they get any supplemental help? If not, how was the study controlled for the possibility that Reading Instruction Plus Something More is simply better than Reading Instruction? Mayhaps answers to these questions are provided in a more detailed report than the one I used.

Of course, volunteers would need coaching and they’d have to learn to execute pretty specific lessons, which raises the related question: What did the tutors do with the students? I quickly scoured the Experience Corps Web site looking for a curriculum or set of guiding practices, but I came up empty handed. I’ll need help with this, and perhaps a kind reader can provide it in the comments.

Now, if the tutoring program was an adaptation of model, such as the one tested by Wallach and Wallach years ago, that would be a good thing. Or, perhaps even better, if it was something predicated on 100 Easy Lessons, that might be good.

If it wasn’t, though, then we need a new study comparing the tutoring methods employed in the sites in this study (call it Method A) to tutoring methods based on some known-to-be-powerful method (call it Method X) and to extra time on reading (which may be what the students in the current study got!). Ideally, this should be crossed against small-group supplements, something like the diagram here.

To be sure, it’s always easier to critique studies than it is to run them. I’m just fearful that the press coverage of this one is going to make more of it than it merits. It’s not a bad study, but those effect sizes are dwarfed by effects of powerful instructional procedures. And, yes, I know I’m ignoring the social validity of teacher satisfaction (would anyone actually expect teachers to disparge getting extra help for the children in their charge?), but it’s students’ outcomes that matter.

Read the article by Ms. Gerwertz, the actual report, the press release, and more about Experience Corps. And, here are some of the headlines that make me recommend caution:

“Students in urban schools get big boost from pioneering tutor program
Comprehension and other critical skills improve dramatically with one-on-one help from Experience Corps’ volunteers, a new study shows”—Christian Science Monitor.

“Study finds students with Experience Corps tutors make 60% more progress in critical reading skills than students without tutors“—Washington University News and Information Office; see also, “Students With Experience Corps Tutors Make 60% More Progress In Critical Reading Skills Than Students Without Tutors“—Medical News Today>; NewsGuide.us; BioMedicine, and others.

Sources

Elbaum, B., Vaughn, S., Hughes, M. T., & Moody, S. W. (2000). How effective are one-to-one tutoring programs in reading for elementary students are risk for reading failure? A meta-analyis of the intervention research. Journal of Educational Psychology, 92, 605-619.

Wallach, M. A., & Wallach, L. (1976). Teaching all children to read. Chicago: University of Chicago Press.

News sources around the US are abuzz with how state and local education agencies will spend the influx of funds for special education that comes with the US government’s increases in IDEA funding under the stimulus plan.

• Schools getting stimulus aid, but is it too late? from AZ Central.com
• Schools’ stimulus a puzzle from the Daily Tribune – Columbia, MO
• State schools to get $118M from the News Sentinel – Knoxville, TN
• A boost for poor, special-needs kids from the Star Tribune – Minneapolis, MN
• Berkeley Unified Will Receive $2.4 Million in Stimulus Funds Next Month from the Daily Planet – Berkeley, CA

Given that these funds may be pretty fleeting (here today, gone in a couple of years?), how wise is it to invest in more teachers whom the LEAs will have to dismiss or materials that are likely to need replacement in just a few years? I’d say, “NOT!”

Why not invest in staff development, using the two-year span to ensure that virtually all teachers know how to measure progress in easy-but-rigorous ways (e.g., curriculum-based measurement), implement school-wide discipline programs, and present lessons in systematic and (dare I say it?) instructive ways?

Here are some relevant links: Research Institute on Progress Monitoring and Student Progress; School-wide Positive Behavioral Interventions & Supports.

Over on Reading Rockets Kathleen McLane has an entry about monitoring progress that’s got a good intro and some valuable links. Take a look at it. There’s also video about progress monitoring in action, a link to a Web cast featuring Roland Good, Mary Ruth Coleman, and Michael C. McKenna discussing assessment including progress monitoring, and an opportunity to ask CBM guru Lynn Fuchs questions about monitoring progress (click the dropdown menu to select Professor Fuchs as the target of your question).

Reporting in the Journal of School Psychology, Elizabeth Crowe and colleagues recount the methods and results of a study of children’s reading growth during the primary grades. They placed special emphasis on questions about whether different core curricula result in different rates of growth and whether students from lower-SES backgrounds achieve more under one or another curriculum. Although the results of the study do not provide conclusive evidence that any one curricula trumps all others, they give glimpses of programs’ different effects.

In their study, Crowe et al. examined growth in “oral reading fluency” for 30,000 students in Florida (US) receiving instruction using six different core reading curricula during 1st-3rd grades. Generally, they found that almost 3/4ths of the variation in students’ scores was attributable to child factors, but the 1/4th attributable to other factors included differences in the curricula they experienced. They also found, of course, that children’s reading performance, as measured in words read correctly per minute, increased over the grades; however, the increases began to slow late in 3rd grade. In addition, they reported that students from lower-SES backgrounds had lower reading rates than their advantaged peers, but that curricula did not produce different rates of growth for low- versus high-SES students.
Continue reading ‘Do reading curricula make a difference?’