Teach Effectively!

Steve Graham
(at another presentation)

Live blogging here in McKim Hall at the University of Virginia as Steve Graham delivers the McGuffey Reading Center’s 25th annual Graham Lecture. After Marcia Invernizzi’s cordial introduction, Steve began with a joke and a couple of humorous anecdotes about students’ writing. Of course, he tipped his hat to his collaborators and the sponsor of the research (Carnegie’s Writing Next).

Steve went into a rationale for the importance of writing instruction (“Why do reading, math, science, and technology get all the attention?)”. He then discussed forms of research, explaining that he was going to draw on experimental and quasi-experimental research, single-subject studies, and examinations of successful teachers. In addition, he noted that when the results from studies from diverse methods align, he has increased confidence in the strength of this findings.
Continue reading ‘Graham Lecture with S. Graham’

Joanne Jacobs covered a story about a professional development consultant advising a teacher to dodge a question about whether introducing calculators will hinder students’ acquisition of basic computation skills. There’s video! With transcription by Wayne Bishop, one of Teach Effectively’s Much Admired Folx, the inanity of the consultant’s response becomes quite clear. If the presenter wasn’t so serious, it’d be humorous, sort of like a parody.

Link to Ms. Jacob’s post.

The US Supreme Court will hear arguments today in Flores v. State of Arizona, a case that captures important concerns about contemporary education in the US. Plantiff argues that English-language Learner (ELL) programs are deficient and receive inadequate funding, violating a provision of a US federal law (the Equal Educational Opportunity Act; EEOA) requiring that states ensure that students for whom English is not a first language can learn how to speak English and, thus, benefit from education.

The class-action case gets its name from Miriam Flores, an elementary student in the 1990s, who had limited English proficiency (LEP) and did not benefit from the ELL services during her primary schooling. US National Public Radio’s Nina Totenberg reported Ms. Flores recollections (she is now 22 and a student at the University of Arizona):

“It was quite a disadvantage, definitely,” Flores says. “For example, even when it comes to math, I mean problem solving, they were all in English. So in order to understand, you need to be proficient in your reading in English.”
Continue reading ‘ELLs deserve effective teaching, too’

Joanne Jacobs hosted the eighty-fourth Carnival of Education this week. Review the posts by following this link.

Using the methods of meta-analysis, Steve Graham and Dolores Perin examined research about alternative means for teaching written expression to students from fourth through twelfth grades. They limited their review to studies that assessed outcomes on measures of the quality of students’ writing. Unsuprisingly, they found that some of the methods used in teaching writing are more effective than others.

There is considerable concern that the majority of adolescents do not develop the competence in writing they need to be successful in school, the workplace, or their personal lives. A common explanation for why youngsters do not write well is that schools do not do a good job of teaching this complex skill. In an effort to identify effective instructional practices for teaching writing to adolescents, the authors conducted a meta-analysis of the writing intervention literature (Grades 4 –12), focusing their efforts on experimental and quasi-experimental studies. They located 123 documents that yielded 154 effect sizes for quality of writing. The authors calculated an average weighted effect size (presented in parentheses) for the following 11 interventions: strategy instruction (0.82), summarization (0.82), peer assistance (0.75), setting product goals (0.70), word processing (0.55), sentence combining (0.50), inquiry (0.32), prewriting activities (0.32), process writing approach (0.32), study of models (0.25), grammar instruction (– 0.32).

The basic, take-home message: Systematic and explicit instruction helps students write higher quality products than the pop-ed alternative that stress thinking, reflection, and such.
Continue reading ‘Effective methods for teaching writing’

Among others, Ed Week reported about findings from a recent study showing the beneficial impact of having adults provide reading tutoring for young children. Under the headline “Volunteer Tutors Found to Help Poor Readers,” Catherine Gewertz wrote “A program that uses older volunteers as tutors has significantly improved the reading skills of students in the early grades, according to a study released today [10 April 2009].”

The study is an evaluation conducted by Nancy Morrow-Howell and colleagues of Washington University in St. Louis (MO, US) and in collaboration with Mathematica Policy Research. In brief, the study compared the reading outcomes (and other measures, e.g., teachers’ endorsement of the program) of 825 1st, 2nd, and 3rd graders on a suite of school measures (including decoding, comprehension, vocabulary, and teacher assessments). About half of the students received tutoring about once per week for a year. The data revealed that the tutored students made statistically greater gains on some measures than those found for the students in the control group.

The study deserves both accolades and scrutiny. Although it has multiple strengths (e.g., students were assigned randomly; pre-tests showed equivalent levels of competence; numbers were fairly large), there are problems, too. Not the least of these is that the report depends upon gain scores. Because the design fits classical experimental procedures, wouldn’t it be appropriate simply to examine the outcomes after the one year of tutoring? Also, what was the intervention?

One way to think about benefits for students is to examine effect sizes. Notably, in the document by Murrow-Howell and colleagues, the reported effect sizes are based on the gain scores and they were actually pretty small (0.13 to 0.17); to get these effect sizes, they—understandably—used only students who received at least 35 sessions of tutoring. Using the data in Table 3 (p. 12) and comparing the means for the experimental and control groups at posttest (a la a classical experiment), I get the even-smaller effect sizes shown in the table at the right. (For the technically inclinded, I calculated simple d using the control SD; the authors used Hedges’ G.)

Tutoring has well-documented benefits, but small-group instruction is equally effective and clearly more efficient (Elbaum et al., 2000; Journal of Educational Psychology). So, tutoring might not be a bad thing, but could the Experience Corps get more bang for their proverbial $$ if they had tutors take groups of, say, three? What is more, we don’t really know what happened to the students in the control condition. Did they get any supplemental help? If not, how was the study controlled for the possibility that Reading Instruction Plus Something More is simply better than Reading Instruction? Mayhaps answers to these questions are provided in a more detailed report than the one I used.

Of course, volunteers would need coaching and they’d have to learn to execute pretty specific lessons, which raises the related question: What did the tutors do with the students? I quickly scoured the Experience Corps Web site looking for a curriculum or set of guiding practices, but I came up empty handed. I’ll need help with this, and perhaps a kind reader can provide it in the comments.

Now, if the tutoring program was an adaptation of model, such as the one tested by Wallach and Wallach years ago, that would be a good thing. Or, perhaps even better, if it was something predicated on 100 Easy Lessons, that might be good.

If it wasn’t, though, then we need a new study comparing the tutoring methods employed in the sites in this study (call it Method A) to tutoring methods based on some known-to-be-powerful method (call it Method X) and to extra time on reading (which may be what the students in the current study got!). Ideally, this should be crossed against small-group supplements, something like the diagram here.

To be sure, it’s always easier to critique studies than it is to run them. I’m just fearful that the press coverage of this one is going to make more of it than it merits. It’s not a bad study, but those effect sizes are dwarfed by effects of powerful instructional procedures. And, yes, I know I’m ignoring the social validity of teacher satisfaction (would anyone actually expect teachers to disparge getting extra help for the children in their charge?), but it’s students’ outcomes that matter.

Read the article by Ms. Gerwertz, the actual report, the press release, and more about Experience Corps. And, here are some of the headlines that make me recommend caution:

“Students in urban schools get big boost from pioneering tutor program
Comprehension and other critical skills improve dramatically with one-on-one help from Experience Corps’ volunteers, a new study shows”—Christian Science Monitor.

“Study finds students with Experience Corps tutors make 60% more progress in critical reading skills than students without tutors“—Washington University News and Information Office; see also, “Students With Experience Corps Tutors Make 60% More Progress In Critical Reading Skills Than Students Without Tutors“—Medical News Today>; NewsGuide.us; BioMedicine, and others.

Sources

Elbaum, B., Vaughn, S., Hughes, M. T., & Moody, S. W. (2000). How effective are one-to-one tutoring programs in reading for elementary students are risk for reading failure? A meta-analyis of the intervention research. Journal of Educational Psychology, 92, 605-619.

Wallach, M. A., & Wallach, L. (1976). Teaching all children to read. Chicago: University of Chicago Press.

News sources around the US are abuzz with how state and local education agencies will spend the influx of funds for special education that comes with the US government’s increases in IDEA funding under the stimulus plan.

• Schools getting stimulus aid, but is it too late? from AZ Central.com
• Schools’ stimulus a puzzle from the Daily Tribune – Columbia, MO
• State schools to get $118M from the News Sentinel – Knoxville, TN
• A boost for poor, special-needs kids from the Star Tribune – Minneapolis, MN
• Berkeley Unified Will Receive $2.4 Million in Stimulus Funds Next Month from the Daily Planet – Berkeley, CA

Given that these funds may be pretty fleeting (here today, gone in a couple of years?), how wise is it to invest in more teachers whom the LEAs will have to dismiss or materials that are likely to need replacement in just a few years? I’d say, “NOT!”

Why not invest in staff development, using the two-year span to ensure that virtually all teachers know how to measure progress in easy-but-rigorous ways (e.g., curriculum-based measurement), implement school-wide discipline programs, and present lessons in systematic and (dare I say it?) instructive ways?

Here are some relevant links: Research Institute on Progress Monitoring and Student Progress; School-wide Positive Behavioral Interventions & Supports.

Over on Reading Rockets Kathleen McLane has an entry about monitoring progress that’s got a good intro and some valuable links. Take a look at it. There’s also video about progress monitoring in action, a link to a Web cast featuring Roland Good, Mary Ruth Coleman, and Michael C. McKenna discussing assessment including progress monitoring, and an opportunity to ask CBM guru Lynn Fuchs questions about monitoring progress (click the dropdown menu to select Professor Fuchs as the target of your question).