Teach Effectively!

TE readers will likely recognize acronyms such as BEE, CSRQ, EEPI, and WWC from previous post on this site. In addition to having been cited in posts, they (along with the Campbell Collaboration) are listed among the Web resources in a sidebar. They are there of a purpose: These sites are important sources of reasoned consideration of the evidentiary basis for teaching effectively.

Regular readers will also recall that from time to time I’ve posted concerns about the evidence that some of these sources have provided. These review houses employ different procedures in their integrations of evidence, and those differences are important to note. Though my concerns may sometimes be strongly stated, they do not negate the general good work that these organizations have done. Just as two cars with different features may both provide many miles of safe transportation, so can these projects provide very helpful guidance for improving education. Indeed, the folks who work at these places are far more capable than I in conducting and reporting reviews of research about educational matters.

Outside of my evaluation of the differences among the approaches taken in these groups’ reviews, there are also analyses of the differences by others who have much greater expertise. The January-February 2008 issue of Educational Researcher includes and article by Bob Slavin the provides just such an analysis. In it, Professor Slavin provides a very timely and coherent commentary on how differences in the methods employed in research syntheses (what others might call “integrative literature reviews,” “meta-analyses,” and etc.) of program evaluation studies affect their validity and utility.

Syntheses of research on educational programs have taken on increasing policy importance. Procedures for performing such syntheses must therefore produce reliable, unbiased, and meaningful information on the strength of evidence behind each program. Because evaluations of any given program are few in number, syntheses of program evaluations must focus on minimizing bias in reviews of each study. This article discusses key issues in the conduct of program evaluation syntheses: requirements for research design, sample size, adjustments for pretest differences, duration, and use of unbiased outcome measures. It also discusses the need to balance factors such as research designs, effect sizes, and numbers of studies in rating the overall strength of evidence supporting each program.

I encourage folks to read it. For the lay reader, there are some inevitably technical sections, to be sure; but overall the exposition is quite clear and accessible. For the professional, this is a valuable summary of current concerns and a hillock from which one can scan the unexplored territory before striking out to explore.

Slavin, R. E. (2008). What works? Issues in synthesizing educational program evaluations: Perspectives on evidence-based research in education. Educational Researcher, 37, 5–14.