Teach Effectively!

In January of 2005, I participated in a tele-conference with state education agency (SEA) administrators from several states. The folks who mediated the tele-conference, the K8Access Center, produced a little PDF about the chat which one can download here. It has this awful image of my staring at my Web camera.

Congratulations to some of the many teachers who have finished or are finishing their school years:

• Dick at The Life that Chose Me;
• Jill at LiquidWaffleGirl:
• Lisa at Letters from Lisa;
• Maria at Teacher SOL;
• Miss Dennis at Your Mama’s Mad Tedious:
• Mrs. Ris at MentorMatters;
• Ms. Frizzle at MsFrizzle;
• QueenAnne at QueenAnneLace.

I’m sure I missed someone who’s teaching and blogging. Write me at JohnL at Virginia.edu if you’d like me to follow your site.

One of my long-standing concerns has been the pass that the Council for Exceptional Children (CEC) gives to unverified methods of teaching. I can understand that the need for funds makes it nearly necessary to run advertisements in print publications and rent floor space at the international conference to people selling products, regardless of whether those products are evidence-based. But I have reservations about giving play to undocumented methods, practices, procedures, and such in other venues.

Thus, I was greatly encouraged when CEC’s newly revised Web site included a section on evidence-based practices. My enthusiasm dropped, however, when I got the latest issue of CEC Today, an e-mail news letter pointing readers to parts of the Web site. The first pointer aimed at evidence-based practices (HOORAY!), but just a tad down the page was a pointer to something for which I know of no effectiveness research (sheeesh)!

Evidence-Based Practice—Wanted, Needed, and Hard to Get
While the law requires teachers to use evidence-based practices in their classrooms, the field has not yet determined criteria for evidence based practice nor whether special education has a solid foundation of evidence-based practices. Also, those teaching strategies that have been researched are difficult for teachers to access.

…snip…

Visible Thinking – Engaged Students, In-Depth Learning, Better Teaching
Visible thinking engages students in substantive thinking and gives teachers insights into their students’ thought processes. This enables teachers to discover students’ incorrect perceptions or knowledge while promoting deeper content learning.

Link to CEC Today.

The squeeze is on! Here’s how it works in education: (a) Jack up the standards and (b) cut back the resources.

I’m glad to see efforts to develop and test student learning using high standards. It’s axiomatic to me that unless we have rigorous measuring sticks, it’s nigh-on impossible to assess whether different teaching methods are producing desirable outcomes. So I favor high standards, with this caveat: There must be adjustments made for individuals with disabilities. (This is a variation on the one-size argument.)

It’s also axiomatic to me that meeting high standards is going to require more than  existing budgets, more than repurposing funds from some less-than-essential activities to teaching effectively. Implementing the successful school reform models described in the report of the Center for School Reform Quality requires funding. Success doesn’t come for free.

So, I become concerned when the overseeing governmental groups cut funds from education. According to Chris Kahn of the South Florida Sun-Sentinel, Florida (US) is reducing funding (link):

Florida cut $2.4 million this year from special education programs in Broward County for children with severe disorders, and the U.S. Department of Education plans to slash an additional half million from the county’s special education budget for the 2006-2007 school year, according to Broward education officials.

The budget proposed for 2006-07 by the current administration included cuts to education programs. Although the relevant House committees are apparently only cutting a couple of programs (e.g., higher education demonstration programs for students with disabilities), those that are not being cut are staying at the same level as last year. So, the net result, given inflation, is a cut. Plus, the federal government is still not providing close to the level of support (40%) the laws declare should be provided.

Editorial comment: Aaargh.

I’m assembling a list of sites that either advocate evidence-based practice or catalog evidence-based practices (E-bP) related to teaching, special education, and related concerns. (Many sites are likely to do both, so I may have to recast the types of sources after assembling a list.) I’m not planning to catalog sites from medicine, though psychiatry or other disciplines that would address the needs of atypical students would be included. There is a broader effort at “Netting the Evidence” from the UK University of Sheffield School of Health and Related Research and the health sciences folks at my own place of employ have helpful directions for those seeking information about evidence-based practice in clinical medicine, Navigating the Maze.

Here are a few examples of what I hope to assemble:

• The Report of the Coalition for Evidence-Based Policy on “Bringing Evidence-Driven Progress To Education” is available as a PDF file.
• US Department of Education resources:
  • Evidence-Based Education (October 2002): Russ Whitehurst’s presentation on evidence-based education in HTML, PowerPoint, or PDF.
  • Identifying and Implementing Educational Practices Supported by Rigorous Evidence (December 2003): A user-friendly guide to help educators distinguish practices supported by rigorous evidence from those that are not .
  • Scientifically Based Research (February 2002): This is a transcript of experts discussin the meaning of “scientifically based research.”
• The US-supported What Works Clearinghouse provides reports about effectiveness of educational methods, practices, and policies, which I’ve mentioned (in chornological order) here, here, here, and here; it’s been slow starting, but it is still promising.
• The Center for Evidence-Based Practice: Young Children with Challenging Behavior aims to raise the awareness and implementation of positive, evidence-based practices and to build an enhanced and more accessible database to support those practices
• The Promising Practices Network (PPN) is a group of individuals and organizations who are dedicated to providing quality evidence-based information about what works to improve the lives of children, families, and communities.
• The American Academy of Child and Adolescent Psychiatry maintains an index of articles from its publication, the Journal of the American Academy of Child and Adolescent Psychiatry, that describe E-bP.
• Bob Slavin’s notes on E-bP for children at risk of later problems in schools (PDF of full comments linked there).
• New Zealand’s “Enhancing Effective Practice in Special Education” (EEPiSE) project is part of an initiative to support and develop teachers’ ability to support learners who require significant adaptations to curricula.

In what’s turning into a recurring theme (should it become a category?), I came across another reference to brain-based education. This one, however, has an appropriate dose of doubt. It’s a brief paper about the brain and reading by Sebastian Wren. Mr. Wren, once of the Southeast Educational Development Lab, has the document available as a PDF; go to his entry on the brain on his Web site, Balanced Reading. Previous Teach Effectively! posts about brain-based education here, here, and here.

Direct Instruction and Success for All models of school reform have better effects on elementary students’ academic outcomes than other models of reform according to a report issued by the Comprehensive School Reform Quality (CSRQ) Center. The CSRQ Center published the report in November 2005 that provides the results of a examination of 22 models for comprehensive school reform. The examination was based on ratings about the extent to which the models met criteria in each of five categories:

• Category 1: Evidence of positive effects on student achievement, including diverse populations;
• Category 2: Evidence of positive effects on additional outcomes (e.g., discipline, attendance, teacher satisfaction, etc.);
• Category 3: Evidence of positive effects on parent, family and community involvement;
• Category 4: Evidence of a link between research and the model’s design; and
• Category 5: Evidence of services and support to school to enable successful implementation.

The center developed ratings based on publically available evidence. Models for which they found multiple examples of rigorous research yeilding consistently positive and relatively strong outcomes got higher ratings than those for which the evidence was weaker, less consistent, or less positive. The ratings were (a) very strong, (b) moderately strong, (c) moderate, (d) limited, (e) zero (evidence available, but no benefits found), (f) negative, and (g) no rating; the “negative” rating was only used for quantitative data, not for qualitative assessments. There are nine pages of detail about the methods used to develop these ratings.

The 22 models evaluated using these methods include (abbreviations are mine): Accelerated Schools PLUS (AS), America’s Choice School Design (AC), ATLAS Communities (ATLAS), Breakthrough to Literacy (BL), Coalition of Essential Schools (CES), Community of Learning (CL), Comprehensive Early Literacy Learning (CELL), Co-nect (CT), Core Knowledge (CK), Different Ways of Knowing (DWK), Direct Instruction (DI), Expeditionary Learning (EL), First Steps (FS), Integrated Thematic Instruction (ITI), Literacy Collaborative (LC), Modern Red SchoolHouse (MRSH), National Writing Project (NWP), Onward to Excellence II (OE), School Development Program (SDP), School Renaissance (SR), Success for All (SFA), and Ventures Initiative Focus System (VI). For each, the report provides an easily understood chart providing a simple description of the model, the grade levels for which it is used, how many schools use it, the costs of using it, and the evidence of meeting the criteria described in the bullets here. The charts for each model are followed by a detailed recounting of the basis for the ratings.

Here’s a simple summary, predicated on the rating system. I looked through the outcomes of the review and noted which models (by abbreviation) had ratings of what level on the criteria I consider most important. If there was no rating on a criterion, I omitted the initials from that row of the table. Within cells, models are listed alphabetically; that is, the order in which I entered them into the table does not imply higher or lower ratings.

My conclusions after reading this report? First, I’m not surpirsed by the results of the evaluation of overall positive outcomes. Others who have looked at this evidence have found pretty much the same thing; take a look, for example at the American Federation of Teachers’ “What Works: Six Promising Schoolwide Reform Programs,” (download PDF) for example. Direct Instruction and Success for All have strong track records and this report recognizes them; if others are effective, they haven’t shown it.

Second, it’s clear that there is only limited evidence on the other objective criteria, effects on additional outcomes and on parents, families, and communities. I consider those outcomes less important, but they are worth considering. Others may consider them more important than students’ outcomes…personal priorities.

Third, the link-to-research criterion is understandable in that it is relatively easy for models’ advocates to document connections to research. However, the nature of the research linkage is important to me. In my opinion, the research that matters is the research reflected in the overall outcomes criterion. A secondary level of important research includes process studies, those that a model’s advocate ought to conduct to examine what parts of the model work and which need refinement, etc. I hardly care whether a model can be connected to developmental research, etc.

This report is worth reading. I’ll be passing it along to the school board members in my neighborhood. Get your own copy from this download link. Full disclosure: I conducted some of the research the report reviewed by the staff on this project; my studies were not used, however, probably because they did not have long-enough time lines.

The CSRQ Center is is operated by the American Institutes for Research (AIR; Washington, DC, US) and funded by a Comprehensive School Reform Quality Initiative Grant fromthe US Department of Education’s Office of Elementary and Secondary Education. The CSRQ Center’s mission is to assist educators and policymakers in choosing school reforms that will improve student achievement and other important outcomes by helping identify reforms that are reliably effective. Flash of the electrons to D-Ed-Reckoning for pointing me to the report.

Over on Ms. Frizzle, Ms. Frizzle has an entry about a mandatory in-service session on violence prevention. She expresses surpise at learning that the workshop was not about preventing violence between students, but about preventing violence toward teachers and then goes on to describe the session. The description reminded me of many in-service sessions I’ve seen; they are so often a combination of entertainment and…and…clich´, Banality? Emptiness? Pablum? Mz. Frizzle’s report is a good quick read.