“Our teachers think they’re all effective.”

According to Stannis Steinbeck, principal of Broadus Elementary School in Pacoima (CA, US), this is the view of the members of her faculty. According to data about the teachers’ effects on student achievement, not all teachers are effective. It should come as no surprise that some are more effective than others and some are woefully ineffective.

Jason Felch, Jason Song, and Doug Smith of the Los Angeles Times aggregated achievement test data over seven years and across many students assigned to 6000 teachers to assess which teachers consistently improved and which consistently diminished their students’ outcomes.

The fifth-graders at Broadous Elementary School come from the same world — the poorest corner of the San Fernando Valley, a Pacoima neighborhood framed by two freeways where some have lost friends to the stray bullets of rival gangs.

Many are the sons and daughters of Latino immigrants who never finished high school, hard-working parents who keep a respectful distance and trust educators to do what’s best.

The students study the same lessons. They are often on the same chapter of the same book.

Yet year after year, one fifth-grade class learns far more than the other down the hall. The difference has almost nothing to do with the size of the class, the students or their parents.

It’s their teachers.

In any normal classroom over the course of a year, a teacher’s effects are bound to vary. That teacher may not “connect” (hedging here) with every student but another find the teacher to be her most inspiring, one student may simply have a “bad year” and another have a “learning spurt,” a particular clique may have a negative (or positive) effect on each other, etc.; given sufficient data, these differences balance.

By examining outcomes at such a large scale, identifying relatively more and less effective teachers becomes a more objective matter than the usual means for assessing effectiveness. To be sure, because the approach as implemented in this analysis depends exclusively on standardized achievement tests, it is not foolproof. It does, however, help to sharpen the focus on students’ outcomes as an important metric and clarify the importance of examing the methods employed by those teachers whose students consistently out- or under-perform.

 

To illustrate the analysis, I created two simple images using hypothetical data. In both cases, I assumed that two different teachers taught an average of 25 students per year and that I had data for four years for both teachers. In addition, I assumed that at the end of the previous school year the 100 students’ scores had averaged around the 50th percentile with a standard deviation of 7.5 percentile points and that the scores were normally distributed. I then manufactured sets of dramatically higher and lower outcomes. In the former case, the teacher’s effect is to move the students from the 50th to the 60th percentile; in the latter, the movement goes from the 50th to the 40th percentile. I bet readers will be able to tell which is which quite easily.

To which teacher’s classroom would you like to have your daughter or son assigned?

In my view, it is critical that educators and the public get past the subjective interpretation of who teaches effectively and what practices (procedures, methods) are effective. These judgments must be made on the basis of verifiable evidence. I do not consider standardized test scores to be the only relevant metric, but where they can be used in responsible and enlightening ways, we should use them.

To read the entire article by Mr. Felch and colleagues, please see “GRADING THE TEACHERS: Who’s teaching L.A.’s kids?.” The Times promises a series of articles about the data its reporters have collected. I’ll be watching and I hope readers will be, too.