Teach Effectively!

Jennifer Jennings and Andrew Beveridge reported that exempting students from tests, a controversial practice sometimes employed with students with disabilities, may have deleterious effects on the performance of younger students with disabilities. Here’s the abstract:

Analyzing data from a large urban district in Texas, this study examines how high-stakes test exemptions alter officially reported scores and asks whether test exemption has implications for the academic achievement of special education students. Test exemption inflated overall passing rates but especially affected the passing rates of African American and Hispanic students because these students were more likely to be exempted. Furthermore, our results suggest that tested special education students in Grades 3 through 8 performed better academically than they would have if they were not tested. However, taking the high-stakes test provided no academic benefit to special education students in Grades 9 through 11.

I rarely work on topics related to high-stakes testing, so I am not well-enough informed to comment on this paper; however, I thought it was interesting enough to merit mention here. What do readers make of this finding?

Jennings, J. L., & Beveridge, A. A. (2009). How does test exemption affect schools’ and students’ academic performance? Educational Evaluation and Policy Analysis, 31, 153-175.

E. D. Hirsch, author of several books worthy of mention, provided a column for the page opposite the editorial page of the New York Times on 22 March to which I’d like to call attention. Under the headline “Reading Test Dummies,” Professor Hirsch argues that the problem with contemporary, high-stakes tests isn’t that they test knowledge, but that they test the wrong knowledge.

Professor Hirsch leads by quoting President B. Obama’s expression of concern for developing assessments that “don’t simply measure whether students can fill in a bubble on on test.” Then he suggests that instead of discarding the tests, we should change their content.

These much maligned, fill-in-the-bubble reading tests are technically among the most reliable and valid tests available. The problem is that the reading passages used in these tests are random. They are not aligned with explicit grade-by-grade content standards. Children are asked to read and then answer multiple-choice questions about such topics as taking a hike in the Appalachians even though they’ve never left the sidewalks of New York, nor studied the Appalachians in school.

Continue reading ‘Hirsch on reading tests’

Time graphic

To what extent does a US state’s high-stakes test correspond with a consistent standard assessment? Using data from the Education Trust and the Colorado (US) Department of Education Time, magazine’s Feilding Cage (with help from Jackson Dykman) created one of those nifty Flashy things that provides an interactive means of displaying data. The interactive map, linked from the thumbnail shown here, shows the degree to which states’ reading and math assessments differ from the results of the National Assessment of Educational Progress (NAEP), the standard US metric for achievement.

The results from fewer than 5% of the states’ tests (~2*50) tests are higher than the NAEP. Mostly, it is far easier to meet standards on the state tests. In the image that I’ve clipped here, one can see that fewer students are judged proficient on South Carolina’s 8th-grade math assessment than on the NAEP. Thus the South Carolina test is more demanding than the NAEP.

To compare across states (i.e., to see where SC’s students stand nationally), one would need to have a consistent measuring stick. The NAEP provides such. States’ high-stakes tests clearly do not.

Because of the way that NAEP scores are developed, this comparison is not based on the scores of the same student on each test, but rather, on an overall comparison. There are lots of technical ins and outs about these comparisons, but the big picture is informative.

I’m attending a meeting of the federal Reading First Advisory Committee and, as serendipity would have it, I stumbled on this article. We were just discussing the problems we’re encountering with not-comparable measures of reading progress across the states; as a result, we’re having a difficult time assessing the effects that Reading First is having on children’s outcomes.

To learn more about the Education Trust, visit the organization’s site. To view another interactive map that allows one to download a detailed comparison of each individual state’s testing, click here. To download a copy of a relevant scholaraly paper from Practical Assessment, Research & Evaluation by Bert Stoneberg, look here.

Flash of the electrons to Maya Frost for her post that alerted me to the Time article.