Perhaps my last post needs some clarification. I’ve run into more than one smart person who argues that all kids can, at least in principle, be high achievers. Why did I suggest that the idea attributed to Michelle Rhee is a statistical impossibility?
I think sometimes people are confused about the statistics of high achievement for two reasons. First, “all” must be taken literally to mean each and every student in school. Second, “high” is an arbitrary designation in a statistical distribution. Still, implicit in some federal laws and explicit in some people’s arguments, is that all students can be high achievers or meet some absolute standard of proficiency. But “universal proficiency” (actually, any designation of a universal achievement standard above the lowest in a distribution) is an oxymoron unless statistics just don’t apply to academic achievement (Rothstein, Jacobsen, & Wilder, 2006; see also Ho, 2008).
“All” used to encompass only a large subset of the population, but since the enactment of federal special education law in 1975 (now IDEA) “all” must include children with disabilities. Fine distinctions must be made in declaring children alive or dead and in judging them to be conscious or unconscious. And I have argued (see Kauffman & Krouse, 1981) that we should make the excruciating fine distinction between children who are educable and those who are not. Of course, we could argue ad nauseam about just what mental retardation is and the determination of different levels of it, even though most of us believe that it (MR) exists and that there are different degrees of it. But the fact is that federal education law does not allow (and I don’t think it should) schools to ignore children who can be taught important things but have cognitive disabilities—or can’t, because of any disability, achieve on average like those without disabilities.
Advocates for children with disabilities just don’t take kindly to the exclusion of students with disabilities from “all.” And I don’t think they should. They argue that the “full house” to which Stephen J. Gould referred in his discussion of statistical distributions (see Gould, 1996) has to be considered. Of course, someone could respond, “Well, I obviously didn’t mean to include students with mental retardation when I said ‘all’.” OK. The former meaning of “all” excluded some. But we’re still stuck with the statistical distribution of the rest of the students, even if we cut off and throw away (in our consideration of “all” or “high”) the left tail in a distribution of achievement that includes them. Problem not solved, even if the “obvious” cases who can’t be expected to meet a standard are excluded.
And we’re left with the statistical designation of “high” achiever. Why is everyone’s being a “high achiever” still statistically impossible? Let’s begin at the beginning.
Should we measure achievement to determine what’s “high?” You might argue that we shouldn’t. OK. Then someone’s achievement is “high” because you say so. I suppose that for those who reject the idea of measurement that’s just great! If we declare “high” achievement without measurement, we need go no further. Case closed. But, then, we’re after statistical possibility here, which does imply measurement.
Can we measure without getting a distribution of what we measure? As far as I know (and I suppose I could be proved wrong in my assumption), the only way to avoid a distribution is to measure so imprecisely that we end up with only one or a very small number of categories (e.g., high; high/not high; low/medium/high; not proficient/proficient/proficient +/extremely high) or don’t measure at all. OK. But we’re still stuck with the idea that everyone can—at least in principle—be judged to fall into a single category: “high.” Not likely at all, on a statistical basis, if your measurement is reliable and valid and if you measure a lot of individuals. Based on what I think I understand about statistics and probabilities, I’d say the probability of everyone’s falling into the same category is so remote that anyone would be wise to bet everything she or he owns against it if the sample is large (let’s say 1,000 or more) and the measurement of achievement is worth a hoot.
Should we use some sort of standardized test of achievement? Well, most of the talk of high achievement and accountability and gaps and so on is based on students’ scores on such tests. So, you may damn the tests we have, or you may come up with a better one, but still I’m supposing that the scores on whatever test is given would have a considerable range and that if you give it to a large and randomly selected group of students (say, all of the students in a medium-sized school district or more) you’d get something approximating the mathematically idealized “normal” distribution. That’s just because I’m assuming (perhaps falsely) that achievement is “normally” distributed. But, even if it isn’t, there are other statistical considerations that are important.
As far as I know, any distribution would have the four statistical “moments” to which mathematicians and statisticians refer (i.e., central tendency, including mean, median, and mode; variability, including standard deviation; skew, negative or positive; and kurtosis, lepto or platy). As far as I know, it’d be impossible, statistically, to find that all of the students are at or above a certain point on that distribution, except the lowest one. And as far as I know, this applies to all distributions, regardless of their statistical moments.
Now, we could, it’s true, pick any point on the distribution and consider everyone above that point a “high” achiever. But, unless I just miss something about statistical realities completely, the only point at or above which everyone can score is the lowest point on the distribution. If we pick a place on the distribution below any one of the points indicating the statistical moment called central tendency, then we’re likely to set ourselves up for ridicule (one reason Garrison Keillor’s Lake Wobegon is obviously fictional and makes people laugh is the realization that all of the students being above average is impossible).
So, I’m left wondering what I’ve missed about what some laws and some bright people presume about academic achievement. I’ve suggested that eliminating all statistical gaps in achievement among groups makes about as much sense as waving to Ray Charles (Kauffman, 2005, in press). Maybe the analogy is flawed. Maybe it made (or still makes) sense to wave to Ray Charles. But I doubt it.
Someone might argue that it’s not an achievement test score itself that defines “high” but a gain score—a comparison of what a student has achieved to where the student started. As far as I know, gain scores will have a statistical distribution, too, and so we’re right back where we started. Or someone might say that it’s not really a test score or a measure of gain but whether the student learns all that he or she can that should define “high.” But, again, we’d have to have some way of judging (or measuring?) what a student can or can’t learn, and I think it’s a pretty safe bet that not all students can learn all things at the same level, so we’re again back where we started.
And, then, there’s the argument that we really just want to get more kids to achieve at higher levels so that the whole curve moves up—so that the central tendency is so much higher that what used to be average is now “high.” Problem not solved for two reasons. First, we could just compare today’s distribution to a distribution of long ago; we always want to make comparisons to current data. And we’d have some obvious statistical trouble even if we compared current to old data. Second, I don’t think it’s statistically possible to detach the low end of the distribution from the lowest score. Besides, we need to ask what will happen to the shape of the distribution of achievement scores if we move its central tendency higher (but that’s a question different from the statistical possibility of every student’s being judged “high” in achievement).
I’m all for improving education, including its outcomes. I think that’s possible, statistically and otherwise. I do think it’d be very difficult to achieve a system of education in which all children learn all they can, although it might be possible. But all children being “high” in achievement? I doubt that it’s possible statistically. Maybe I just need to get over the idea that statistics apply to academic achievement, but I doubt it.
Gosh, I hope I haven’t written something I didn’t (or shouldn’t) mean! Please tell me if I have.
Gould, S. J. (1996). Full house: The spread of excellence from Plato to Darwin. New York: Three Rivers Press.
Ho, A. D. (2008). The problem with “proficiency”: Limitations of statistics and policy under No Child Left Behind. Educational Researcher 37, 351-360.
Kauffman, J. M. (2005). Waving to Ray Charles: Missing the meaning of disability. Phi Delta Kappan, 86, 520-521, 524.
Kauffman, J. M. (in press). The tragicomedy of public education: Laughing, crying, thinking, fixing. Verona, WI: Attainment.
Kauffman, J. M., & Krouse, J. (1981). The cult of educability: Searching for the substance of things hoped for, the evidence of things not seen. Analysis and Intervention in Developmental Disabilities, 1, 53-60.
Rothstein, R., Jacobsen, R., & Wilder, T. (2006, November). “Proficiency for all”—An oxymoron. Paper presented at a symposium on “Examining America’s commitment to closing achievement gaps: NCLB and its alternatives.” New York: Teachers College, Columbia University.
Recent Comments