Link to Original Story

The Answer Sheet
By Valerie Strauss

Rhee’s problem with D.C.’s new test scores D.C. Schools Chancellor Michelle Rhee has a problem, and it’s not the fact that elementary school standardized test scores just went down (at a bad time for Mayor Adrian Fenty, who appointed Rhee and is seeking reelection).

The problem is that she has made rising standardized test scores a central measure for achievement -- hers, students and teachers.

When test scores go up, as they very often do when a great emphasis is put on the results and teachers “teach to the test” (either consciously or subconsciously), it is easy to claim credit. School reforms are working! Yeah!

But scores invariably go down after a time, no matter who is giving them and who is taking them, and they do so for reasons that may have nothing to do with the teacher, or the student, or the schools district chief.

My colleague Bill Turque reported Tuesday that reading and math test scores declined somewhat this year in D.C. elementary schools, halting a two-year run of significant gains and dealing a setback to Rhee. Middle and high schools showed continued gains in reading and math on the District of Columbia Comprehensive Assessment System (DC CAS), administered every April. The decline, it should be said, was much less than the overall improvement in the past few years.

But a rise in scores doesn’t necessarily mean that more student learning took place, and a decline doesn’t necessarily mean that students learned less.

Researchers in the field say that it is the nature of standardized tests that they rise from year to year when the same design of a test is given in the same schools. And they go down when a new design is given, or when a different demographic of students takes the test, or a bunch of kids in a class had a cold, or... well, you get the idea.

There are too many variables that can affect the scores of a single test to make the result completely reliable, which is why scores should never be used as a sole measure for any high-stakes decision.

“Test scores going down can reflect less adequate teaching and learning but it may reflect so many other things that you can’t be sure,” said Robert Schaeffer, public education director of the nonprofit National Center for Fair & Open Testing, or FairTest. “You can’t look at one-year test scores changes.”

So, feel free to judge Rhee on her insistence on placing so much importance on test scores (as have today’s other titans of education reform, such as Joel Klein in New York City). She believes they are so important, in fact, that she recently announced that she wants to expand their use in city schools, so that, in time, every D.C. student from kindergarten through high school is regularly assessed to measure academic progress and teacher effectiveness.

But judging her reforms on the actual test scores, well, as my kids say, “Not so much.”

Rhee said she and her team would “dig into the data” to find out why the elementary reading proficiency rate, which had risen 11 percentage points from 2007 to 2009, fell 4.4 points, to 44.4 percent, and why, after rising 20 percentage points from 2007 to 2009, the elementary math proficiency rate dipped 4.6 points this year, to 43.4.

The proficiency rate is essentially a measure of the portion of students who pass the tests.

This is just one reason why the current fashion in school reform to link teacher pay to standardized test scores is so wrong. So while Rhee digs into the data, it might do her well to dig into the reasons she thinks standardized test scores are worthy of being used for high-stakes decisions.

My colleague Jay Mathews, the longtime dean of education reporters in the country who for years championed the use of test scores to rate schools, suddenly just had an epiphany. He wrote in a recent post on his Class Struggle blog:

"I have to question my own judgment and fairmindedness when I ignore--for three years!-- a report that raises important questions about the way we have been using test scores to rate schools.

"I have always been open to better ways of assessing how our children are taught. But I usually say standardized tests are the best available tool at the moment. So I am embarrassed that it took me so long to read “Keeping Accountability Systems Accountable” by Martha Foote, published in the Phi Delta Kappan education journal in January 2007.

"I am indebted to the Monty Neill, executive director of Fairtest: The National Center for Fair and Open Testing, for pointing me toward the article and its author. Foote is director of research for the New York Performance Standards Consortium, which she describes in her article as “a coalition of 28 small, diverse public high schools across New York State that exemplify education reform based on strong commitment to school-as-community, to ongoing professional development, and to innovative curricula and teaching strategies.” That sounds good to me, but it gets better."

Read the rest here, at Jay’s blog.