Jeff Bryant
Published: Sunday 20 January 2013
Schoolteachers, principals, and parents on the ground have long understood that enforcing education practices on the basis of government mandated metrics is a losing proposition for children.

Why Progressives Should Care About the Backlash on Standardized Testing

Article image

Last week, celebrated statistics guru Nate Silver laid a bugger on advocates for test-driven education, the current policy fad that enamors Republicans and Democrats alike who fancy themselves as “education reformers.”

In an online conversation at the aggregator site Reddit, the man known for being the “Lord and God of Algorithms” was asked, “What are your thoughts on data-driven metrics for teacher evaluation? Do you think a system that accurately reflects teacher value could ever be created, or will it always be plagued by perverse incentives (teaching to the test, neglecting certain types of students, etc)?”

Silver replied, “There are certainly cases where applying objective measures badly is worse than not applying them at all, and education may well be one of those.”

Silver explained that it would take “a book- or thesis-length treatment to really evaluate properly”

Of course, our current crop of leaders in Washington, DC and state capitals are not about to wait for “a book- or thesis-length treatment.” Because they are imbued with the “fierce urgency of now” for “education reform,” they want a formula instead.

But schoolteachers, principals, and parents on the ground have long understood that enforcing education practices on the basis of government mandated metrics is a losing proposition for children. Now they’re speaking out.

What’s Brewing In Seattle?

Since the creation of No Child Left Behind, the imposed metrics driving education policy have been student scores on standardized tests. Schools not making Acceptable Yearly Progress on raising test score results for specific populations of students have been subjected to all kinds of punitive actions, which include being shut down or turned over to a private management firm.

The Obama administration has intensified the situation. Its grant programs –including Race to the Top – and NCLB Waivers all require schools to base teacher and principal evaluations, as well as school rating systems, on student test scores, to a great deal of extent.

The intent of these policies was to impose “measured progress.” But lots of educators, parents, and public school activists don’t see it that way.

Recently, teachers at a Seattle high school refused to give the district-required MAP tests to students, saying the tests are bad and waste time and resources. Amazingly, people rushed to the schoolteachers’ support.

One of the teachers opting out of the test, Jesse Hagopian, reports at the online magazine TruthOut, “Thousands of people from around the country have signed on to a petition supporting the Garfield teachers. The school’s PTSA and student body organization  have stood behind the teachers. Other schools in the district are starting to line up behind Garfield, too, starting with Ballard High.”

More recently, the Seattle Education Association called for an elimination of the testing regime, calling for funding of the tests “to go to classroom and student needs first.”

At the online news outlet NationofChange, Hagopian explains the teachers’ rationale:

“To use this (the test) as a tool to evaluate our teaching makes no sense . . . “They’re setting us up for failure. And Garfield High School is not a failure. We’re the home of (former students) Jimi Hendrix, Bruce Lee and Quincy Jones . . . No one cares how Jimi Hendrix scored on a high school math test. And no one should.”

This isn’t happening in just Seattle.

Testing Backlash Grows, Strengthens

Reporting from her blog at The Washington Post, education journalist Valerie Strauss traced the arc of growing unrest over metric-driven education policy:

“Parents have started to opt out of having their children take the exams; school boards have approved resolutions calling for an end to test-based accountability systems; thousands of people have signed a national resolution protesting high-stakes tests; superintendents have spoken out, and so have teachers. It has been building momentum in the last year, since Robert Scott, then the commissioner of education in Texas, said publicly that the mentality that standardized testing is the ‘end-all, be-all’ is a ‘perversion’ of what a quality education should be.”

Also at Strauss’ blog, Lisa Guisbond, a policy analyst for the National Center for Fair and Open Testing, known as FairTest, reviewed how Scott’s outcry evolved into a meme traveling to other states, including Pennsylvania, New York, Florida, Oklahoma, Ohio, and Virginia.

Both Democrats and Republicans are taking action. California’s school chief Tom Torlakson, a Democrat, recently proposed reducing the number of standardized tests that students must take. And the Republican dominated House of the Texas legislature has zeroed-out the state’s budget for standardized testing.

What People Are Upset About

A big problem with test-obsession is that basing education policies primarily on test scores has no basis in research, and evaluating teachers by students’ test scores – sometimes, unbelievably, students they don’t even teach – is a particularly bad idea.

As the Liz Dwyer at the website good.is recently explained, “individuals and organizations have laid out the case against the practice pretty thoroughly.”

Dwyer quotes a letter written by the National Research Council to the Obama administration “warning them against including the policy in their Race to the Top reform agenda,” noting that “research does not support the practice.” She also quotes Richard Rothstein, a research associate at the nonprofit Economic Policy Institute, who wrote that “holding teacher accountable for growth in the test scores (called value-added) of their students is more harmful than helpful to children’s educations.

The practice, Rothstein wrote, “creates rational incentives for teachers and schools to narrow the curriculum to tested subjects, and to tested areas within those subjects. Students lose instruction in history, the sciences, the arts, music, and physical education, and teachers focus less on development of children’s non-cognitive behaviors—cooperative activities, character, social skills—that are among the most important aims of a solid education.”

A Teacher’s-Eye View

Classroom teacher and prolific blogger Kenneth Bernstein recently described exactly how the “rational incentive” Rothstein pointed to works in the classroom. In a post titled “Warnings from the Trenches,” at the website for the American Association of University Professors, Bernstein warned university faculty to expect high school graduates to be “unprepared for higher education.” The reason? Test-driven “reforms” stemming from NCLB and Race to the Top.

Bernstein’s first complaint is that because only two subjects – math and reading– are being tested, “anything not being tested was given short shrift.”

Bernstein also explained how test-driven school practices are dumbing down students’ writing skills. Because the tests are made up of multiple-choice items – which, he notes “are cheaper to develop, administer, and score” – the tests don’t demand “higher-level thinking” or even “proper grammar, usage, syntax, and structure.”

Even in more advanced courses such as AP, which Bernstein taught in addition to his regular classes, basing student performance solely or even substantially on mass-produced tests – in this case, AP exams and ostensibly “more rigorous” than state tests – enforces an inferior level of education. Because the exams are constructed with questions graded by a rubric that is, according to Bernstein, “concerned primarily with content and, to a lesser degree, argument,” with “no consideration of grammar or rhetoric,” or fundamentals of good composition such as “a clear topic sentence and a proper conclusion,” students get no credit for the quality of their writing.

“If, as a teacher, you want your students to do their best, you have to have them practice what is effectively bad writing,” Bernstein concluded. “My students did well on those questions because we practiced bad writing.”

Bernstein and his colleagues are exhorted by reform enthusiasts like Education Secretary Arne Duncan to not to “teach to the test” and still strive for learning goals that include higher-order skills. But “high schools are forced to focus on preparing students for tests,” Bernstein maintained, “and that leads to a narrowing of what we can accomplish in our classrooms.”

Is Resistance Futile?

The resistance to the test-driven approaches to education is getting some left-leaning people worried. At Slate.com, Matt Yglesias fretted that “backing off” from test-driven education “sounds like a mistake.” He pointed to Texas, where much of the test backlash is taking place, as an example of the state where the “testing craze . . . seems to be working out well.” (emphasis original)

His evidence was more test data (what else), in this case, the most recent (presumably) results from the main National Assessment of Education Progress. Texas’ NAEP scores show that African-American and Latino students perform better than the national average on the 8th grade and math. With this statistic only, he declared, “Those are pretty good results!” because Texas “is unusually stingy of its funding of public schools.”

Setting aside the unsurprising conclusion that the state that “led the way in the testing craze” (his words) might perform better than average on standardized tests, the greater reality is that the test-driven policies mandated by NCLB have produced meager gains in achievement on the NAEP after so many years of intense concentration on reading and mathematics required by the law. In fact, the largest gains in the NAEP occurred before metric-driven education became the law of the land.

When “Reform” Is Really Old School

Contrary to what testing acolytes would have you believe, trusting the magic of metrics to guide education is really not something new. In a series of thoughtful posts at his website, educator Larry Cuban declared, “It’s time to question the rationale of a business model applied to education.” Cuban explained that using data in education decision making is not something new and sourced the practice back to the “scientific management movement” that started over a century ago.

Educators have used data “for decades,” Cuban recounted, to make all kinds of decisions. But “just like facts from the past do not speak for themselves and historians have to interpret those facts, neither do numbers speak for themselves.”

Instead of going by-the-numbers alone, data like test scores need to fit into “existing models” or “algorithms” that determine whether the numbers really explain something valid.

The “policy by algorithm,” as Jeff Henig, a contributor to the blog series, put it, has become “in vogue” in all sorts of policy arenas, especially education, a pursuit that can often defy clarity.

But “policy by algorithm” has a dark downside. “When data are thin, algorithms theory-bare and untested, and results tied to laws that enshrine automatic rewards and penalties,” Henig explained, the algorithm can become indifferent to “the specific processes that link interventions to outcomes.” In order to compensate for the problem, algorithms have to change. And whereas Google can make 500 changes a year to its algorithm, we certainly can’t expect that from national education policy.

What education policy by algorithm is leading to, Cuban predicted in a third post, is quite probably similar to what happened to the collapse of the housing market in 2008, when “all the finely-crafted algorithms available to hedge fund CEOs, investment bankers, and Federal Reserve officials” were no help in predicting an economic fiasco.

What’s needed, Cuban concluded, is much more transparency and straightforward communication about the nature of education models and much more reliance on the good judgments of professional educators and parents.

There are reasons progressives should care about this.

Wasn’t Education “Reform” Supposed To Be Progressive?

NCLB was originally sold to us as “progressive” legislation. Miraculously, now NCLB Waivers are being touted as “progressive” too.

Armed with the reams of testing data unleashed by metrics-driven school reform, progressives everywhere were going to have the information they needed to hold schools “accountable” for educating children, especially the least served.

Yet what we are seeing instead is a form of education that actually threatens students’ civil rights. Writing at the blogsite Daily Kos, education professor Sherman Dorn explained what test-driven education is resulting in:

“When schools with low academic achievement receive test-prep booklets, the cost of those purchases is stolen from instructional materials for the general curriculum. When children with low academic performance find their classroom time occupied by activities that mirror multiple-choice test formats, that is a denial of access to a broad curriculum. When teachers, aides, school counselors, and others spend hours in early spring drilling students on test-taking techniques, that is time that children are not reading, are not learning about math and science and history, and are not experiencing or creating art or music.”

Does that sound progressive to you?



Author pic
ABOUT Jeff Bryant

Jeff Bryant is a Marketing and Communications Consultant for Nonprofits. He is a marketing and creative strategist with nearly 30 years of experience – the past 20 on his own – as a freelance writer, consultant, and SEM provider. He's written extensively about public education policy, most recently at OpenLeft.com and NationofChange.

These comments are all good,

These comments are all good, but don't address one other important issue: the tests are biased toward success for white, middle- or upper-class students. One expert opined that one might as well substitute socioeconomic status data for student test scores, as the distribution of "good" and "bad" schools would be essentially the same. Using these tests tends to reinforce existing stereotypes about poor and minority families being less intelligent or less motivated, and also encourages more competent students whose parents have more resources at their disposal to avoid the "bad schools," thus driving them further into the "bad school" category by removing high achievers from the pool.

Perhaps, instead of trying to "measure" the unmeasurable variables, we ought to observe what is happening in schools that obviously outperform their counterparts with similar levels of funding and SES status, see what's working and try to duplicate it. Using test scores without looking at the characteristics of the students and community, the funding of the schools, or the processes that are being used is very similar to trying to eliminate waste in a budget by simply cutting the total funds available. It doesn't address the real problem, it only addresses the end manifestation of the problem.

The situation requires a nuanced analysis that no "objective" test can possibly supply.

There is an ugly underside to

There is an ugly underside to high-stakes testing that has been documented by teams of researchers, such as the one led by Prof. Will Haney at Boston College. Schools that need to boost their test scores do things like holding back a very large number of students in the ninth grade (so that they won't lower the scores of the tenth-grade class), increasing suspensions of low-scoring students during testing times, and even pushing low-scorers out of school altogether. Grade retention and suspension also increase a student's risk for dropping out. Then the schools conceal their low graduation rates by not counting the students who left before the beginning of twelfth grade. "No child left behind," indeed.

If testing is used to reward

If testing is used to reward the schools who show the greatest improvement in test scores, then by extension it would penalize schools where the students were already doing very well on tests, scoring near the top of the range and thus having little room for improvement. Test scores are only one datum in the realm of relavent info pertaining to student and teacher performance, and should only be discussed in their context, which is as unique as the school. using them for comparing schools or teachers or students is faulty logic. They can be a tool for evaluating a single student's change in performance in the very limited area of multiple choice questions on very limited subjects.

If we're going to measure

If we're going to measure success by standardized tests, then we should give all students IQ tests to have a baseline to measure. Performing well on IQ tests, such as matrices tests (pattern recognition), is a necessary condition, although not a sufficient condition, of doing well in math.

Some students will have deficiencies where others do not. We can't expect everyone to perform at the same level in all subjects. That being the case, it's unrealistic to expect a standardized test to measure true student expectations, and then derive from that test how well a teacher is doing.

In other words, a student who scores 80 IQ on a matrices test is never going to do well in math, although he or she can learn basic arithmetic skills. The reason I used 80 IQ is because that score is considered "normal" intelligence, and thus part and parcel of the standardized test results.

Of course basing performance on IQ tests alone is controversial, and that's the point--entirely.

8o is not normal IQ. 90 to

8o is not normal IQ. 90 to 110 is considered normal or average, which is probably a better word choice. Forty years ago when I started teaching, a child with an IQ of 80 was assigned to a self-contained special ed classroom with little to no contact with mainstream students. Today with the legal requirement of the least restrictive environment, that child with an IQ of 80 is assigned to a regular classroom where he/she will struggle mightily despite any supports put in place. Placing such a child in a class with "normal" children & expecting results on standardized tests equal to those of the so-called normal children is simply not fair - to the child, the teachers, or the school

DW you are sooooo right. I'm

DW you are sooooo right. I'm a teacher and I've long believed that IQ testing in the 6th or 8th grade would be so helpful, not to limit the students but to give us all some sense of where certain students might struggle and when they're choosing courses that might be a big reach for them or require more time than they have. A lot of seemingly average students have short term memory deficits and variants of dyslexia. I teach foreign language and you really see it there because they can't rely on their compensatory strategies like they can with English. A lot of students hit a wall around their junior year where subjects they used to find easy suddenly become considerably more difficult and I believe that IQ testing would help us predict that plateau. It might also give us some idea about where there block or problem lies. No one will support it though because it can be damaging to know you're just average, or worse, not even average. It would certainly identify people who will struggle with simple computational math most of their lives as well as those whose reading ability will be challenged. It would help us design better programs and even find more useful strategies for these people. A lot of people in education don't get the continuum between the child, the teen and the adult. They treat children and teens like they're some sort of separate breed of cattle. We're all in this together. The student who struggles with reading will be the adult who struggles with reading, a condition that can make you very hostile and sensitive. I wish there was a movement for this. Any chance?

I am a progressive and have

I am a progressive and have always thought putting standardized testing on a pedestal for any reason was pretty silly. Teaching to the tests is teaching for consumption of fact only. Facts are nice to have in one's repertoire, but stating a "fact" has little to do with understanding that "fact" within the context of ever-more complicated concepts. Doing that requires critical thinking skills.

I remember how I hated multiple choice tests. I always found myself selecting an answer and then appending a short essay off to the side - which must have driven my teachers nuts, but allowed me to explain my choice within a larger context. My understanding was not a function of simply short-term memory.

Certainly, I am not the smartest person on this planet, but I do believe that having the basics (math, writing, reading, science) in hand, along with critical thinking skills, gives us much of what we need to navigate an information-overloaded world. The answers are out there. We need to know HOW and WHERE to find them. And the cultural/societal permission to seek them.

Fixing the system and making it work for students may require loosening our grip on how we measure their successes. We can continue teaching to the tests and putting out the cookie-cutter version of a perfect employee or we can set free the human mind to explore whatever intrigues it by making darn sure we have the basics, critical thinking skills and access to a "world-ful" of resources.

If they want standardized

If they want standardized tests to really mean anything (though there will always be problems with them, as some students simply do or do not test well,) then they need to broaden the range of subjects included, radically change the content of the test each time it's administered, not inform the teachers or administrators in advance of what will be in the test, and not permit any sort of special preparation before it is given. Then they might actually get some idea whether the students are getting anything out of their regular curriculum or not, without compromising that curriculum to try to improve the test scores.

W. Edwards Deming, considered

W. Edwards Deming, considered by most to be the father of statistical process control, commented on this subject at least two decades ago. He thought the idea of applying SPC to education was ridiculous, since you can't control your primary inputs (students) in most settings.

Nate Silver is a statistician He uses algorithms. It is both distracting and inaccurate to refer to him as the "Lord and God of Algorithms". Algorithms are not his field of expertise.

Comment with your Facebook account



Comment with your Disqus account

Top Stories