Popham on Standardized Achievement Tests

[George Sheridan <learn@jps.net>]

California Educator is the monthly magazine of the California Teachers 
Association.

***
Judging schools is right task, but it uses the wrong test
California Educator, September 2003
Volume Eight, Issue 1
http://www.cta.org/CaliforniaEducator/v8i1/Action_8.htm

"You can't measure a temperature with a teaspoon," says testing and 
assessment expert Dr. W. James Popham. "It's the wrong tool."

Likewise, the emeritus professor of education at UCLA told teacher leaders 
at CTA's Summer Institute, you can't use standardized achievement tests to 
measure student performance. They're not designed to measure what's being 
taught or learned, but what knowledge or aptitude is being brought to the 
table. In fact, if a test item is answered correctly by too many students, 
it's taken off the test in the next revision. A teacher who does a good job 
and teaches students the material thoroughly will not be rewarded for the 
effort.

The original test was developed to identify good officer candidates among 
Army troops. It was a good test for its purpose - to create a spread of 
test-takers, spot the best/ worst, and predict how well individuals would 
perform - but the alpha model, as it was called, was "adopted lock, stock 
and norm-referenced barrel by the makers of the standardized achievement test."

What teachers want and need is "instructionally supportive tests that 
provide clear descriptions of what's tested; measure a modest number of 
curricular aims; and supply instructionally informative results." In other 
words, they need tests that give them information they can use to adjust 
their teaching and fill gaps in student knowledge in a timely fashion.

Teachers are in a predicament, said Popham. They can't complain about the 
tests too loudly. "Attempts to have the [test] replaced by instructionally 
supportive assessments will be seen as a flight from accountability."

However, teachers can arm themselves with information as to why 
standardized tests don't measure what they're supposed to and help parents 
understand how the wool is being pulled over their eyes. They need to be 
able to explain the likely impact of unsound tests, why inappropriate tests 
give schools inappropriate labels, how tests may measure what students 
bring to school rather than what they learn, and how to judge what schools 
are doing the right way. "We have to educate parents and pertinent 
policy-makers about the evaluative inappropriateness of standardized 
achievement tests. And we have to collect convincing evidence signifying 
that students are learning worthwhile things."

If something isn't done, Popham said, test prep will become even more 
excessive than it already is; schools will continue to emphasize 
test-focused drilling at the expense of the curriculum; and students who 
are poor test-takers will get the subtle message that they should not even try.

The education profession did not sound the alarm effectively enough before 
the federal government enacted the most recent version of the Elementary 
and Secondary Education Act (the so-called No Child Left Behind Act of 
2001). The federal program is using the wrong kind of test to judge 
schools. "It's our fault as a profession," said Popham. "It's time to fix it."

But before the education profession can tell others how to fix the problem, 
"we have to understand it ourselves."

Trudy Stephenson Willis

George Sheridan