Fwd: [ARN-state] Standardized Tests Don't Make the Grade

[Susan Harman <susanharman@igc.org>]

Begin forwarded message:

> From: Bob Schaeffer <bobschaeffer@earthlink.net>
> Date: Sun Mar 19, 2006  9:07:10 AM US/Pacific
> To: ARN Main List <arn-l@interversity.org>,  arn2-strategy 
> <arn2-strategy@yahoogroups.com>, arn state <ARN-state@yahoogroups.com>
> Subject: [ARN-state] Standardized Tests Don't Make the Grade
> Reply-To: ARN-state@yahoogroups.com
>
> STANDARDIZED TESTS DON'T ALWAYS MAKE THE GRADE
> RECENT SNAFUS IN SAT SCORING HAVE BROUGHT ATTENTION TO THE INHERENT
> FALLIBILITY OF SUCH EXAMS
> Los Angels Times -- March 19, 2006
> by Stuart Silverstein
>
> High school seniors and their parents, along with college counselors 
> and
> admissions officers, have been roiled by recent disclosures that at
> least 4,600 SAT exams taken in October were graded incorrectly.
>
> But education testing experts point out that similar mistakes have
> happened before — and inevitably will happen again.
>
> "Tests are a fallible product," said Kathleen Rhoades, a research
> associate with the Center for the Study of Testing, Evaluation and
> Educational Policy at Boston College. "The way we treat tests in this
> country, it's as if the tests deliver these strong, solid, positive,
> irrefutable results. And that is not the case."
>
> By Rhoades' count, there were 137 publicly disclosed cases of
> large-scale testing errors by educational testing companies from 1976
> through early 2004, with most of them occurring since 1997.
>
> "People are under the false illusion that because the tests are graded
> by a machine that the process is objective," said Robert Schaeffer, a
> spokesman for the National Center for Fair and Open Testing, or
> FairTest. "But everything, including programming the machine and doing
> quality control on it, is, in fact, done by human beings. And all 
> humans
> make mistakes."
>
> Rhoades and Schaeffer say a lack of regulation, cost pressures and 
> tight
> deadlines aggravate the problems.
>
> The consequences of grading mistakes and other mishaps have intensified
> over the last decade with the increasing reliance on so-called
> high-stakes testing. In recent years, President Bush's No Child Left
> Behind Act has triggered expanded testing by many states.
>
> At the college level, standardized tests play an important role in
> deciding who is admitted, who wins scholarships and who gets into 
> honors
> programs.
>
> In K-12 education, they help make such determinations as school
> rankings, teacher licensing and pay, and whether students graduate high
> school.
>
> The SAT and other tests are usually graded accurately, but when 
> problems
> occur, "it gets people crazy because it's high stakes and it makes
> important decisions," said Eva L. Baker, co-director of the National
> Center for Research on Evaluation, Standards and Student Testing at 
> UCLA.
>
> The potential for problems was driven home last week with the 
> disclosure
> that the Educational Testing Service, one of the nation's biggest
> testing organizations, had agreed to pay $11.1 million to settle a
> class-action lawsuit.
>
> The case was brought on behalf of plaintiffs who received incorrect
> scores on teacher-licensing tests known as the Praxis exams.
>
> In all, 4,100 test takers were wrongly told they failed.
>
> The tests are used in many states but not in California.
>
> California has suffered other testing problems, however. They include a
> foul-up in which a company now known as Harcourt Assessment Inc.
> miscalculated the results of 19,000 students and 22 schools on a
> Stanford 9 achievement test given six years ago.
>
> The SAT grading problem, which came to light this month, involves
> incorrect scoring on at least 4,600 college entrance exams, less than 
> 1%
> of the total. About 400 students received lower grades than they 
> deserved.
>
> The College Board, the owner of the exam, later disclosed that there
> were an additional 1,600 SAT exams from October and that some of those
> tests may have been incorrectly graded. Those exams had been flagged 
> for
> scrutiny due to concerns about possible cheating, among other reasons.
>
> Many admissions experts said the effect on this year's high school
> seniors would be muted. Although some test scores were off by nearly 
> 400
> points — out of a possible 2400 — 83% of the errors were in the range 
> of
> 10 to 40 points.
>
> College Board officials have repeatedly expressed regret over the
> problem, which sent many colleges scrambling in the final days of the
> college admissions season to reevaluate applications from affected
> students.
>
> But the officials said that with a test taken by 2.3 million students a
> year, including 495,000 in October, it was difficult to eliminate all
> errors.
>
> "It is a very high-volume operation, and you are dealing with millions
> of tests a year, and you're dealing with many questions on each test,"
> said Laurence Bunin, the College Board's senior vice president for
> operations. "In any high-volume operation like this, there's
> statistically the chance for some rare error."
>
> In this case, the problems were blamed on the scanning of the
> multiple-choice portions of the test performed by a College Board
> contractor, Pearson Educational Measurement, one of the nation's 
> biggest
> educational testing and assessment businesses.
>
> Pearson officials said the problems stemmed mainly from humid weather
> that caused answer sheets to expand.
>
> Also, some answers were penciled in too lightly or too incompletely to
> be read by scanners, they said.
>
> Some testing experts said they were perplexed by that explanation and 
> by
> the fact that two students, not Pearson, discovered the problems.
>
> Craig Hoyle, a research consultant at Boston College's education 
> school,
> said there were ways to prevent problems, such as using equipment to
> screen out humidity-tainted answer sheets. In addition, he said, 
> "dummy"
> quality-check tests can be interspersed with the tests to detect
> scanning problems.
>
> Pearson spokesman David Hakensen said his company had used 
> quality-check
> tests, but not until the exams arrived at its processing center in 
> Texas.
>
> He said they were not mixed with exams at test sites — where they might
> pick up the same humidity that affected the actual tests — because any
> stray marks made on the quality-check tests would undermine their
> effectiveness.
>
> Critics attribute scoring problems partly to cost-cutting by testing
> companies.
>
> Pearson officials dispute that point.
>
> The company has experienced problems in Minnesota, where in 2002 a 
> judge
> found that a company Pearson acquired "continually short-staffed" a
> testing program, and in other states.
>
> But experts point out that no technology, be it an airline or a testing
> system, is foolproof.
>
> "Funny stuff happens," said Robert Boruch, a professor of education and
> statistics at the University of Pennsylvania. "The main objective is to
> reduce the likelihood of it happening. You can't reduce it to zero."
>
>
>
>
> Yahoo! Groups Links
>
> <*> To visit your group on the web, go to:
>     http://groups.yahoo.com/group/ARN-state/
>
> <*> To unsubscribe from this group, send an email to:
>     ARN-state-unsubscribe@yahoogroups.com
>
> <*> Your use of Yahoo! Groups is subject to:
>     http://docs.yahoo.com/info/terms/