Error found in school rankings

[George Sheridan <learn@jps.net>]

Error found in school rankings
Comparison of similar campuses is pulled off Web for correction.
By Dan Nguyen -- Bee Staff Writer
Published Saturday, April 8, 2006
Story appeared on Page A3 of The Bee
http://www.sacbee.com/content/politics/story/14240708p-15060410c.html


The discovery of a miscalculation has caused the state Department of 
Education to withdraw a ranking used to compare academic performance of 
schools of similar characteristics.

The state said Friday it had omitted a variable - the number of students 
whose ethnicity was unspecified - from its calculation of "similar schools" 
rankings.

The erroneous rankings, which were released in late March, were pulled from 
the state's Web site and are expected to be re-released in three weeks, 
said Pat McCabe, the state's director of policy and evaluation.

These rankings have been released every year since 1999, when the state 
created the Academic Performance Index - the API - based on a school's test 
scores.

The similar-schools ranking compares the API of one California school 
against 100 other schools that most closely share its characteristics and 
challenges.

The comparability of schools is measured using 14 factors, such as parent 
education level, percentage of English learners, average class size and 
ethnicity.

The re-evaluation of the similar-schools rankings does not affect API scores.

Since its debut in 2000, the similar-schools ranking has sparked confusion 
and controversy. The state had to recalculate the first year results after 
it learned that schools had submitted incomplete data.

A parent in Southern California subsequently sued the state for not 
releasing details of its rankings formula.

On Friday, opponents of the rankings system welcomed the state's admission 
of an error.

Lynn Winters, assistant superintendent of the Long Beach Unified district, 
said her district for years had dismissed the rankings as irrelevant.

"Mathematically, it's uninterpretable," Winters said.

"When you have a similar schools ranking, it needs to make sense to 
people," she said.

But Winters said she took notice this year after principals had complained 
about inexplicable and dramatic drops in their school rankings.

Winters believes there are other errors in the state's formula that will 
necessitate an overhauling of the rankings system.

McCabe said the state's testing department will meet Monday with the Long 
Beach district to hear its complaints. But he said there are complaints 
about the rankings every year and they often stem from misunderstandings of 
the system.

"There's a lot of disgruntlement around the similar schools rank," he said. 
"A lot of it comes from people thinking that schools (on their lists of 
100) ought to look like them."

McCabe said schools are grouped together because of the amount of 
challenges they face, even if the types of challenges are different.

This year's ranking gaffe comes in the same year that the state revised its 
formula and added six new factors, including the number of gifted and 
special education students.

San Juan Unified district officials believe the inclusion of gifted student 
numbers might have resulted in several of its schools taking a dive in 
their rankings this year.

Del Paso Manor Elementary saw its ranking plummet from a top rank of 10 to 
a rank of 2. This means its test scores were in the bottom 20 percent 
compared to the 100 schools in its group. Only about a dozen schools 
statewide fell as sharply.

Principal Phyllis Westrup said she was perplexed, because the school 
actually increased its Academic Performance Index from 862 - which is well 
above the state's goal - to 872. And she said the school's low-income 
students had made large increases in test scores.

"It's kind of hard to understand how your school could make such wonderful 
progress and yet drop in the similar schools rankings," she said.

Donna O'Neil, San Juan's director of accountability, said she had not seen 
the details of this year's calculation method, but has in the past 
encouraged principals to contact schools on their lists and compare notes.

She suspected, however, that Del Paso Manor's number of gifted students - 
nearly 40 percent of the school's population - put it in a more elite 
bracket of schools.

McCabe said he expected complaints about the new formula. In the past, he 
said, schools with small gifted programs complained about being compared 
against other schools with larger gifted programs.

"You get complaints and concerns from people on both sides. You'll always 
have winners or losers," McCabe said.

Long Beach's Winters said the fact that a school like Del Paso Manor could 
be ranked so dismally despite actually improving academically was proof of 
how unintuitive the ranking system is.

Winters said she supported the idea of comparing similar schools, but 
wanted the state to find a different system for doing so.

"We don't mind the (concept). But the system they chose is particularly 
complex and uninterpretable." she said.

About the writer: The Bee's Dan Nguyen can be reached at (916) 478-2653 or 
dnguyen@sacbee.com. Bee staff writer Deepa Ranganathan contributed to this 
report.



George Sheridan