NY Times 9/17 on test fiasco

[Monty Neill <Mneillft@AOL.COM>]

NY Times
        September 17, 1999


        Miscalculation on Scores Shows
        a Weakness of Tests


        Related Articles
        8,600 in Summer School by Error, Board Says (Sept.
        16, 1999)
        Chancellor Cites Test Score Errors (Sept. 15, 1999)

        Forum
        Join a Discussion on New York City's School
        System


        By ANEMONA HARTOCOLLIS

           he revelation that a national testing company
           miscalculated the reading and math scores of
        thousands of New York City students opens a
        window into the fragility of standardized testing,
        even as school districts across the country are
        increasingly using test scores to make high-stakes
        decisions.

        Officials for both the New York City school system
        and the testing company, CTB/McGraw-Hill, one of
        the three biggest in the country, have admitted that if
        not for the persistence of one whistle-blower -- the
        city's testing director -- the errors might never have
        been discovered.

        Yet those errors led to one of the bleakest chapters in
        the tenure of Chancellor Rudy Crew, who had made
        literacy a priority and then in June was forced to
        announce the original low scores. The Chancellor
        used the scores to make a host of decisions with
        huge impact on the lives of children and educators:
        from taking over failing schools to removing
        superintendents to sending more than 8,600 children
        to summer school who should not have been forced
        to be there.

        Now thousands of schoolchildren are trying to salve
        the ache of being unfairly branded as failures, and the
        mistake has tarnished the reputation of standardized
        testing carefully cultivated by the testing companies
        themselves.

        "I don't think these kinds of errors are rampant," said
        Michael J. Feuer, director of the board on testing and
        assessment of the National Academy of Sciences.
        "But obviously, when you're dealing with attempts to
        analyze large quantities of data, errors are always
        possible."

        The error on the CTB/McGraw-Hill test, which the
        company said also affected eight other systems, was
        not the first for the testing industry. In the
        mid-1990's, California introduced a statewide test
        and was criticized after improperly releasing scores
        to hundreds of schools that were based on inadequate
        samplings of students.

        The problems, critics say, graphically illustrate why
        many educators and testing companies themselves
        advise against using a single test given on a single
        day as the sole criterion for making decisions about
        promotion or graduation.

        "It's clear that CTB/McGraw-Hill messed up," said
        Monty Neill, executive director of the National
        Center for Fair and Open Testing in Cambridge,
        Mass. "But CTB did not make the Mayor, the
        Chancellor or the Board of Education decide to use
        test scores only, in violation of the standards of the
        testing industry, to make decisions about kids."

        In the New York City episode, the testing company
        and Dr. Crew find themselves on the defensive. Yet
        Dr. Crew, who just three months ago said he would
        make "no excuses" for some of the worst test scores
        in years, now boasts just the opposite situation: one
        of the brightest pictures in years. The adjusted
        statistics show that 48.5 percent of New York City's
        third, fifth, sixth and seventh graders read at or above

        grade level, not 44.6 percent as announced in June.
        After a calculation to make the 1999 test comparable
        to the 1998 test, Dr. Crew announced on Wednesday
        that reading scores had risen 4.9 percent from 1998,
        an unusually high one-year gain for any school
        system.

        Yet instead of basking in the glow of positive
        headlines, the Chancellor has found that achievement
        virtually ignored in the atmosphere of skepticism
        created by the errors. While almost no one -- except
        the school system's own director of assessment,
        Robert Tobias -- raised doubts about the dramatic
        10-point decline in math scores announced in June,
        the new reading scores have drawn a different
        reaction. (The board has not yet recalculated math
        scores.)

        "The fix is too good to believe," said Stephen Ivens,
        vice president of Touchstone Applied Science
        Associates, in Brewster, Putnam County, which
        supplied its reading test, Degrees of Reading Power,
        to New York City schools for 12 years before it was
        replaced by CTB/McGraw Hill.

        "Could you as a publisher make the data come out
        more or less favorable?" he asked. "The answer is,
        yes you can, particularly once I know what the
        results are."

        Dr. Crew said yesterday that he saw no reason to
        abandon the use of standardized testing in making
        high-stakes decisions like whether to promote
        students. "That is a really, really cynical view," he
        said of comments like Dr. Ivens'. " 'I paid them off';
'I
        sold them my first-born.' There's a level of it being
        absolutely insidious -- when the scores look like they
        might be anywhere near decent, to just make this a
        scandal. These teachers and these children, they
        earned something."

        Dr. Crew said he used the reading and math tests to
        screen for summer school, despite Tobias's
        suspicions, because the testing company strenuously
        denied until last week that there was any problem.

        David Taggart, the president of CTB/McGraw-Hill,
        said yesterday that such scoring problems occur
        rarely, and are normally detected, as in New York
        City's case, by school district officials who notice
        something amiss when they put the standardized test
        scores in the context of everything else they know
        about their students.

        CTB/McGraw-Hill has posted a prominent warning
        on its Web site, advising school districts that "no
        single test can ascertain whether all educational
        goals are being met."

        Dr. Crew has removed five superintendents and put
        four more on probation, again citing low scores.

        Now, eight of the nine districts where he removed
        superintendents or put them on probation show gains
        in reading instead of declines.

        Robert Riccobono, who was ousted by Dr. Crew as
        superintendent of district 19 in East New York,
        Brooklyn, said he believed his fate and that of the
        other superintendents would have been different if
        the erroneous scores had never been announced.
        Riccobono is the only superintendent appealing his
        ouster. The new data, he said, could only help his
        case.

        "Wow," he said, as he learned of the revised scores.
        "It has to help, because it's clear to me that the only
        reason we were under scrutiny to begin with had to
        do with the lower performance on the reading test."
http://www.nytimes.com/library/national/regional/091799ny-summer-school-edu.ht
ml

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.