Make the Tests Public

[George Sheridan <learn@jps.net>]

 "The answer's not here." When the second child raised a hand to tell me the same
thing, I knew what was coming. One by one, fourteen children would raise their
hands to tell me that there was no correct answer choice for the fourth question
on their test. To each child I replied, "This is a tricky question. If you read
the passage carefully, you will see that one of these answers is correct." On
this second grade reading test, the four answer choices were numbers. There was a
number in the text on the preceding page - a different number, accompanied by a
phrase like "not greater than." One of the four answer choices was within the
range described by that phrase.

At lunch that day, a colleague described how one of her third graders beat his
head on the desk and burst into tears before ever picking up a pencil. He had
just learned that, on the test section they were about to begin, he had been
given a version with more questions than the version given to some other students
in the room. To evaluate test items for possible future use, bureaucrats had
created several versions of the test and prescribed how they were to be assigned
to students.

If reporters, parents and teachers could assess the quality of the tests on which
all the statistics about "improvement" and about "failing schools" are based,
they might be a little less willing to accept test scores as an accurate
portrayal of student learning.

In California, tests are secret. Even when the scores and API rankings are
released, months later, the tests are still secret. Some tests items are
eventually released, but not in a way that has enabled researchers to say, "This
test score was based on this (possibly flawed) item."

Some of the potential problems I saw in the test given to my second graders this
week (in addition to the reading question requiring students to choose X-3 rather
than X+1 or X+10) included:

  * A question in which students must choose as the definition of the word
    "fastest" that someone "finished the race first." All my students are
    familiar with Aesop's tale of the hare and the tortoise, in which the fastest
    animal does not finish first.

  * A question in which the circles were not aligned with the answer choices, so
    that the circle for the correct answer was halfway between that answer and
    another.

  * A question for which a student can only determine the correct answer if s/he
    knows that "Janet" is not a boy's name. Crossing out in his test booklet all
    the wrong answers, one of my students left two names that seemed to meet the
    criteria specified for this question. He  did not cross out "Janet" or a name
    that he knew to be a boy's name. So he incorrectly answered that two boys had
    accomplished the task. His error had nothing to do with his reading ability
    or comprehension strategies. It simply revealed a difference between his
    cultural background and that assumed by the test makers.

  * A question in which students were asked, "Which sentence is in the wrong
    place in this paragraph?" The answer choices in their entirety were 1, 2, 3,
    and 4. Students had to understand that the paragraph referred to was the
    paragraph preceding the question preceding the question they were now
    answering.

  * A question is which a student was to identify what happened on the "third day
    of the week," by which the test makers meant Wednesday. Our state-adopted
    math program teaches that Wednesday is the fourth day of the week.

  * This same question was based on a table showing two names beside each day,
    with one name written above the other in the column. Some students assumed
    that the two names were in fact first and last names for one person, an error
    facilitated by the fact that two of the second (first) names are in fact
    common last names, similar to Barry and Johnson. Two other names appeared to
    be Japanese and African, and thus unfamiliar as first or last names to most
    students.


These are just the potential problems noticed by one teacher who gave the test
over a three-day period last week, returning the test to the school site test
coordinator each afternoon. How many more inappropriate questions,  formatting
errors, and wrong answers might be found if an interested public were given a
chance to review the tests?

I noticed two things about the second grade math test.

  * For at least nine of 71 test items, the answer was expressed as a fraction or
    was a picture depicting a fraction. This seems like a pretty heavy emphasis
    on fractions for this grade level.

  * This year, as in previous versions of the Grade 2 Star, some students who
    know how to subtract three-digit numbers with regrouping will miss a test
    item simply because of the test format and directions. Question eight
    required three-digit addition with regrouping. Before students began the
    problem, the teacher read aloud from the "Directions for Administration,"
    "What is the solution to this problem?" Question nine was in the same format.
    The numbers were changed, and so was the sign. The "Directions for
    Administration" required the teacher to read again, "What is the solution to
    this problem?" The first answer choice was the sum of the two numbers. A
    careful student might note that the sign had been changed, and could have
    then gone on to examine the other answer choices to find the difference
    between the two numbers. But it is hard to escape the conclusion that the
    test designers purposely set up students to fail. Knowing that at least some
    students would experience a certain amount of tunnel vision as a result of
    test anxiety, they could have signaled to students that the second problem
    was different from the one before it. It would have been simple to say, "Find
    the sum" and "Find the difference," or even "Add" and "Subtract." By creating
    problems that looked and sounded alike, and by putting the sum first among
    the answer choices for the second question, the test makers greatly increased
    the number of students who would be reported as less than proficient in
    subtraction.


The Grade 2 math test also penalizes students who like to reread directions,
because they are not given written directions. They must listen to the teacher
read the directions exactly two times.

George Sheridan