Achieve Inc.'s Pinnochio report

["Lisa Guisbond" <lisa.guisbond@verizon.net>]

I'm passing on Anne Wheelock's response to Jerry Bracey's post, since
she's not a subscriber to this list.

 

Lisa

 

At 09:32 AM 6/16/04 -0400, Jerry Bracey wrote:



One graph on page 8 purports to show that while only 48% of the class of
2003 passed the MA high stakes test, MCAS, in math, 95% of the class of
2003 eventually passed..  The graph is from the Massachusetts Department
of Education, which should arouse immediate suspicions. ...The problem
with the graph is that it does not account for those who dropped out or
switched to GED or other programs (in Florida the number of kids getting
a GED almost doubled from 2002 to 2003).  When those are taken into
account, the pass rate drops into the 70's, lower in areas that are
heavily black and Hispanic. 


That's right. When you base pass rates on all students who were original
members of the Class of 2003, (and don't factor out students who dropped
out, repeated a high school grade, transferred out of state, etc.), the
on-time pass rates in Massachusetts show wide disparities among groups.
For the class of 2004, the Project Through the Education Pipeline
Project at Boston College reports that MCAS "on-time pass rates" --
rates that reflect the capacity of schools to help students both pass
MCAS and also progress on time to graduate with the Class of 2004 -- are
as follows:
- All students:  74%, not 96%.
- White students: 80%, not 98%.
- African American students: 59%, not 88%.
- Latino students: 54%, not 85%.
- Asian students: 89%, not 95%.

There's a more detailed report at:
MA Dept. of Ed "Progress Report" Inflates MCAS 'Pass Rates' for Class of
2004 (June 2004)
http://www.massparents.org/news/2004/passrate_2004.htm

A similar analysis for the Massachusetts Class of 2003 is at:
MA DoE inflates pass rates for the Class of 2003 (March 2003)
http://www.massparents.org/news/2003/inflated_scores.htm 

There are other problems with attributing the increase from 48% passing
MCAS math (in 1988) to 95% (in 2003) only to the impact of the "pass or
don't graduate" threat on student motivation.    

The 1998 "passing" rate of 48% refers to the class of 2000. This class
took MCAS once and only once in May of 1998.  These students had not
taken MCAS in any grade prior to their taking it in 10th grade. They did
not take retests in their 11th or 12th grade years. They had not had
MCAS prep tests, MCAS test prep classes, MCAS prep software programs.
Only about 7% of the class (5,018 students) had been lost from the class
between grade 9 and 10.  Students who scored "219" on the test (there
were thousands) that year were considered "failing."

The 2003 "passing rate" of 95% refers to the  Class of 2003, who had a
different set of experiences.  This "passing rate" is based only on
students from the class of 2003 who were still enrolled in the class as
of February 2004 - eight months after their scheduled graduation.  These
students had had six rounds of MCAS testing, including five rounds on a
"focused retest" that included easier questions.  Students in this class
had had the experience of taking MCAS in 8th grade; they had had MCAS
prep courses and exposure to MCAS software programs.  This class had
lost 18,735 students along the way -- to dropping out, repeating a
grade, transferring out of state or other reason (Of 77,733 students,
only 58,994 had survived), so the scores of 24% of the class -- one out
of every four students -- are not included in the 95% pass rate.
Finally, for the class of 2003, the MA Department of Education had
decided to change the way they treated scores of "219." Starting with
this class, the "219" scores were rounded up to "220" and treated as
"passing,"  thus putting several thousand more students, who would have
"failed" in earlier years, into the "passing" column.

It is disingenous to attribute the increase in pass scores in
Massachusetts to the effects of high stakes test on motivation. There
may be some impact, but a lot of other things were at work.

Anne Wheelock