Re: More intersting revelations from EdTrust

[Art Burke <aburke@VANSD.ORG>]

>>> PAVURSOL@AOL.COM 01/08 8:00 PM >>>
...
> The process for determining this is called "validation."  It's done as the
> test is constructed and continuously afterwards in light of experience with
> the results.
>
And how is it measured and reported?  HOw do you know you have it or you don't?
------------------------------------------------------------------
Mickey, consider, as an over-simplified example, a standards-based test of, say, fourth grade math.

First, you would make sure that the items adequately represent the universe of math skills that the standards say are important and that kids should know.  Next, you would make sure that the items don't depend too much on reading skills or other skills.  You would, of course, check to make sure the items (and test format and procedures) were free from bias - gender, racial, ethnic, socio-economic.  You would make sure that scoring procedures are accurate and reliable (it is especially important to have adequate procedures and training when the test, or parts of it, are scored by raters.)  You would establish clear rules for reporting scores in ways that are understandable and accurately reflect what the test tells you about kids' achievement.  You would see whether kids who score high also score high on related measures of math achievement and whether kids who score low also score low on related measures of math achievement.  You would study how the quality of the teaching of the material and skills covered on the test relate to kids' achievement.  

Finally, and this is a biggie that I think most everybody misses, you should take a kind of "step back" look.  You've got to look at the whole system:  the standards (what they are, how they are set, and the purposes for which they were established); instruction (whether teachers have adequate understanding of the standards, adequate training in teaching them, and adequate resources for doing so);  assessment (everything above, plus   
now the accumulating consequences of the test - the effects it has on instructional decision-making, gaming the system, media coverage, decisions kids and parents make, and so on).  You've got to address whether the whole system is working the way you thought it should and, if it is not, whether it should be changed.  This is a tripping place for most state systems.  We'll wait to see how much tripping occurs after the new federal regulations are in place.

To address your last question, there is, Mickey, really no simple "measure" and no single methodology behind all of this to tell you whether you have it or you don't.  And that's fine, really, because important matters are the ones you should keep coming back to.     

Art

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.