34967547 - All About Assessment W James Popham An...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: All About Assessment W. James Popham An Unintentional Deception ligament is a nine—letter word which, when preceded by the word assessment, seems to trigger a flock of four-letter words. Yes, assessment alignment has caused more than a few educators to completely abandon curse control. innocently, assessment alignment simply calls for a state’s accountability tests to appropriately measure that state's official curricular aims. But this is a far tougher task than it appears to be. Especially since the 2002 enactment of No Child Left Behind (NCLB) and the subsequent release of regulations implementing that federal law, officials in state departments of education have been scrambling to come up with NCLB-required evidence that their accountability tests are suitably aligned with their states content standards—that is, with their states curricular aims. To secure such alignment evidence, state officials typically hire an external firm to conduct an independent study focused on the degree to which a state’s accounta- bility tests mesh with the state’s content standards. Currently, the most popular of the available alignment approaches is the one devised by Norman Webb of the University of Wisconsin.1 Webb has developed a systematic procedure for deter- mining the degree to which curricular “expecta— tions and assessments are in agreement.” Webb's approach revolves around what the measurement community has, for decades, referred to as “content-related evidence of validity," which tells us whether a test accurately measures students’ possession of the skills and knowledge embodied in whatever curricular aims the test is supposed to assess. In almost all of today's assessment-alignment approaches, a group of judges is first trained, then asked to make item—by—item judgments about the degree 80 EDUCATIONAL LEADERSHJNOCTOBER 2008 to which an accountability test’s items match the particular curricular aims those items were intended to assess. Webb’s approach to assessment alignment reflects a traditional way of thinking about whether a test measures what it purports to measure. It is a wellaintentioned procedure, and it satisfies federal demands for such alignment evidence. The only problem with Webb‘s system—and with most of the assessment— alignment approaches based on it—is that in an accountability context, it is misleading! Let's consider, for a moment, the two most important parts of an educational accountability system. First, we have curricular aims—referred to these days as content standards—representing the stuff we want kids to learn. Second, we have accountability tests whose function is to deter- mine whether educators have been successful in getting kids to learn the stuff identified in the content standards. If students perform well enough on these accountability tests, we laud educators for having gotten kids to master the content standards. But if students don‘t perform well enough, we invoke sanctions intended to make unsuccessful educators shape up. These features make sense. Yet, for this approach to work properly, the tests used to evaluate instructional quality must be capable of determining whether teachers have, in fact, been instructionally successful. Accordingly, the following question would be inappropriate to ask judges as they review the relationship between content standards and the items on an accountability test: is this test item in agreement with the curricular aim it is intended to measure? Such a softball question misses the mark. A more appropriate question would be something along these lines: \Mll this test item accurately measure how effectively students have been taught to master the curric- ular aim that the item measures? You see, because the heart of an educational ;ieti>iiiit:ihiliiy :ipprotieh is the use ol test results to gauge iitstrueiiorial tgiitiltttiantl heezuise irtsii'tieiioiirtl Lilltll|l\' should he (lClL‘l'lttlIiL‘Ll hy what students lt‘fll'nil he ei'nphrtsis ill rt tlelenSihle alignment stritly must he on the issue ol lt'llt'lllt'l iiisti'iirjttoii mix Slit't't‘N'Sltil. If an .ieeouriiritirlity test: items are not Luiprilfle ol distinguishing hetween good and htitl instruction. their educational .iet'uunliihility Slit‘tpl)‘ makes no sense. Test items that are onlyjutlpetl on whether they it‘th‘i‘ u'ith :1L‘tlt‘t‘tt‘tilttl‘nlttt may or may not rttrt‘tti‘nlely l't‘lrlt‘L'l iiisirtit'iioirtil ttiitrlity For example, suppose .i stateuppr'rwetl L‘Lirrieulut' arm in soetril studies calls lor elertreiiirtry school stutlents to he .ihle to employ stmple maps to generate ti sensihle plan [or getting lroni Point .-\ to Point B. Nile that the L‘sSL‘tiCC ol ihrs trui'rieulrir arm is lot sttttleiitb' to he rihle ttt llgttre out lit‘ll’ irtgr‘t some plrtr'e But what ii an item supposedly lllt’drr lll'lllg lltlr-i ttiii'i'ittilar tum (letili \tiih suhile aspects oi rntip retitliiig—stieh Al‘- \/ how thtLllltl.tl'lL‘< ill't" graphically tlt‘littetliihtit \\’Clt' not central to \ihttl students actually needed when ll‘filltt“ .t iiitip to get sortiepltiee? Siteh .iii item might very we“ pass muster when using iriost ol today.» LlShL‘SSlllctll-tl igntiieni procedures. The item lr- elertrly related to map rattling, But the lliti tlirri ti test Assessment alignment causes educators to abandon curse control. item lFujtlt'lL‘CLl to he "in agreement" utth this L'ui‘rit‘tiltit' tiim does not sit-xiiin that the item will help its tleierinit'ie him sut'eessliilly lt‘;lt‘llt‘t'~ promoted students mastery ol .i "get lr‘rtm Point .»\ to Point P)" etirrit'tilar .iim. And yet the tieeurtite t’letei'miiitition till instruetioritil quality [:1 the very eot’iier'stoiie Ull ti tlelettsil‘ile etltieuttwit .ieeut.tnt.;thilily prourxiin. Literacy is hard. itemitise most states rtou li.i\e et'itlenee lll‘ll their .ii‘i'niiiitalitliit tents .ll‘t‘ apparently “.ihgneil' to their stiles ttti‘t'tettltii‘ titnis. it is Ll‘mltllle‘tl ilt.it illt‘Nl‘ teHl< iltt it *Elllfililk‘ll‘fl titlt l‘l iiie.i~iii titty etltterttot's' 'ti‘islt‘ut'tional tlllillll) ltit this is the moment it hen tlt‘i'etitioii tl.itir es onstage. We rtsstiiiie i|i.ii .iei,oiiiit;ilii|ii\ lt'hlhibt‘ttlihi' illi't tllt‘ iilit;iir'rl——pro\ irle us with an .ieeiii'.iie [Wlt'llll't‘ Ul eihietitora' institutional quality Mirth ill the tirrie. this assumption is iiiiuni‘rtiriieri. -‘\l.iin fitlt'tessltil seliriols \\ ill he IL‘ELtlL'Lll’tl .is losers; intim iirisiit‘et‘s’sltil schools \\'|ll be seen Ll: Winner's. \\-t- \i ill he do t'l\'t‘tl heeutise we tillt‘\\rL'kl ourselves to he sedueetl h) an iiisuliieieiitli' tlerir.iiitlirig toneeptioii ill.1.‘w.\t'r\‘illtt‘lll .iliuiuneiii ‘\\el1|r, \ | tJtltlJ‘ ,‘tlieitiiii‘iir \Elliill in lriiierrtiee illl\ tilttlllt'ttiritli \_ xi lt‘l'li :1 .rii.l w. i.il siiiiltn til «trite xiriiitliiiilx \\.t~ltlll;‘,li‘ll lit totini'il ol t hiel Ni.iie \tliiml illin t‘tu W. James Popham is Emeritus Professor in the UCLA Graduate fielrool of Education and llllttttt‘ir-tlli‘ilt Sliiilieb, wijophaleiicla eriii Thinkfinity Literacy Network helps. Free resources for teachers, school leaders, parents and students. - Verizon Literacy Program Self-Assessment Tool — Assess your program's research-based literacy practices for children or adults Parent Involvement Collection torTeachers — Multimedia resources to engage parents meaningfully in their child‘s education Reading Readiness Collection for Parents — Fun literacy activities parents can incorporate in their everyday routines Self-Paced Courses ~ Staff development at your fingertips. 24/? National Center firFami ly Literary verian :\‘~‘-Ui l\t|il\ Iiilt 5i I‘l lt\l~lt|\ PROLITERACY www.thinkfinity.o lliteracynetwork llltlll‘!'\!l“- 81 \kll L l HRII l‘l l \l. Copyright of Educational Leadership is the property of Association for Supervision éCurriculum Development and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. ...
View Full Document

This note was uploaded on 10/07/2009 for the course EDP 351 taught by Professor East during the Spring '09 term at West Chester.

Page1 / 3

34967547 - All About Assessment W James Popham An...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online