31501219 - ASK ABOUT ACCOUNTABILlTY What’s Valid What's...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ASK ABOUT ACCOUNTABILlTY What’s Valid? What's Reliable? W. James Popham Question: Assessments originally created to measure stan- dards and evaluate schools are now being “retrofitted“ to measure students‘ grade-to—grade progress. What are the risks of retrofitting assessments in this way? -—Emily A. larsen, White Bear lake Area High School, White Bear lake, Minnesota Answer: A single test can accomplish multiple measurement missions in only a limited number of instances. For example, if an accountability test is built to measure a modest number of important and well—described curric- ular outcomes and does so by providing per—outcome and per-student information about each assessed outcome, it can do a good job of evalu- ating schools yet also help classroom teachers make better inStructiona] deci— sions. But to construct a test capable of It’s clearly better to use teaching practices that probably will measures something with consis- tency; think of validity, however. as accuracy. But it’s not the validity of the test that‘s involved here. Instead, it‘s the validity of the infer- ence—or, if you prefer, the inter- pretation—that’s based on a test takers performance. Lots of folks refer, far too loosely, to the validity of tests. How many times, for example, have you heard someone ask, Is this test valid? But tests aren’t valid or invalid; it’s the test—based inference that’s valid or not. To make a valid (that is, accurate) inference about a student‘s mastery of a particular cognitive skill, we need tests that are reliable. If a test measures l2-UWLUK‘UL m®®®® [email protected]®@@ doing this sort of double duty, it’s SUCCEEd than to something inconsistently, how can almost always necessary to tackle those educators be confident that their test- two assessment functions from the get- use those that based inference is accurate? Test relia- go. Belatedly retrofitting solo-purpose , bility is a necessary condition for us to te5ts for a second measurement mission Probably won 1:. make valid inferences, but it is not a is almost always a dumb idea. Folks who pursue such an approach may know scads about psychometrics, but they know little about teaching. Question: Someone once told me that the more reliable a test is, the less valid it is—and vice versa. ls this true? —Karen Ruffner, Rockford Public Schools, Rockford, Illinois Answer: Sorry, Karen, but you’ve been running with the wrong crowd. it’s not true that when a test becomes more reliable, it becomes less valid. Nor is the vice-versa version of that same proposition true. Here’s an easy way of keeping these two assessment concepts straight: Think of reliability as whether the test 78 EDUCATIONAL LEADERSHIP/FEBRUARY 2008 sufficient condition. A genuinely reli- able test may or may not provide evidence leading to a valid inference. For instance, my bath- room scale may measure my weight accurately, but if I tried to use that scale to make an inference about my problem- solving skills, the resulting inference isn’t going to be valid. If you keep in tnind, with regard to assessment, that reliability equals consistency whereas validity equals accuracy, you‘ll do just fine. Question: What kinds of tools can we use to assess younger students‘ performances in citizenship, in grades 1~3, for example? —~Shailzhah oi Shamsi, Citizenship and Development Center, Abu Dhabi, United Arab Emirates Answer: Measuring students' status with respect to citizenship is tricky It might seem that all you need to do is to identify the skills and knowledge you want students to acquire and then measure whether students have, after instruction, acquired such skills and knowledge. But there is scant unanimity among social studies educators regarding what the appropriate citizen» ship skills and knowledge really are. When you focus on very young chil— dren, tnany of whom are not yet skilled readers, the assessment difficulties increase dramatically 1 encourage you to consider the possibility of assessing your students' citizenship-related attitudes as well as their cognitive achievement. A number of powerful attitudes and values are associated with being good citizens, such as respect, tolerance, honesty, responsibility, courage. and compassion. To assess the affective status of students who have not yet become skilled readers. teachers sometimes rely on observations of those students” routine behaviors in class. Unfortunately, expe« rience suggests that such behavior-based inferences about students‘ affect are frequently incorrect. Regrettably, for young students who can’t read, its almost impossible to establish a "setting of anonymity" required for accurate affective assessment. However. as soon as the children can read reasonably well, you can employ anonymous self—report inventories to assess their attitudes toward, for example, people from different ethnic or social groups. Question: Students in a classroom assessment course I teach sometimes raise the issue of whether instructional procedures “based on sound science“ really are adequately supported. When Robert Marzano and others identify research—supported teaching practices. how much confidence should teachers place in those practices? —}e_fl'rey Glunz, Professor of Education, Yeshiva University Answer: I‘ve been following with admi- ration the work of Bob Marzano for many years. Thankfully, when Marzano or any other sensible synthesizer of empirical evidence about teaching arrives at a research-based recommen- dation regarding instruction, he or she always offers it with appropriate caveats. Because teaching is so consummately particularistic—that is, an activity involving particular teachers teaching particular students in particular settings—it is impossible to find research results specific to such situa- tions that will yield infallible instruc- tional guidelines. The best we can do is to tell teachers that when they use a given research-supported tactic, the tactic will probably prove effective. The odds are on the teacher's side. And having the odds on your side can be a real boon. just ask any professional gambler. So when your students raise ques- tions about the soundness of research— supported teaching practices, point out that even Bob Marzano doesn‘t supply iron-clad money-back guarantees. But it's clearly better to use teaching prac- tices that probably will succeed than to use those that probably won’t. W. James Popham is Emeritus Professor in the UCLA Graduate School of Education and Information Studies; woopham©ucla.edu. Are you facing a challenging accountability dilemma in your school or ciassroom? Do you have a question about assessment? Submit your queries to noted assessment authority W. James Popham at AskPophamfiascdorg. Responses to selected questions will appear in subsequent columns. FlT4L€NZNlNéiLom Bring Your Own Body! Wellness-Activities that provide: -maximum movement -mim'mal equipment -in any classroom -for any subject The Brain, Exercise & Learning Session # 1166 Booth # 1306 Assor I:\T|0l\‘ t-ua St'rertslok AND CURHILUI UM Dti‘izwrutut 79 Copyright of Educational Leadership is the property of Association for Supervision éCurriculum Development and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. ...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern