PDI-7: Knowing What They Know: Writing Assessment Questions That Reveal Student Thinking
Date: Wednesday, March 18, 2009
Location: New Orleans—Ernest N. Morial Convention Center, room 342
Science Area: Integrated/General
Intended Audience: Elementary; Middle Level; High Shcool; Supervision/Administration
Recommended Pathway Sessions
Horizon Research, Inc.
Sean Smith, Senior Research Associate, Horizon Research, Inc.
Melanie Taylor, Research Associat, Horizon Research, Inc.
- What are the characteristics of assessment items that reveal what students know and understand?
- What are the most common pitfalls in writing assessment items and how can teachers avoid them?
- How can student feedback improve assessment items?
- What strategies do students use to answer questions when they don’t know the content?
- What characteristics of items make them accessible for English language learners?
Consider this scenario…After a unit on seasons, a science teacher asked students the following item on a test:
Which of the following locations would you expect to have the least temperature variation throughout the year?
A. Toronto, Canada
B. London, England
C. Sydney, Australia
D. Manta, Ecuador
The teacher had done a thorough job teaching the unit, and based on student responses to questions she asked during class, she was confident they would recognize a location on the equator as the correct answer. She was disappointed and confused when three-fourths of the class missed the item. As she returned the tests, she asked several students why they got the question wrong. Almost without exception, each student asked, “Where’s Ecuador?”
Most teachers have probably had a moment of realization similar to this teacher’s. They write a test item they are sure is crystal clear, only to find later that students lacked some prerequisite knowledge, or that students interpreted the item differently than intended. As a result, the item reveals little or nothing of what students understand about the science.
For the last several years, we at Horizon Research, Inc. (HRI) have been developing assessment items and tests for students through a project funded by the National Science Foundation. Through this work and through workshops we have conducted around the country, we have had the opportunity to compile item-writing principles from several sources and to add our own. In this PDI, teachers will have focused, practical experience applying these principles by writing and revising assessment items in a collaborative setting.
The PDI will involve participants in a process for developing questions that uncover what students understand about science concepts. The process begins with a clear focus on the science content being assessed and continues with drafting and revising items, continuously evaluating them against the principles discussed. Individual teachers can use the process, and it is especially well suited for collaborative item writing. It serves as an effective form of ongoing, collegial professional development.
The PDI is intended for any K–12 science teacher. It will focus on writing multiple-choice items, but the principles apply equally well to open-ended or constructed response questions.
Each teacher who attends will have access to the items written during the PDI. We encourage teachers to bring laptop computers to the session, but they are not required.