Science Assessment: Research and Practical Approaches
Boston Convention and Exhibition Center
Rooms 205 A–C
Agenda
| 7:00–7:55 AM |
Continental Breakfast |
| 8:00–8:25 AM |
Welcome and Introductions
The Importance of Formative Assessment
Page Keeley, NSTA President, Senior Science Program Director, Maine Mathematics and Science Alliance |
| 8:30–10:00 AM |
Breakout Block A |
| 10:00–10:15 AM |
Break |
| 10:15–11:45 AM |
Breakout Block B |
| 11:50 AM–12:30 PM |
Lunch |
| 12:30–2:00 PM |
Breakout Block C |
| 2:10–3:10 PM |
Plenary Session
It's not just what we know, but how we know and why we believe it?
Arthur Eisenkraft, Distinguished Professor of Science Education at University of Massachusetts, Boston, Director of the Center of Science and Math in Context (COSMIC) and NSTA Past President |
| 3:10–3:30 PM |
Reflection/Closing/Evaluation
Rowena Douglas, Assistant Executive Director for Professional Development, NSTA |
Plenary Session: It’s Not Just What We Know, But How We Know and Why We Believe It?
Session C-1 (Room 205ABC)
Arthur Eisenkraft, Distinguished Professor of Science Education at the University of Massachusettes Boston, Director of the Center of Science and Math in Context (COSMIC), and NSTA Past President.
Session Summary
In this presentation, Dr. Eisenkraft will describe the essential questions of science. He will contrast them with the questions we often find on teacher generated tests as well as on high-stakes assessments. The implications for curriculum will then be discussed.
Breakout Sessions
Aligning Science Assessment Items with Content Standards
Session C-2A (Room 206B), Session C-2C (Room 206B)
George E. DeBoer, Deputy Director, AAAS Project 2061, Washington, DC
Framing Questions
- Is the knowledge or skill specified in the content standard needed to produce a correct response or is there some other way to correctly perform the task?
- Is the knowledge enough by itself to make a satisfactory response or is other knowledge also needed?
- Are students likely to understand the task statement, diagrams, and symbols?
- Could students respond satisfactorily by simply guessing or using other generic test-taking strategies?
Session Summary
Current reform efforts emphasize the importance of aligning assessment with content standards. Developers and users of assessment instruments are faced with the challenge of determining whether items effectively reveal what students know and are able to do with respect to content standards. In this session I will demonstrate an alignment procedure developed by Project 2061 and provide examples of items that are aligned and not aligned to content standards from the National Science Education Standards and Benchmarks for Science Literacy. The procedure is useful to national and state assessment developers and to curriculum developers and classroom teachers who use test items as a basis for important instructional decisions.
Assessment of Laboratory Activities
Session C-3A (Room 206A), Session C-3B (Room 206A)
Arthur Eisenkraft, Distinguished Professor of Science Education, University of Massachusetts, Boston
Matthew Anthes-Washburn, West Roxbury High School, Boston, MA
Framing Questions
- What student learning goals do we have for laboratory activities?
- What evidence will we accept that students have accomplished these goals?
- What formative and summative achievement measures have been shown to be effective for evaluating laboratory activities?
- How can laboratory activities be modified to support inquiry and improved learning?
Session Summary
Laboratory experiments are a staple of all high-quality high school science programs. The National Academy of Sciences has released a study about the future of high school labs that explores what research tells us about their value. Most National Science Foundation-supported curriculum projects have the lab as a central component of instruction. In this session, teachers and administrators will discuss the recommendations of the Academy, reflect on the purpose of labs, share varied means of assessing labs, and explore ways in which laboratory activities can support and improve instruction.
Assessment for Learning: Content and Tools to Support Teacher and Student Learning
Session C-4A (Room 204B), Session C-4B (Room 204B), Session C-4C (Room 204B)
Jim Minstrell, Research Scientist, FACET Innovations, Seattle, WA
Stamatis Vokos, Professor of Physics, Seattle Pacific University, Seattle, WA
Pamela Kraus, Research Scientist, FACET Innovations, Seattle, WA
Framing Questions
- How do you identify students’ problematic conceptions and reasoning?
- How do you choose lessons to address problematic understandings?
- How do you use formative assessment to monitor students’ understanding while they are learning?
- How do you use assessment information to inform instructional decisions?
- What are some of the essential building blocks of learning in science and how can they be assessed?
- What kinds of tools are available to help teachers realize a “diagnostic learning environment”?
- How do you help teachers learn to use diagnostic assessment tools?
Session Summary
Research suggests that formative assessment may be the most powerful instructional practice for promoting student learning in all disciplines. When well implemented, formative assessment provides teachers and students the information they need to make both learning and teaching more focused and effective. Diagnoser Tools are one of a growing number of resources to support assessment for learning in the science classroom. Used formatively, this online resource can help to “diagnose” and address problematic student thinking in science. Diagnoser Tools initially focused on Physical Science and Human Body Systems, but new content has recently been added, including Properties of Matter, to target essential building blocks of learning in all areas of science.
This session will explore the essential elements of what we term a “diagnostic learning environment”—one in which formative assessment is at the center of instructional decision-making and actions. Participants will also be introduced to the effective use of Diagnoser Tools to support an effective cycle of teaching and learning. This session will be most appropriate for teachers and administrators in middle and high school and for facilitators of teacher professional development.
Making Meaning: The Use of Student Science Notebooks as an Effective Assessment Tool
Session C-5A (Room 252B), Session C-5C (Room 252A)
Michael Klentschy, Former Superintendent, El Centro Elementary School District, El Centro, CA
Olga M. Amaral, Associate Dean, San Diego State University - Imperial Valley Campus Calexico, CA
Framing Questions
- How can student science notebooks be effectively used as an assessment tool, especially for English learners?
- How can lesson design strategies be used to maximize student opportunity to learn as evidenced from student science notebooks?
- What are some feedback strategies with the use of science notebooks that teachers can use in formative assessment situations?
Session Summary
Making meaning from classroom science experiences is a major goal of classroom science instruction. Current research suggests that student science notebooks are an effective classroom assessment tool. Combining writing in science with effective language development strategies is proving to be an effective tool for English learners to make meaning from classroom science experiences. The student science notebook then becomes an important element to facilitate student learning, a vehicle for the teacher to provide formative feedback, and a record to provide data to inform decisions regarding instructional planning and alignment. This session is targeted to elementary and middle schoolteachers, administrators, and professional developers.
Measurement of Conceptual Understanding in Physical, Earth, and Space Science Using Misconception-based, Standards-based Multiple-Choice Tests
Session C-6A (Room 254A), Session C-6C (Room 251)
Philip M. Sadler, Director, Science Education Dept., Harvard-Smithsonian Center for Astrophysics, Cambridge, MA
Nancy Cook Smith, Psychometrician, Harvard-Smithsonian Center for Astro-physics, Cambridge, MA
Framing Questions
- What are the characteristics of multiple-choice items that assess conceptual understanding in science and not simply factual knowledge?
- How can teachers accurately diagnose the misconceptions of their students and gauge their effectiveness in promoting conceptual understanding?
- How can professional development providers best assess the weaknesses and strengths in teachers’ subject matter knowledge?
- To what extent have science students mastered concepts taught in earlier grades and how does earlier knowledge acquisition play out in advanced classes?
Session Summary
The weak link in standards-based instruction and professional development is assessment that accurately gauges understanding of science concepts. Our efforts have focused on misconception-driven diagnostic tests that draw upon advances in cognitive science while using a simple multiple-choice format; they include the most common nonscientific beliefs of learners as answer choices. We have constructed an item bank and gathered data from a national sample of 15,000 students and their teachers based upon the NRC’s national Science Education Standards and AAAS Benchmarks. In this session we will characterize the content knowledge of participants and student gains from instruction, and examine the attributes of effective test items. The session is designed for administrators, teachers, and professional developers.
Reformed Teaching Observation Protocol (RTOP): Providing Feedback to Improve K–20 Science Instruction
Session C-7A (Room 251), Session C-7B (Room 251)
Dale Freeland, Physics Teacher, Portage Central High School, Portage, MI
Paul Hickman, Project Consultant, PhysTEC, Boston, MA
Framing Questions
- How do we break the cycle of “teaching as we were taught”?
- Does reformed teaching foster greater student understanding?
- Can we really quantify science teachers’ practice?
- Will formative feedback accelerate professional growth and increase teacher retention?
Session Summary
Recent education scholarship asks that teachers include active-learning, inquiry-based, and problem-solving strategies in their science instruction (Beichner, 2004). These reform strategies have been proven to spark student interest in science, help students—especially women and underrepresented minorities—learn more and get better grades, and lead students to enroll in advanced science courses (Handelsman, 2004). This session will provide the rationale for using RTOP and engage participants in exploratory use of the tool through selected video segments. The session is targeted to a wide audience—elementary teachers, middle level teachers, high school teachers, curriculum specialists, professional developers, school administrators, and higher-education faculty.
Summative and Formative Kit-based Science Assessments
Session C-8B (Room 203), Session 8C (Room 203)
Paul J. Kuerbis, Director, Crown Faculty Center, Colorado College, and Co-Principal Investigator for STEP-uP, Colorado Springs, CO
Linda B. Mooney, Lead Principal Investigator for STEP-uP, Colorado Springs, CO
Framing Questions
- How can conceptual storyline development be used in creating standards-based classroom science assessment storylines?
- How can formative and summative assessments in elementary science be designed to facilitate student learning?
- What is the role of performance assessment in “assessment for learning and of learning”?
- How does assessment development enhance science conceptual understanding of teachers?
Session Summary
This session will focus on the process of developing revised assessments for elementary kit-based science programs (FOSS, STC, Insights). Through a novel, effective professional development program, teachers have developed assessment binders for 20 different kit guides. The binders include conceptual storylines, with attention to standards, embedded formative assessments and performance assessments, sample student work (and/or papers), writing prompts, and rubrics for revised summative performances. This session is aimed at elementary teachers as well as curriculum coordinators, professional development staff, and principals.
The Benefits of Classroom-Level Assessment for Grades 3–6
Session C-9A (Room 207), Session C-9C (Room 207)
Kathy Long, ASK Project Coordinator, Lawrence Hall of Science, University of California, Berkeley, CA
Larry Malone and Linda De Lucchi, ASK Project Co-Directors, Lawrence Hall of Science, University of California, Berkeley, CA
Anne Schoenemann, Elementary Science Specialist, Madison Metropolitan School District, Madison, WI
Framing Questions
- Why assess science at the classroom level?
- How do you know what to assess?
- What kinds of assessment items do you use to get the information you need?
- How do you extract useful information from student responses to assessments?
- What do you do with the information once you have it?
Session Summary
Assessment should play a central role in every elementary classroom. But how do you decide what needs to be assessed? What kinds of assessments provide the best evidence of student learning in grades 3–6? What actions should be taken once you have the evidence? What is the student's role in the assessment process? This session focuses on lessons learned from the ASK Project: the importance of a formative assessment system, its design, its implementation, and its day-to-day activities. We'll provide a framework to help you think about the elements of a model classroom-level assessment system.
Using National and State Science Standards and Cognitive Research to Inform the Design and Use of Formative Assessments
Session C-10A (Room 204A), Session C-10B (Room 204A)
Page Keeley, Senior Science Program Director, Maine Mathematics and Science Alliance, Augusta, ME
Francis Eberle, Executive Director, Maine Mathematics and Science Alliance, Augusta, ME
Joyce Tugel, Science Specialist, Maine Mathematics and Science Alliance, Augusta, ME
Framing Questions
- What role does formative assessment play in a standards-based system?
- How can teachers use standards and research to balance formative assessment with summative assessment?
- What types of probes and formats can help make students' thinking visible to themselves and to teachers?
- How can development and use of formative assessments that probe for students' ideas impact instructional decisions?
Session Summary
There is ample evidence showing that effective formative assessment practices can make a significant impact on student learning. This session will focus on the relationship between specific ideas in a topic, research on students' commonly held ideas, and teachers' use of formative assessment. Using a process developed for the National Science Foundation-funded TPC Project Curriculum Topic Study (CTS)—A Systematic Approach to Utilizing National Standards and Cognitive Research combined with CTS-developed probes in NSTA's Uncovering Student Ideas in Science series, this session will show how K–12 teachers can learn how to develop and use a variety of formative assessments that uncover students' preconceptions prior to and throughout instruction that are linked to key concepts and learning goals addressed in their curriculum. This session will show K–12 teachers, administrators, and professional developers’ ways teachers are engaging in this process and using results to make informed instructional decisions based on standards and research as well as advance their own professional knowledge about student learning.
Using Science Assessment Data to Improve Results: The Power of Collaborative Inquiry
Session C-11A (Room 252A), Session C-11B (Room 252A)
Nancy Love, Principal Investigator and Project Director, Using Data Project, TERC, Cambridge, MA
Framing Questions
- How can science educators make effective and collaborative use of multiple student learning assessments and other data to improve teaching and learning?
- What are the shifts in culture, data use, leadership, and capacity that result in assessment data being put to their best uses?
- What are some tools and processes for engaging in data-driven dialogue and ongoing reflection about assessment results?
- How can science education leaders become even more effective in leading continuous science improvement school- and districtwide?
Session Summary
Schools have more science assessment data available, including high-stakes science test results. But having these data available does not mean that they are used effectively to guide instructional improvement. In this presentation, participants will learn about collaborative inquiry, a structured improvement process that is contributing to significant and continuous gains in local and state assessment results in schools across the country. Through collaborative inquiry, teachers learn how to use research and multiple sources of student learning data to pinpoint student-learning problems, verify causes, generate solutions, and monitor and achieve results. The session will provide an opportunity for participants to engage in data-driven dialogue using science assessment data and to gain insights into how to marshal the power of assessment data as a catalyst for school- and district-wide science improvement.
Aligning Middle School Assessment to High-Stakes Tests
Session C-12B (Room 252B), Session C-12C (Room 252B)
Marian Pasquale, Senior Research Scientist, Center for Science Education, Education Development Center, Inc., Newton, MA
Marian Grogan, Project Director, Center for Science Education, Education Development Center, Inc., Newton, MA
Framing Questions
- What different kinds of assessment formats are currently found on national and state tests?
- What information about student understanding do these different kinds of assessments provide?
- How can teachers incorporate these kinds of assessments into their classroom instruction?
Session Summary
This session is designed for middle-grades science teachers and curriculum specialists who would like to understand more about the kinds of assessments found on national and state science tests, and when and how each type can be incorporated in classroom science instruction. Participants will examine the purpose of these different kinds of assessments and the information each provides about student understanding. In addition, participants will be introduced to a framework for analyzing an assessment task, to insure that the nature and cognitive demand of the task match opportunities to learn and practice in the classroom.
An Assessment System That Works
Session C-13B (Room 206B), Session C-13C (Room 206A)
Elaine Woo, Science Program Manager, Seattle’s PreK–12 Inquiry-Based Science Program, Seattle Public Schools, Seattle, WA
Framing Questions
- What are the unique characteristics of assessments from Washington?
- How do we bridge the gap between classroom instruction, classroom assessment, and high-stakes assessment?
- How do we specifically support teachers in aligning classroom-based assessments with high-stakes tests to improve instruction and student learning?
- How do we use science writing and science notebooks to support achievement on the state science assessment as well as on classroom work?
- What are the challenges and accomplishments?
Session Summary
This session provides K–12 educators with an example of a state model for high-stakes assessment in science achievement. The presenter will share how teachers and state leaders are collaborating to align instruction and classroom-based assessments with high-stakes tests to improve instruction and student achievement. Accomplishments and challenges in bridging the gap between classroom instruction and assessment and these high-stakes assessments will be shared.
Assessing Middle School Students’ Content Knowledge and Scientific Reasoning through Written Explanations
Session C-14B (Room 254A), Session C-14C (Room 254A)
Joseph Krajcik, Professor, University of Michigan, Ann Arbor, MI
Katherine L. McNeill, Assistant Professor, Boston College, Chestnut Hill, MA
Ann Novak, Middle School Teacher, Greenhills School, Ann Arbor, MI
Framing Questions
- How do I assess students’ understanding of evidence and scientific reasoning in explanation tasks and provide feedback to promote student learning?
- How do I create everyday assessments that support students in creating scientific explanations?
- How do I determine if the explanation assessment tasks that I create align with national standards?
Session Summary
Current science education standards call for students to construct explanations of phenomena where they justify their claims with appropriate evidence and reasoning. Yet helping students write explanations and assessing the quality of these products is challenging. In this session, we will provide an instructional model of scientific explanation, including a general rubric, which can be used across different content areas and tasks. Participants will use examples of student work to gain experience in applying and adapting the rubrics to assess students’ strengths and weaknesses. We will also discuss strategies for providing feedback to students to promote student understanding. Finally, participants will design assessment tasks that align with national standards and support students in creating scientific explanations.
Introducing the Science Framework for the 2009 National Assessment of Educational Progress (NAEP)
Session C-15B (Room 207)
Senta Raizen, Director, National Center for Improving Science Education, WestEd, Washington DC
Susan Mundry, Associate Director, Mathematics, Science, and Technology Program, WestEd, Boston, MA
Framing Questions
- What are the major recommendations contained in the Science Framework for the 2009 NAEP?
- How does the 2009 Framework compare with the 1996–2005 Framework?
- What is the role of “complex” item types (such as hands-on performance and interactive computer tasks) in the 2009 NAEP Science Assessment, and what can we hope to gain from their inclusion?
Session Summary
For more than 35 years, the National Assessment of Educational Progress (NAEP) has gathered information on student achievement in selected academic subjects. This information is used to evaluate the condition of education and trends in student achievement at grades 4, 8, and 12 for the nation and, more recently, for the states and a set of large urban school districts. NAEP frameworks and reports are also increasingly used as models for other large-scale assessments and for secondary research purposes. Under contract to the National Assessment Governing Board (NAGB), WestEd and the Council of Chief State School Officers (CCSSO) developed the Science Framework for the 2009 NAEP. Collaborating organizations included the American Association for the Advancement of Science (AAAS), the National Science Teachers Association (NSTA), and the Council of State Science Supervisors (CSSS). This session will provide an overview of the Framework’s major recommendations, including science content, science practices, and item types. A comparison of the 2009 Framework with the 1996–2005 Framework will characterize the types of changes to be anticipated on the NAEP Science Assessment in 2009 and beyond. The rationale for including “complex” item types will be discussed. Participants will have the opportunity to look at illustrative items and discuss the aspects of student understanding that these items are intended to tap.
NAEP 2009: The Cognitive Demands of Science Assessment Tasks
Session C-16A (Room 203), Session C-16C (Room 204A)
Audrey B. Champagne, Professor Emeritus of Chemistry and Science Education, Department of Educational Theory and Practice, University at Albany, State University of New York
Linda Gentiluomo, Science and Mathematics Elementary Level Specialist, Schenectady City School District, Schenectady, NY
Framing Questions
- What are cognitive demands?
- Why are they important to the assessment of science literacy?
- As an intelligent consumer of large-scale assessment data, how do teachers judge the cognitive demands of items that have appeared on state, national, and international assessments?
Session Summary
The Science National Assessment of Educational Progress 2009 Framework introduces the idea of cognitive demand to science assessment. The analysis of science assessment tasks in terms of cognitive demand provides teachers insights into why students perform poorly on certain assessment tasks and what makes an assessment task difficult.
This session will provide teachers, curriculum coordinators, and administrators with the opportunity to judge the cognitive demands of items that have appeared on large-scale assessments, and to discuss how to use the idea of cognitive demand to design assessment items and to develop students’ ability to respond successfully to external assessment tasks.