Structure-function relationships are foundational concepts in physical and life science disciplines (Loertscher et al., 2014; Tansey et al., 2013; Wright et al., 2013). In the biological sciences, understanding how basic structural units influence function is identified as a foundational or core concept for undergraduate biological literacy (American Association for the Advancement of Science, 2011; Tansey et al., 2013). Helping students grasp the functional significance of molecular structures, however, is a particularly difficult instructional challenge, in part because these relationships occur at a scale that is outside the normal human experience. Johnstone (2010) recognized this problem of scale and identified three levels of chemical understanding—macroscopic and visible, molecular and invisible, and symbolic and mathematical—with these three levels at the apexes of a triangle. Experts seamlessly move among all levels, while novices struggle to make connections between levels and think only at the apexes or along the edges of the triangle. Similarly, the DNA triangle describes connections among chromosomes, DNA, and hereditary information and is used to move students toward expert thinking (Wright et al., 2017). Likewise, while experts can readily transition among genetic mutations leading to changes in protein structure and function resulting in a different phenotype, students struggle to make these connections. This example reminds educators to provide opportunities for students to practice moving among these levels of understanding so they become more expert-like in their thinking.

Improving learning through formative assessment and feedback

Formative assessment is particularly effective because it engages students by providing timely, in-class feedback to both instructors and students about learning. Frequent formative and summative classroom assessments that inform a flexible instructional approach are a hallmark of student-centered classrooms (Angelo, 1999; Handelsman et al., 2006; Huba & Freed, 2000). For example, a cyclic assessment process involves the statement of student learning outcomes, development of assessments, creation of aligned learning experiences, and use of assessment results to improve student learning (Huba & Freed, 2000). Student-centered classroom approaches that include formative assessments have been shown to improve student content knowledge and understanding (Connell et al., 2016; Maskiewicz et al., 2012), while assessment-centered classrooms allow for more student ideas to surface in order to provide opportunities to examine and revise thinking (Bransford et al., 1999). For example, student-generated reading questions have been shown to provide valuable evidence of undergraduate science students’ thinking (Offerdahl & Montplaisir, 2014).

Benefit of 3D physical models and interactive images

There is a growing body of evidence that active-learning pedagogies that include 3D, hand-held, and physical models have a positive effect on students’ understanding of molecular structure-function relationships. The benefit of physical models in the chemistry classroom has been fairly well-documented, particularly for higher-order chemistry concepts (Copolo & Hounshell, 1995; Gabel & Sherwood, 1980; Stull et al., 2012; Talley, 1973). Several studies have also examined the use of physical models in biology and biochemistry. For example, a combination of physical model use with computer 3D-like molecular imagery over several weeks increased biology students’ performance on high-order interview questions about molecular structure-function relationships (Harris et al., 2009); 3D protein model use in biochemistry courses increased learning on assessments aligned with a variety of biomolecular visualization learning goals, including structure-function relationships (Howell et al., 2019, 2020); and physical models were found to be more effective than other active-learning techniques for learning the mechanisms of gene expression (Newman et al., 2018). Additionally, after a semester-long undergraduate biochemistry class, students rated physical models as the most helpful of seven learning tools for understanding molecular structure-function relationships (Roberts et al., 2005). Interestingly, in one study, short-term use of physical models improved female students’ ability to predict how specific changes in the structure of a protein affect its function, whereas there was no effect in males using physical models (Forbes-Lorman et al., 2016).

Data suggest there is a learning benefit when students physically manipulate materials, as assessed through a meta-analysis of various instructional techniques (Wise & Okey, 1983) and more recent case studies (Giffen & Carvalho, 2015; Marcondes et al., 2015). Dynamic visualizations may particularly help students with lower spatial visualization skills (Höffler, 2010). A key additional benefit of introducing these visual, manipulative models and interactive activities is that they become shared mental models (Newman et al., 2018; Schönborn & Anderson, 2008). This provides rich, immediate, and formative feedback to instructors as they observe students’ struggles with molecular scale, size, and dynamic concepts via classroom discourse and physical interactions. The research supporting the use of both 3D physical models and dynamic interactive images aligns with the constructive philosophy of learning, in which students create their knowledge, understanding, and mental models (Lord, 1994), in this case mental models of dynamic molecular events.

Current study

The basis of this study arose primarily from our concern over our undergraduate biology students’ consistent poor performance on course summative assessments, which asked students to make complex connections among key molecular players in a well-known cell signaling pathway (WNT). We suspected students had difficulty understanding the effects of genetic mutations on protein structure and the resulting implications for complicated molecular structure and function relationships. Specifically, we observed that students struggled with (i) making mental transitions between size scales; (ii) thinking about three-dimensional (3D), dynamic structures when many of the instructional materials present static, schematic images (such as those shown in Figure 1) that can be deceptively simplistic (Linn et al., 2010); and (iii) visualizing dynamic interactions between structural components that impact function.

The WNT pathway is exemplary for illustrating dynamic structure-function relationships. This pathway regulates important developmental and differentiation processes; however, disruptions in the WNT pathway can lead to cancer when deregulated (Figure 1). In our course, students learn this pathway as part of a multiweek unit on colon cancer. They read a primary research paper describing the location of mutations found in colon cancers that lead to deregulation of β-catenin, a key WNT pathway protein (Morin et al., 1997). Before our study began, only 17% of students demonstrated a basic understanding of the consequences of different genetic mutations. Our goal was to increase student achievement of key learning objectives, demonstrating their understanding of how genetic mutations affect protein structure and function.

We introduced new curricular materials: an interactive WNT pathway activity (Figure 2) and 3D physical models (Figure 3). These curricular materials were designed to provide appropriate cognitive scaffolds for students to understand the dynamic molecular interactions in the WNT pathway. We hypothesized that these curricular revisions would result in improved student understanding, measured by performance on our standard summative assessments. Early in our study, classroom observations of students interacting with our new curricular materials served as crucial formative assessments that informed subsequent use of our new curricular materials. In this article, we describe how repetitive formative assessments and student feedback influenced our use of the 3D models and the interactive WNT pathway activity. Specifically, we detail how five assessment cycles resulted in positive improvements in student performance on summative assessments (Figure 4).

Methods and data

Study sample and program curriculum

Study participants were students in the Biology Core Curriculum (Biocore), a 4-semester integrative undergraduate honors biology program that emphasizes group learning and the process of science (Batzli et al., 2018). Students typically apply to the program as freshmen and begin as sophomores. Biological Interactions is the capstone course in the program sequence; thus, the majority of students are juniors. Students attend three 50-minute lectures and one required 50-minute discussion section per week. Three faculty members each teach a multiweek unit to the entire group of 70 to 90 students, and discussion sections are led by a graduate teaching assistant (TA) for groups of 12 to 15 students. This study was approved by the University of Wisconsin-Madison Institutional Review Board.

The focus of the present study is the unit on hereditary cancer susceptibility. One course goal is for students to apply what they learn from primary literature to understand specific mechanisms that explain how gene mutations result in altered protein structure and function, as well as how cancer develops as a result. Students read a primary research paper describing the location of mutations found in colon cancers that lead to deregulation of β-catenin (Morin et al., 1997). This primary research article and a group worksheet were used all 5 years, and additional instructional materials were incorporated over the following 4 years. Table 1 summarizes the curricular materials used over the assessment cycles.

Year 0 (2012) represents the year prior to the introduction of the new instructional tools. Two curricular innovations were added in Year 1. The interactive WNT pathway activity (Figure 2) was suggested by a team of four former students, who helped instructors develop it. The student team was also involved in designing physical molecular models of β-catenin, an important protein in the pathway. In Years 1 and 2, physical models (Figure 3) were used during lecture class meetings, while the interactive WNT pathway activity (Figure 2) was used by students in small groups during discussion meetings. In Years 3 and 4, this was reversed: Students used physical models in discussion and the interactive pathway activity during lecture time. Note that in Year 3 students used either physical models or pictures of these models during discussions in our attempt to investigate the specific effect of the physical models. A regression analysis determined that there was no difference in performance between students who used the physical models and those who used pictures of the models (p > 0.4; data not shown).

Summative assessments

Each year of this study, the unit culminated with a short-answer essay exam. Exam questions were aligned with the following learning objectives: (i) Diagram and explain the WNT signaling pathway under various signaling conditions, (ii) describe the interaction of β-catenin and the tumor suppressor protein APC in a normal cell, (iii) describe the effects of loss of APC function on β-catenin expression and function, (iv) outline the potential effects of loss of APC on a cell or group of cells, (v) describe the potential effects of mutations in the β-catenin gene on cell function, and (vi) compare and contrast the types of mutations and the results of these mutations on cell functions. The wording of the exam questions was modified slightly in Year 3 in an attempt to guide the students to address key mutation-function relationships that were missing in earlier iterations.

The exam was scored independently by two reviewers using a detailed rubric with a maximum of 9 points. A basic understanding corresponds to at least 5 out of 9 points on the exam questions assessing students’ achievement of the learning objectives related to β-catenin and WNT signaling. To assess a deeper understanding of the material, we also recorded the number of instances that students answers included the following details: (i) phosphorylation state of β-catenin, (ii) interaction with GSK (the kinase that phosphorylates β-catenin), and (iii) whether β-catenin is in cytoplasm or the nucleus. There was a high degree of inter-rater reliability (Pearson’s correlation = 0.94), so the average of the two scores was used.

Performance on exam questions related to β-catenin structure and function increased over the 5 time points (F[4,374] = 34.8, p < 0.001; Figure 5A). Average scores on β-catenin content increased from 3.6 in Year 0 to 4.3 in Year 1 (not significant) to 4.8 in Year 2 (not significant compared to Year 1, p = 0.02 compared to Year 0) to 6.5 in Year 3 (p < 0.001 compared to Year 2) to 7.6 in Year 4 (p = 0.03 compared to Year 3).

While the average score on all exam questions is significantly different over the 5 years (F[4,374] = 11.3, p < 0.001; Figure 5B), scores did not consistently improve over these years. Average exam score was 80.5% in Year 0, 75.9% in Year 1 (p = 0.01 compared to Year 0), 74.8% in Year 2 (not significant compared to Year 1), 78.5% in Year 3 (p = 0.048 compared to Year 2), and 83.1% in Year 4 (p = 0.01 compared to Year 3). These data indicate that student performance in the course did not consistently improve over the 5 time points of the study, while performance on questions aligned with the targeted structure-function learning goals did.

Formative assessments

Prior to the introduction of the curricular materials, formative assessment was minimal. When the new curricular materials were introduced in Year 1 in a large lecture setting, the instructor observed that many students seemed confused by the physical models as they tried to follow her verbal directions. Specifically, the instructor noticed that students struggled during lecture to use facts they had learned about a structural mutation to identify a specific region on the 3D physical model.

In Year 2, during the same 3D physical model activity in lecture, the instructor noticed that students had great difficulty when asked to translate features of a 2D schematic image of a protein from Morin et al. (1997) onto the 3D physical model of that protein. The instructor realized that simply giving the students a set of tools did not mean students knew how to use them. Indeed, one student said, “The models were not the most helpful in remembering the concepts; they were sort of distracting.” The instructor realized she had to model for the students how to make explicit connections between the materials provided.

These observations informed the instructor’s decision in Year 3 to move the 3D physical models into the smaller discussion sections (maximum of 16 students), which were led by the instructor. The instructor also converted the WNT signaling puzzle activity into a lecture class group activity that alternated between students working in small groups and the whole class, with the instructor using a larger version of the puzzle pieces to have student groups summarize the various WNT signaling pathways and how mutations can lead to cancer (Figure 2). This provided key formative assessment information to the instructor and allowed her to give specific feedback to the whole class. We believe this switch of physical models into the smaller discussion sections and the WNT signaling puzzle activity into a lecture class group activity was a key turning point in this study.

However, one-on-one interactions between students and instructors while using the 3D physical models during Year 3 discussions revealed students’ continued difficulties with the curricular materials. These observations prompted the instructor to further refine instructions for working with models in Year 4. This included (i) revising the description of the model in the study guide (Online Appendix) so students would come to discussions with a better understanding of the material, and (ii) making more explicit connections during discussion to a schematic figure from a published paper in the study guide. In Year 4, most students demonstrated that they could translate features of the schematic protein image onto the physical model.

Student self-reported SALG data

In Years 2 and 3, students answered survey questions about their perceived achievement of learning goals and helpfulness of curricular materials using customized Self-Assessment of Learning Gains (SALG) surveys. Implementing SALG is a part of our iterative cycle of formative assessment. That is, after the first couple years of the study, it became evident that we needed more information on the students’ perceptions of their learning.

Ninety-eight percent of students responded in Year 2 and 100% responded in Year 3. In Year 3, the SALG questions were revised to include more precise language aligned with learning outcomes, which coincided with our curricular approach shift to using the 3D physical models or pictures of the models in discussion sections and the interactive WNT pathway activity in lecture (Tables 1 and 2).

Although we asked different questions over the 2 years, the responses indicate that students believed they understood how genetic mutations are related to cancer both years (Figure 6). Specifically in Year 2, the majority of students reported either a “good gain” or “great gain” in their understanding in response to the following prompt: “As a result of your work in this class, what gains did you make in your understanding of how to compare and contrast the types of mutations and the results for the cell of mutations in tumor suppressor genes and oncogenes?” In Year 3, however, the majority of the students reported that they either “mostly understand” or “understand very well” in response to the following prompt: “Presently, I understand the types and consequences of mutations found in tumor suppressor genes and oncogenes in cancer.”

Additional SALG responses showed that student perceptions of the helpfulness of the physical models improved from Year 2 to Year 3, suggesting that the students viewed the 3D physical model more favorably as an effective curricular tool when it was used in discussion sections. Specifically, in Year 2, 39.2% of students indicated that the 3D physical model used in lecture was of great help or much help “to get a better understanding of the protein and its interactions,” while in Year 3, 51.6% of students who used the 3D physical models in the discussion section rated them as fairly or very helpful to their “learning of the genetic basis of disease.”

Furthermore, students considered the interactive WNT pathway activity to be more helpful when it was used during a guided group activity during lecture. In Year 2, 60.3% of students reported the interactive WNT pathway activity during discussion as being of great or much help in “getting a better understanding of the protein and its interactions.” While Year 2 students rated lecture as the most helpful aspect of class, the interactive WNT pathway activity was rated as the second most helpful. In Year 3, after the interactive WNT pathway activity was moved to lecture, 83.1% of students viewed it as fairly or very helpful to their learning of “the genetic basis of disease.” In Year 3, the interactive WNT pathway activity was ranked nearly equal to attending lectures in terms of how helpful it was.

Students’ responses to open-ended questions in Year 3 (Table 2) also help reveal what students found valuable about the curricular tools. Note that students were invited to consider the entire unit curriculum, and their responses indicated that they appreciated a wide variety of tools and activities such as study guides, weekly quizzes, and group work. The quotes in Table 2 were chosen because they provide insight as to how the physical protein model and interactive WNT pathway activity helped students connect molecular structure (i.e., how specific genetic mutations resulted in β-catenin protein structural differences) to cellular function (disruptions in WNT cell signaling ultimately resulting in cancer).

Discussion

Prior to our study, most students failed to connect genetic mutations to altered protein structure and its function in a key cell signaling pathway. We predicted that introducing a 3D physical model of a key protein in combination with an interactive WNT pathway activity would improve student performance on summative exam questions assessing these concepts. This prediction was supported, but only after several iterations of curricular innovations informed by formative assessments spanning several years. Our data indicate that students’ deep understanding of a challenging concept is positively affected by the careful implementation of 3D physical models and an interactive WNT pathway activity. SALG data further corroborate student perceptions of learning and indicate that our curricular tools helped students develop the mental scaffolds necessary for understanding molecular structure-function relationships. The 3D physical models and the interactive WNT pathway activity supported the development of shared mental models, which allowed for rich formative assessment opportunities during class time because they made student thinking and voice visible to the instructor (Babilonia-Rosa et al., 2018; Newman et al., 2018; Schönborn & Anderson, 2008).

Johnstone’s (2010) triangle of chemistry concepts is readily adapted to a protein structure and function triangle, with apexes of DNA sequence, protein structure and function, and phenotype. At the onset of this study, students were not thinking like experts. Instead, they could explain that changes in DNA sequence (mutation) led to cancer (phenotype), but they appeared to have limited understanding of the third piece of the triangle: how mutations in DNA impacted protein sequence (and therefore structure and function) and how those changes produce the phenotype of cancer. Our curricular materials were designed to specifically address a lack of appropriate cognitive scaffolds for students to understand dynamic molecular structure and interactions and to help students make mental transitions from molecular to organismal scales; think about 3D dynamic structures when they see schematic 2D images; and visualize dynamic interactions between the β-catenin protein and key components in the WNT cell signaling pathway, resulting in uncontrolled cellular growth. In the process of incorporating these materials into the class, we uncovered a number of gaps in the students’ knowledge and skills in understanding figures from the manuscript, interpreting models, and comprehending dynamic molecular interactions. These learning challenges are similar to those observed when students connect DNA structure, chromosome structure, and cellular functions, depicted as the DNA triangle (Newman et al., 2012).

We documented an increase in the proportion of students demonstrating basic understanding over all 5 time points measured. Students demonstrated the greatest level of understanding after 4 years of curricular revisions and when 3D physical models were used in all discussion sections. The environment in which these curricular materials were used was also important: The greatest increase within a single year (between Years 2 and 3; see Figure 5A) occurred when 3D physical models (or pictures of models) were used for the first time in discussion sections and the interactive WNT pathway activity was used in lecture for the first time. Furthermore, student reports in their learning (Figure 6) also indicated that these changes were effective.

We also assessed student understanding at a deeper level by evaluating whether students addressed key details related to β-catenin: phosphorylation status, interactions with GSK, and cellular location. Between Years 0 and 1, there was a large increase in the percentage of students identifying one key detail. The largest increase in the proportion of students identifying two key details happened in Years 3 and 4, when the 3D physical models were used in discussion and the interactive WNT pathway activity was used in lecture. Thus, models and interactive activities, when used in optimal class settings, contribute not only to students’ basic understanding but also to their deeper understanding of key details in the regulation of cell signaling.

While it is possible that the improvements in summative assessments are in part due to changes in the wording of questions, our data indicate that using 3D physical models in discussion section, where instructor-to-student ratios are highest, helped students most. Observations of students’ struggles during lecture to comprehend the 3D physical models in Year 2 resulted in moving the models to discussion sections in Year 3. This change helped us uncover specific obstacles that were revealed by observing dialogue as students interacted with the physical models in the discussion sections. One student in Year 2, reflecting on how class activities helped their learning, supports the instructor’s suspicions that the implementation of 3D physical models during lecture was problematic: “I didn’t really know what was going on during the physical model activity because it wasn’t explained that well, but once I went back after class to look more at the material it was nice to use the activity as a mental reference.” These observations improved subsequent implementation of the curricular materials.

Similarly, the use of the interactive WNT pathway activity in lecture helped the instructor realize that students did not understand the depth of detail in the manuscript figure, course discussions, and so on. Students got the opportunity to struggle with the pathways in small groups, then report to the whole class, with feedback and questions from the instructor. As with any novice learner (Chi, 2006), the students had been unable to integrate new information into their undeveloped schema of complex genetic pathways, which prevented them from completing a full picture of the pathways and the dynamic roles of the pathway components, so they did not perform well on summative assessments. Expert-novice studies remind us that instructors must unpack their own metacognitive process of integrating new ideas, including how they use visuals to categorize and organize information, in order for students to begin to use these same skills on their own. In the following 3 years, the instructor integrated other strategies to better implement the models and puzzle activity, and student performance increased significantly during these subsequent years of our study.

The student comments from the SALG data indicate that the two curricular tools supported student learning in distinct ways. The 3D models helped students visualize key molecular structural details impacting cellular function (e.g., common binding site for two regulatory proteins and the fact that this binding site was physically separated from regulatory phosphorylation sites). These nuanced structural details were not sufficiently depicted in the two-dimensional (2D) schematic images shown in textbooks. Student reflections highlight how their manipulation of WNT signaling components during the puzzle activity helped them visualize and contrast interactions between β-catenin and regulatory molecules, then connect this to genetic mutations that lead to uncontrolled cancerous growth.

With the introduction of our new curricular materials, we needed to identify the best delivery venue (discussion or lecture), as well as appropriate scaffolding for students to know how to use the new materials. As is true with all types of models, including graphs, tables, diagrams, and molecular models, students also need practice in interacting with materials and transitioning among various scales of knowing to become proficient at “thinking inside the triangle,” or thinking like an expert (Dries et al., 2017; Schönborn & Anderson, 2009, 2010) .

Formative assessment involves students being given strategic guidance on how to improve their work (Clark, 2011). In this case, student interactions with physical models and dynamic interactive activities provide a unique insight into student thinking. This valuable formative assessment, providing student voice and perspective, then drives curricular change to improve student learning. The curricular materials allow the instructor to see what the students are thinking. Our findings indicate that the unique formative assessment made possible by observing students’ interactions with physical models and dynamic interactive activities is part of an effective student-centered teaching approach that science educators can use to help students address challenges in understanding molecular structure-function relationships. Table 3 describes formative assessment strategies that we found improved our implementation of instructional materials and thus helped our students overcome learning challenges related to structure-function relationships.

Conclusion

Our data show that our curricular revisions resulted in improved student performance on summative assessments. Perhaps more important, our study exemplifies how the effective implementation of novel curricular materials is an ongoing process. Our successful active-learning approach emerged from our iterative assessment cycle of examining summative assessments, implementing curricular innovations, carrying out formative assessments, and using findings from the formative assessments to further improve curriculum and pedagogy.

This resulted in significant improvements in students’ ability to understand molecular structure-function relationships. Successful implementation of new curricular materials requires an investment of time and energy, particularly as more teaching materials are shared among instructors (e.g., CourseSource). In other words, promising materials that are poorly implemented do not improve learning.

Our challenge as STEM educators is to integrate more effective formative assessments instead of solely relying on exams (summative assessments). The assessment strategies outlined in this study provide further evidence of the value of student-centered assessment cycles and classroom discourse that allows students to evaluate their own learning. We believe that our curriculum evolved to be particularly effective in helping students access expert levels of understanding regarding troublesome concepts such as dynamic molecular interactions. Indeed, another Biocore instructor has recently created and implemented a similar dynamic activity. Our data demonstrate the importance of the systematic use of formative assessments that can inform evidence-based practices to provide undergraduates with better access to complex scientific concepts and build even deeper critical-thinking skills. We encourage educators to apply these pedagogical strategies to other similarly complex pathways.

Acknowledgments

We thank Janet Batzli, Jeff Hardin, and the University of Wisconsin (UW)-Madison Biocore Program for supporting this work. We thank Olivia Harris and Mark Hoelzer for the Figure 4 graphics. We thank the students who worked to develop the materials for this unit for their insight and willingness to point out what concepts were challenging for them: Jim Heffernan, Ben Hierlmeier, Katie Strobel, and Devan Van Lanen-Wanek. This material is based on work supported by the National Science Foundation under award numbers DUE-1022793 and DUE-1323414. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation. The UW-Madison Education Research Institutional Review Board approved this research protocol (protocol 2012-1060).

Robin Forbes-Lorman (forbesr@ripon.edu) is an associate professor in the Department of Biology at Ripon College in Ripon, Wisconsin. Michele Korb (michele.korb@csueastbay.edu) is a professor in the Department of Teacher Education and Science Education at California State University, East Bay, in Hayward, California. Amy Moser (armoser@wisc.edu) is a retired associate professor in the Department of Human Oncology at the University of Wisconsin-Madison School of Medicine and Public Health in Madison, Wisconsin. Margaret A. Franzen (crestprogram@gmail.com) is retired program director of the CREST Program in the Center for BioMolecular Modeling at the Milwaukee School of Engineering in Milwaukee, Wisconsin. Michelle A. Harris (maharris@wisc.edu) is an emeritus distinguished teaching faculty associate in the Biology Core Curriculum (Biocore) Program at the University of Wisconsin-Madison in Madison, Wisconsin.