College instructors teaching large-enrollment courses face many challenges when trying to engage in formative assessment of student understanding in the classroom. Even when they actively involve students in collaborative group work where student thinking becomes more visible, physical and time constraints limit instructors’ ability to collect evidence of student understanding that can help them make better instructional decisions. Many instructors who implement evidence-based teaching strategies in large classrooms rely on learning assistants (LAs) to scaffold student learning. However, the focus of these LAs is often directed at helping students complete the learning tasks, and the evidence of student thinking that they may collect is rarely communicated to the instructor.

Formative assessment has been shown to be a critical activity for promoting meaningful understanding in the classroom (Bennett, 2011; Sadler, 1989). It allows instructors to elicit student thinking, build inferences about student understanding, and make decisions and take actions to guide instruction on the basis of actual evidence (Bell & Cowie, 2001). Engaging in formative assessment systematically helps instructors not only to develop a clearer picture of where students stand in relationship to the learning objectives of a course, but also to design learning tasks that can better help students meet the learning goals.

Our project team has designed and implemented an instructional model that seeks to actively engage instructors in formative assessment with the support of an undergraduate student trained to collect evidence of student understanding in the classroom. We call this highly specialized individual a learning researcher (LR), as her or his main tasks include gathering and analyzing data about student thinking during classroom activities and then generating reports for the instructor that inform subsequent planning as well as instructional decisions and actions. In this article, we describe the LR role and present a qualitative analysis of its impact on formative assessment thinking and practice.

Our project is being implemented at a large research-intensive university in the Southwestern United States. In recent years, the number of instructors engaged in teaching STEM (science, technology, engineering, and mathematics) courses in large (>80 students) collaborative learning environments (CLEs) at our institution has increased considerably (Burd et al., 2015). With the support of the National Science Foundation (NSF-IUSE Program), we have embarked on a project designed to develop an instructional-teams model and professional-development program to support instructors teaching in large CLEs. Our project aims to create and support cohesive instructional teams that can design and implement high-quality learning tasks through effective communication among team members and continuous opportunities for formative assessment. Figure 1 depicts and describes the main components of our instructional-teams model. In this contribution, we focus our attention on one of its core elements: the LR.

Instructors teaching in large, collaborative learning environments must negotiate diverse competing demands: Student work in multiple small groups has to be organized and properly scaffolded; the work of many LAs has to be coordinated and monitored; and class time and student participation need to be properly managed. With so many issues calling for their attention, most instructors struggle to effectively engage in formative assessment of student understanding (Heritage, 2013; Wiliam, 2011).

Seeking to open more opportunities for instructional teams to engage in formative assessment of student understanding, we decided to explore the effects of introducing the role of an LR in our instructional-teams model. An LR serves as an observer and explorer of student thinking in real time during classroom activities. Through careful observation of student work, listening to students’ conversations, and asking exploratory questions, the LR creates a daily written report of what is actually happening as students engage in the designed learning tasks. These reports are shared with the instructor who analyzes and uses them to inform planning and instructional decisions. Although instructors are not the individuals collecting the formative assessment evidence directly, their interactions with the LR and the analysis of their reports positively affects teaching decisions and improves instructors’ capacity to plan and use formative assessment strategies.

Most LRs in our project are undergraduate students who have successfully completed the course they assess and have served as LAs for the instructor in that same course. They are thus familiar with the course content, have experience helping students learn the material, and have built a working relationship with the instructor. The LRs receive specific training through a 1-hour weekly meeting focused on both formative assessment and how to adopt a research mindset in the classroom. In these meetings, the LRs are

• introduced to, discuss, and practice different formative assessment strategies to collect data on student understanding;
• engage in interpretation of the formative assessment data;
• and practice creating productive reports that clearly communicate their observations, interpretations, and suggestions for actions to foster student learning.

Particular emphasis is put on developing the LRs’ ability to notice the substance of student thinking through the analysis and discussion of conversations among students and between students and LAs. As part of the weekly-meeting activities, LRs engage in noticing student thinking and interpreting it through the analysis of videos of diverse types of interactions in actual classrooms. The LRs also read and discuss samples of exemplary LR reports that demonstrate sophisticated levels of noticing, interpretation, and suggestions for action.

LRs are trained to build daily formative assessment reports in which they are expected to briefly describe the learning task(s) they observed, the associated learning objectives (explicit or inferred), and the perceived level of difficulty of these task(s) (difficult/medium/low). The core of the report includes a summary of what the LRs noticed while students worked on the learning tasks, their interpretations of what their observations mean in terms of student understanding based on the evidence they have, and recommendations to foster student learning of the concepts and ideas targeted in the class. LRs are given a report template to guide their work (see Figure 2).

In the past 2 years, we have worked with 27 different LRs placed in 20 classrooms from a variety of STEM disciplines, including general chemistry, chemical engineering, introductory biology, and environmental science. Excerpts from LR reports, representing the range of written accounts generated by our project participants, are included in Figure 3. The quality of these reports varies, particularly at the beginning of the semester, but improves with training and experience. Novice LRs often focus their attention on students’ behaviors rather than on the substance of students’ ideas as expressed in the different products (e.g., written explanations, diagrams, drawings on whiteboards) that students generate when working on a task. Novice LRs tend to take an evaluative approach toward student work, describing whether students got the correct answers, rather than adopting an interpretive stance and analyzing the challenges students face to learn from the activity (Talanquer, Bolger, & Tomanek, 2015). Initial reports frequently lack specificity, and claims are presented without evidence to back them up. However, as the LR training and experience in the classroom progress, most LRs learn to focus their attention on student thinking, generate insightful interpretations about student understanding supported by evidence, and make productive suggestions to foster student learning.

To evaluate and better understand the role of the LRs in STEM courses and their impact on teaching practice, we have collected data using a variety of methods, including the gathering of LR written reports and pre- and post-semester interviews with LRs and lead instructors. The analysis of these data has allowed us to identify important traits that individuals in the LR role should possess to be most effective and productive in their work. The most effective LRs have been students who have a keen interest in understanding how people learn and what factors foster or hinder student learning. They tend to be students with a solid understanding of the concepts and ideas discussed in the class and a clear sense of core learning objectives. Our most successful LRs have been undergraduate students who have served as LAs for that same course in the past, or who have experience tutoring students in diverse contexts.

Given that LRs are expected to collect evidence of student understanding using different strategies in real time in the classroom, they should be interested in listening to students’ conversations and feel comfortable asking questions to further explore student thinking. One of the major challenges faced by LRs has been to curb their tendency to act as “helpers” who support students as they work on tasks rather than to focus their attention on collecting evidence of student understanding. Switching from a teaching mindset to a research (formative assessment) mindset has been a major challenge for some of our project participants.

Successful LRs also have excellent communication skills, both verbal and written. The written reports are the main tool that LRs use to communicate their observations, interpretations, and suggestions to the instructor. The LR report must be carefully crafted to ensure that the information is conveyed clearly and tactfully. LRs are expected to generate an objective report that avoids making evaluative judgments about the performance of specific students, instructional team members, or the instructor. LRs should also be able to verbally communicate the results of their work during the weekly course team meetings and confidently present their ideas and suggestions to all other members of the teaching team.

The work of the LRs cannot be effectively performed without the full support and trust of the instructors who create the space for the LRs to do their job, provide feedback on the LRs’ formative assessment reports, and remain open and responsive to the LRs’ comments and suggestions. Our experience working with 22 different instructors suggests that some of them initially struggle to find value in the LR reports. Although they are certainly interested in monitoring student understanding, many of them are not used to engaging in making sense of student thinking and using these analyses to inform their teaching decisions and actions. Over time, however, most of the instructors engaged in our project have come to see the benefits of analyzing and reflecting on the information presented by their LR. They also recognize the value of adopting a research mindset through formative assessment and using the LR’s input to modify in-class tasks based on actual evidence of student thinking. The following quotes from some of the instructors in our project illustrate the impact of the LRs on instructors’ awareness of student thinking:

Having somebody who will look at what I am teaching, what is going on in the classroom, and then give me the feedback…it seemed to be an interesting idea…but I never expected the benefits of having that person in the classroom.…There were times in which I got a really low percent of students getting it right and she wrote what she heard around her…and I could understand why they were thinking in that way. (Instructor A)

Suggestions for how I might either introduce the topic if I was to do the task again or suggestions for the next lecture, places that I need to still spend a little time on.…They were highlighted as either medium or hard in the learning report which sort of calibrates my thinking more towards where the students are instead of where I feel it should be. (Instructor B)

In addition, instructors appreciated that their LRs opened their eyes to how differently novices in their field may think about certain concepts and ideas. Most instructors are years removed from their students’ perspective on the content areas that they are teaching. LRs frequently are in a better position to make sense of the struggles that students face to understand the content and can provide insights into student thinking that experts may overlook:

She has offered suggestions and the other thing that she has done which has been really great is remembering what she felt when she was a student in my class…she remembers struggling and she’s noticing the current students’ struggling in those same areas, which is very useful to me. (Instructor B)

LRs also benefit from their training, their experiences in the classroom, and their interactions with the instructors. They gain major insights into how people learn and recognize the importance of adopting a research mindset when exploring and analyzing student understanding, including their own. Additionally, acting as LRs allows them to gain a richer perspective on curricular and classroom environment issues. As a result of their involvement in the project, several of the LRs have expressed an increased interest in teaching as a professional career. The following excerpt illustrates the perceived impact of the LR experience in the words of one of our project participants:

Being an LR has caused me to develop a different paradigm in many aspects of life. I find myself analyzing my own understanding of subjects I am taking, trying to formatively assess myself, and find the root of how I understand specific concepts and what I should focus on while studying. It even causes me to think about effective task implementation in the other classes I’m enrolled in, and how I can improve them! I have noticed the “LR mindset” seeping into many aspects of my life. (LR 4)

The presence of an LR in a classroom increases an instructional team’s capacity for assessing student understanding, as well as for engaging in reflective and responsive teaching practices. However, a focus on student thinking has to be reinforced on a regular basis in all team members’ minds. It is not uncommon for instructors to initially center their attention on course-planning and class-management issues (e.g., designing class activities, ensuring students are on task, balancing time constraints). Similarly, LRs can be expected to initially direct their observations to management problems (e.g., students’ off-task behaviors when working in group activities) and to whether students are capable of generating the correct answers during learning tasks. Although instructors may find this information useful, it is critical to redirect everybody’s attention to the substance of student thinking. Helping all team members to adopt an inquisitive and interpretative perspective on formative assessment, rather than a managerial and evaluative one, is important to reap the benefits of the presence of an LR in the classroom.

Some instructors who have participated in our project have initially doubted that undergraduate students, who are not experts in the discipline, can successfully take on the role of an LR. This hesitation is often ameliorated when the LR has served as an LA for the instructor in prior semesters for the same class. However, we have learned that it is of central importance to cultivate a strong relationship between instructors and LRs based on mutual respect for their knowledge, experiences, and opinions. Instructors need to recognize that LRs can contribute valuable insights into student understanding and that their observations are not a judgment of the faculty members’ teaching skills. On the other hand, LRs need to feel comfortable communicating with the instructor and justifying their observations and suggestions with the evidence they have.

The novel role of the LR in our project is innovative in that it embraces students as major contributors in the development of learning environments in which evidence of student thinking is constantly used to guide practice. It is well-known that collaborative feedback improves a teacher’s planning, decision making, and acting in the classroom (Fujii, 2016; Lewis, 2002). In our project, LRs provide invaluable information to instructors, offer productive suggestions for improvement, and direct their attention to the analysis of students’ ideas and their impact on learning. In our university context, using undergraduate LRs who are capable of sophisticated engagement in formative assessment is a more sustainable model than the use of postdoctoral teaching mentors as implemented in other projects (Wieman, 2017).

Despite our focus on large-enrollment classes, we believe that the LR role introduced in this article could improve teaching practices in diverse classroom contexts. The LR’s attention to formative assessment has improved responsiveness to students’ ideas, learning-task design, and adoption of evidence-based teaching practices in multiple classes across diverse STEM disciplines at our institution. It has also helped transform the views of many individuals, including instructors, LRs, and other members of the instructional teams involved in our project, who now look into classrooms through the lenses of student learning and are better prepared and disposed to approach teaching with a student-centered perspective.

Acknowledgments

We want to thank all students and instructors involved in our project. Without their interest, effort, and enthusiasm to improve STEM education, our project could not exist and move forward. This work is supported by the National Science Foundation grant DUE-1626531.

Young Ae Kim (youngaek@email.arizona.edu), Jonathan Cox, and Katelyn M. Southard are research scientists in the Department of Chemistry and Biochemistry at the University of Arizona in Tucson. Lisa Elfring is Head of the Office of Instruction and Assessment, Paul Blowers is a Distinguished Professor in the Department of Chemical and Environmental Engineering, and Vicente Talanquer is a Distinguished Professor in the Department of Chemistry and Biochemistry, all at the University of Arizona in Tucson.