Video has become an increasingly popular educational medium, widely used in both traditional and flipped courses. When instructional videos clarify course content, students respond favorably and report having learned from them (Dawson & Van Loosen, 2012; Kay, 2012; Kay & Kletskin, 2012). Students in college classes often watch instructional videos multiple times for exam preparation (Richards-Babb et al., 2014). Use of video in preclass assignments can enhance motivation as indicated by boosts in class attendance (Stockwell et al., 2015). Other learning gains have been associated with the use of video, including better conceptual understanding, mastery of complex problem solving, and improved laboratory preparedness (Dupuis et al., 2013; He et al., 2012; James et al., 2013; Stieff et al., 2018; Jolley et al., 2016).

In contrast with older forms of multimedia learning tools, advances in technology have all but eliminated the technical hurdles involved in the production, dissemination, and use of educational videos. Videos may be produced by the course instructor, by groups of students as course projects, or by other amateur or professional developers who share their videos publically on YouTube and other social media platforms. The downside is that the quality of the content of educational videos is highly variable, and videos may fail to engage viewers or enhance learning and may even increase learners’ confusion (Guo et al., 2014; Hill & Nelson, 2011; Tversky et al., 2002). To maximize their effectiveness, the relevant scholarship on learning should inform the design of educational videos.

Three general areas of research provide vital lessons. First is research on the use of video in various course settings, including traditional courses, flipped courses, and Massive Open Online Courses (MOOCs). This literature reports on instructional use of videos, students’ attitudes, and qualitative and quantitative impacts on learning, but less so on the learning impacts of specific design decisions (e.g., Jolley et al., 2016; Richards-Babb et al., 2014). Research on best practices for designing animations and other visuals and combining and sequencing audio and visual information is found in the body of work on multimedia learning, which has a considerably longer history than YouTube, MOOCs, and the current surge in educational video-making (e.g., Hegarty, 2005; Moreno & Mayer, 1999). Finally, video design should draw on relevant lessons from the broader research base on how people learn, including both cognitive and affective factors (e.g., Johnstone, 1991; NRC, 2000).

Instructors who wish to develop and work with videos, but who are not educational technology specialists, need guidance in the form of a synthesis of the literature with clear recommendations for practice. Helpful recent reviews exist, but these devote much attention to how the videos should be integrated into the classroom (Brame, 2016; Prud’homme-Généreux et al., 2017). These are important considerations, but given the myriad and unpredictable ways in which a given video may be used, it is important to evaluate a video as a self-contained curricular building block. Thus the design of the video itself deserves specific attention. Here we synthesize and operationalize the lessons from scholarship in the form of a concise checklist. This instrument can be used to design and assess the strengths of produced videos as well as scripts and storyboards.

Supported by a National Science Foundation Improving Undergraduate STEM Education grant, our interdisciplinary team (the authors, representing mathematics, physics, chemistry, and biology) has been collaborating to develop a set of videos on rate of change concepts in introductory mathematics, physics, chemistry, and biology curricula. As we embarked on writing and editing scripts and producing the videos, it quickly became clear that our work would benefit from a guiding document informed by the relevant literature base. The creation of this instrument began as a set of questions that synthesized our collective knowledge. As we refined the questions, they grouped naturally into categories of design decisions. As we applied the questions to develop and critique scripts, it became clear that a checklist format would be more convenient than a list of questions. After several phases of iteration, we developed a one-page checklist with 12 items that fall into three categories of design decisions: (A) Content and sequencing, (B) Cognitive supports, and (C) Affective considerations (see Figure 1). Each checklist item is supported by scholarship, as we now describe.

Concepts. Students come to learning situations with prior knowledge, which is often a combination of scientifically normative ideas and misconceptions (NRC, 2000). Although debate is ongoing about the exact form of students’ prior ideas, there is strong agreement that they need to be considered in the process of curriculum design, and that the goal should be to help students integrate their prior knowledge with the new information taught (diSessa, 2014; Duit & Treagust, 2012). Videos can help students make links between prior knowledge and new concepts (Mitra et al., 2010). When research has identified commonly held misconceptions about a STEM topic, learners benefit from instructional videos that specifically address those misconceptions (Muller et al., 2007).

Logic. Research on learning progressions or learning trajectories has combined empirical studies of how students’ knowledge of science or mathematics topics develops over time with content analyses of the disciplines themselves, with the aim of informing appropriate curriculum sequences (Duschl et al., 2011). Although any individual video can only address a narrow portion of a learning progression, the content of the video should still be consistent with overall learning progression for a topic, based on grade or age. Gaps in logic are common in textbooks (Seethaler et al., 2017). Videos should avoid them, and may be especially useful if they address gaps in traditional curriculum materials.

Story. Narrative formats can increase comprehension and engagement (Dahlstrom, 2014). This suggests that video designers should consider incorporating narrative, but it would be overly limiting to suggest that all educational videos should follow one particular format, because various formats of videos can promote learning (Guo et al., 2014). Nevertheless, ample research indicates that students are more likely to learn when they are engaged with a specific question or problem (Bybee, 2014). Video designers should endeavor to construct an intellectual story beginning with a puzzle that is resolved by the end of the video. Such a video can be brief while still encompassing a story arc.

Language. With respect to language, research supports “personalization” and “content first.” Mayer’s personalization principle is supported by research that shows students learn better from multimedia lessons when the narrative is conversational (i.e., using first and second person and directing comments at the viewer) rather than formal in style (Mayer et al., 2004; Kartal, 2010). Content-first approaches to teaching are supported by research demonstrating that academic language is a significant hurdle in science learning and that students perform better when scientific concepts are introduced before the relevant technical terms are presented (Brown & Ryoo, 2008; McDonnell et al., 2016).

Visualizations. Many aspects of our world are too large or too small to be seen with the naked eye and many processes occur on too long or short a timescale for us to observe them directly. Science visualizations have a long history in science education in helping students understand microscopic, submicroscopic, and astronomical structures, physical phenomena, and complex systems (Eick & King, 2012; Linn, 2003). Dynamic visualizations such as animations can support learning by portraying things that static visualizations cannot (Tversky et al., 2002). Dynamic visualizations can also address specific misconceptions and close gender gaps in understanding (Yezierski & Birk, 2006). Visual attention limits what can be learned from an animation, particularly when changes are happening simultaneously in different parts of the animation (Hegarty, 2005). Visualizations should therefore take advantage of the affordances of the video medium to make science visible to students, while taking care to avoid overwhelming visual attention.

Signals. Moving between the macroscopic and microscopic or submicroscopic worlds, and between representations (in the form of words, graphs, and symbols), is a significant challenge in science learning (Johnstone, 1991; 1993). Novice learners may not perceive key features of representations without assistance, and may focus on design elements instead of the underlying concepts (Cook, 2006; Tasker, 2016). For learners to create referential connections between representations, instructors and instructional materials should make these links explicit through the use of cues, including arrows, highlights, and verbal guidance. Variations, such as in color, size, or motion cue learners to focus on the varying features, which means designers must take care to avoid inadvertently emphasizing unimportant features (Bussey & Orgill, 2015). For example, video developers should take heed of studies of traditional curriculum materials, which have demonstrated that use of (static) arrows is often inconsistent and confusing to students and that students need support to interpret them (Wright et al., 2017).

Synchronization. People learn better from words and pictures than from words alone. Words and graphics should be presented concurrently, rather than successively, to help learners build connections between them to be stored in long-term memory (Moreno, 2006). However, the visual channel can become overloaded when both words and images need to be processed through it. Thus, presenting graphics with narration supports learning better than presenting the words as written text, as spoken words can be processed through the auditory channel while the visual channel is free to process the images (Mayer & Moreno, 2003; Moreno & Mayer, 1999). Likewise, multimedia formats that present slides and a separate window showing the speaker can split viewers’ visual attention and interfere with learning; videos should only require one visual focal point at a time (Chen & Wu, 2015). That being said, for accessibility, the use of captioning should be available.

Segmentation. Information should be divided into temporal segments that learners can digest one at a time before moving on (Mayer & Moreno, 2003). An often-cited study of student engagement with MOOCs recommends that educational videos be no more than six minutes in length (Guo et al., 2014). MOOCs, however, are a unique educational environment because enrollment is typically not for credit. In a study of problem-solving videos in undergraduate chemistry courses, in which the videos averaged 10 minutes in length (range 2–28), engagement with the videos was high, feedback overwhelmingly positive, and nearly 10 times as many students (32.5%) wanted videos with additional problems versus recommended (3.9%) decreasing video length (Richards-Babb et al., 2014). A one-size-fits-all recommendation on video length is thus unwarranted; instead, video length and pacing should be carefully contingent on the amount and complexity of information covered.

Streamlining. Cognitive load theory is a key theoretical underpinning for the design of multimedia (Sweller & Chandler, 1994). The cognitive load imposed by a video should be germane, and extraneous processing—that which does not serve instructional objectives—should be minimized (Mayer & Moreno, 2003). For example, irrelevant sound or music can impede learning (Moreno & Mayer, 2000). Overly long explanations, and conflicts between written and spoken words that occur when text-heavy slides compete with narration, are also detrimental to learning (Mayer et al., 1996; Moreno, 2006). Thus, video content and decoration should be streamlined as much as possible, with exceptions noted in the “Affective Considerations” section.

Relevance. A large body of research supports the importance of “relevance” as a motivational factor in STEM education, where relevance can be summarized in three overlapping dimensions: individual, societal, and professional (Stuckey et al., 2013). Designers of educational videos should strive to select content and scenarios that appeal to students’ curiosity and interests, and help prepare them for civic life and future careers. Students’ motivation is also influenced by multimedia design features, such as color and appealing graphics (Mayer & Estrella, 2014; Plass et al., 2014). When designs induce positive emotions, learners’ intrinsic motivation is enhanced to continue working with the materials (Heidig et al., 2015). To avoid cognitive overload, emotional design elements must relate to the essential content of the lesson (Mayer, 2014).

Rapport. Vygotsky’s Zone of Proximal Development refers to the difference between what a learner can do alone and what the learner can do with support from others (Vygotsky, 1978). With respect to video, the social support comes in the form of vicarious learning—being able to “listen in” on peers’ discussions with one another or with a tutor. Dialogue in video, despite the extra cognitive load it can impose, is at least as effective as expository instructional formats with respect to learning gains (Cox et al., 1999). Dialogue can have additional affective benefits. Seeing their ideas represented by peers can help students feel part of a community of learners, even to the degree of treating characters in videos as quasi-collaborators (Lobato & Walker, 2019). Students in videos should be empowered to ask questions and make mistakes (Muller et al., 2008). Although research to date does not support the hypothesis that students necessarily learn better from instructors or models who are like them with respect to age, gender, or ethnicity, we would argue that being inclusive and avoiding stereotypes is an important aspect of respecting and empowering learners (Hoogerheide et al., 2016a and b; Moreno & Flowerday, 2006; Liew et al., 2013).

Accessibility. On the one hand, learner engagement with multimedia is not contingent on high production value (Guo et al., 2014). On the other hand, poor video quality negatively influences audience perceptions of scientific research and researchers (Newman & Schwartz, 2018). Quality, therefore, must be adequate to support learning. Similarly, to make educational videos useable for as diverse a population as possible, design should incorporate the Principles of Universal Design, a set of seven broad principles with finer-grained guidelines to make products and environments widely accessible and useable (Center for Universal Design, 1997). The guidelines relevant to video design include: make the design appealing to all users, eliminate unnecessary complexity, be consistent with user expectations and intuitions, maximize legibility of essential information, arrange information according to its importance, and provide compatibility with devices used by people with sensory limitations (Story, 2001). For example, select the contrasting colors and shades of signals so that they can also be distinguished by those with colorblindness, ensure that closed captions will not cover important content, and incorporate pauses in the video soundtrack to leave time for audio descriptions for the blind. A distinct set of accessibility design guidelines, Universal Design for Learning (UDL) Guidelines (with an accompanying 37-item UDL Scan Tool), can help instructors assess the accessibility of their overall curriculum (Smith & Harvey, 2014). The UDL Scan Tool’s exclusive focus on the need to provide learners with options (e.g. for expressive skills, self-regulation, executive function), however, makes it less relevant for evaluating individual videos.

The Checklist for the Development and Critique of Instructional Videos is: grounded in a broad literature base, including (but not limited to) research on multimedia learning. organized and formatted in a way that makes it easy to navigate. widely applicable, because it does not assume that educational videos have the same style or format. focused on the video as a self-contained curriculum unit, allowing one to review the video without needing “insider” knowledge of how the video will be used. a set of guidelines for video developers and a tool for video reviewers to provide targeted design feedback.

Two limitations deserve mention. First, evaluating to what extent a video satisfies each item is inevitably subjective. A way to address this is to solicit feedback from multiple reviewers (ideally including members of the target audience), and then discuss the feedback as a group. Our group had these kinds of discussions regularly during script development, and we found that having the checklist made the discussions more targeted, constructive, and expedient. For the purpose of ranking a set of videos, items could also be assigned a Likert-scale value.

The second limitation is that the impact of a video depends on how it is integrated into the curriculum (Ljubojevic et al., 2014). Before watching the video, students’ interest should be piqued through the making of predictions or the induction of cognitive dissonance (Smetana & Bell, 2012). Embedded questions help support learning, though only on the topic of the questions themselves (Lawson et al., 2007). Questions should thus be tailored to the demands of the course, and one may not want to put questions in a video that can be used in different ways in different courses. In short, satisfying all the items on the Checklist for the Development and Critique of Instructional Videos does not guarantee that a video will lead to the desired learning outcomes, but the instrument is an important step in that direction.

We would like to acknowledge our beloved team member and former principal investigator, Jeffrey B. Remmel, who passed away in the first year of this project. The project is supported by a National Science Foundation Improving Undergraduate STEM Education grant, number 1610193.

Sherry Seethaler (sseethaler@ucsd.edu) is the director of education initiatives in the Division of Physical Sciences, Adam J. Burgasser is a professor in the Department of Physics, Thomas J. Bussey is an associate teaching professor and vice chair of undergraduate education in the Department of Chemistry and Biochemistry, John Eggers is a professor in the Department of Mathematics, Stanley M. Lo is an associate teaching professor in the Section of Cell and Developmental Biology, Jeffrey M. Rabin is professor in the Department of Mathematics, Laura Stevens is a professor in the Department of Mathematics, and Haim Weizman is a teaching professor in the Department of Chemistry and Biochemistry, all at the University of California San Diego in San Diego, California.