Embedding Evaluation Into the Work Practice of Educators
By Peter Wardrip, Lisa Brahms, and Annie McNamara
Learning Design Experiments (LDEs), which take their name from design experiments in the learning sciences (Brown 1992), support educational innovation and progressive refinement of learning experiences by taking an evidence-centered design perspective (e.g., Kim, Almond, and Shute, 2016).
The cyclical LDE process (see Figure 1) begins with identifying a problem of practice that an educator wishes to address. Next, the practitioner offers conjectures to identify evidence of learner engagement that they hope to see. The conjectures are meant to explicate theoretical assumptions that are embedded in the process of designing. The educator then observes how the design impacts the learning experience (Sandoval 2014). Once conjectures for engagement have been established, the educator modifies—or tweaks—the current design and collects data on subsequent learner engagement. This iterative process continues until a satisfactory design for the learning experience is achieved.
MAKESHOP is a makerspace at the Children’s Museum of Pittsburgh. Within MAKESHOP, families engage in interest-driven creative production at the crossroads of disciplines such as science, technology, engineering, art, and math (e.g., Wardrip et al. 2016). An important feature of MAKESHOP is that it is staffed by dedicated teaching artists—museum educators trained in a mode of making, as well as pedagogy to support engagement in making.
Below, we illustrate the LDE process using an example from a teaching artist named Kevin.
Making with circuit blocks is a popular activity in MAKESHOP. Circuit blocks are wooden blocks with different components and power sources affixed to them that allow learners of all ages to design, create, and test their ideas (see Figure 2). Kevin noticed that young children were not engaging with the circuit blocks in ways that allowed them to tinker and inquire like slightly older children were able to do.
Kevin’s initial conjecture was that the table and materials needed to be more accessible and within reach. After making this change, he observed young children were still not engaging as desired. He noticed that they were having a hard time opening alligator clips and connecting them to nails to build circuits in the learning experience’s initial design (see Figure 2).
Kevin wanted to see young children attaching and detaching wires, completing circuits, and exchanging power sources, switches, and outputs. Recognizing that alligator clips are attracted to magnets, Kevin decided to substitute magnets for the nails. In this design, young children would not need to open and close the alligator clips.
Kevin’s first attempts at redesign featured raised posts with small, powerful magnets embedded into the posts (see Figure 3). This design drew attention to the point of connection, but the embedded magnets hid elements of construction, therefore not making the completion of the circuit apparent. These blocks were also quite difficult and time-consuming for the museum staff to make.
In his next tweak, Kevin used magnetic screws instead of the small magnets (see Figure 4). This both lessened the time of creation and used materials that visitors more easily identified as magnets. This supported an unanticipated element of engagement: inquiry-based discussions about the screws being magnetized and allowing for conductivity. Following these tweaks, Kevin documented young children attaching and detaching the alligator clips and completing circuits. This process led Kevin, in collaboration with other teaching artists, to design a template for circuit blocks that honored important design elements, while affording young children’s successful engagement in an authentic making activity.
As the team constructs new circuit blocks using Kevin’s approach, the MAKESHOP team has discovered that in addition to being enabling and approachable for young children, the blocks have increased engagement for everyone, including learner groups with low dexterity or vision.
Ultimately, we believe that LDEs provide the Children’s Museum of Pittsburgh teaching artists, and educators in other settings, with an intentional approach to improving hands-on learning experiences.
Peter Wardrip (firstname.lastname@example.org) is assistant professor of STEAM education in the School of Education at the University of Wisconsin–Madison in Madison, Wisconsin. Lisa Brahms (email@example.com) is director of learning and research at the Pittsburgh Children’s Museum in Pittsburgh, Pennsylvania. Annie McNamara (firstname.lastname@example.org) is a senior research associate at the Fred Rogers Center in Latrobe, Pennsylvania.
Brown, A L. 1992. Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences 2 (2): 141–78.
Kim, Y.J., R.G. Almond, and V.J. Shute. 2016. Applying evidence-centered design for the development of game-based assessments in physics playground. International Journal of Testing 16 (2): 142–63.
Sandoval, W. 2014. Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences 23 (1): 18–36.
Wardrip, P.S., L. Brahms, C. Reich, and T. Carrigan. 2016. Supporting learning in museum makerspaces: A national framework. Museum (Sept./Oct.): 18–24.
Journal ArticleIs a Framework of Support Enough?
While undergraduate research is known as a high-impact practice, little research has been conducted for the online educational setting. Early research...
Journal ArticleAn Examination of Constructivism, Active Learning, and Reflexive Journaling and Their Independent and Combined Effects on Student Acceptance of Biological Evolution
Instruction that increases acceptance of evolution is essential to effective biology instruction, but instruction about evolution is not consistently ...
Journal ArticleSTEM Faculty Institute
As change agents at our university, we sought to facilitate a transition within our STEM college toward more extensive use of evidence-based instructi...