Microgenetic analysis and open discussion

Inviting students to observe the data, the most obvious phenomenon will be that every team overall gets faster, although their paths will, of course, differ, and there are likely to be some points of slowing. The greatest speedup is usually in the early trials, with the speedup decelerating as practice progresses. This is the famous “power law of practice,” although, as discussed later, practice effects may not really fit a power function. I often emphasize this by drawing an “eyeball power curve” through the data (dashed blue curve in Figure 2).

What changes from round to round, leading every team to speed up, and why does the learning take the observed shape? To explore these questions I lead the class in a microgenetic analysis (Siegler, 1987, 1995; Agre & Shrager, 1990) by eliciting detailed descriptions of how each team’s procedure changed from one round to the next.

The first things that will come to mind are cognitive changes in the seeker. For example, the seeker’s memory of the pattern is building up, and they are learning to anticipate where to go next; they are “getting it into their head and hands,” so to speak, a phenomenon sometimes (too simplistically) called “motor memory.” However, it can easily be observed that learning is multifactorial; there are many changes in the strategy employed by the seeker and guide, both individually and as a team. Some of the most obvious changes will be in the communications between seeker and guide. For example, players commonly run into a “left ↔ right” confusion, resulting from the seeker and guide having different points of view, especially if they are sitting across from one another. Most teams facing this sort of problem quickly figure out how to resolve it by using compass points, for example, or more simply moving to a shoulder-to-shoulder arrangement. Another class of common strategic improvements surrounds how the team resolves errors. Importantly, even changes that appear to take place purely in the head (or “hands”) of the seeker rapidly lead to changes in the structure of team collaboration. For example, some seekers learn to use several fingers, or even their whole hand, to find the next number by feel. The discovery of this new sensory facility enables improved efficiency in the guide’s directions. For example, the guide may now only have to give rough direction, being able to lean on the seeker’s new capability.

Open-ended microgenetic analysis, that is, discussion of other changes experienced by the participants, should be encouraged, and occupy a significant period of time. Given space constraints, I cannot go into very many of the changes that will commonly be discovered once the class becomes engaged in microgenetic analysis, but it is worth drawing out the core concepts that should arise:

• Multifactorial change: As previously mentioned, microgenetic analysis will lead to the observation that many things are changing from one trial to the next, including changes in memory, individual actions, communicative acts, and many others. Although the seeker’s memory of the pattern undeniably improves, albeit slowly, many other things change from trial to trial, such as the structure of the collaboration between the seeker and the guide, and the three-way interaction between seeker, guide, and setting, that is, the pattern, how the paper is positioned, what the seeker feels with their fingers, and so on. Early on, the changes in the structure of the team’s strategy generally overwhelm improvements in memory (Haider & Frensch, 2002). Later on, learning is driven by changes that seem more “internal” or cognitive, such as improvements in the seeker’s “motor memory” of the particular path.
• Unpredictability (“contingency”) of learning: At the microgenetic level, each team proceeds along a particular path whose details are due to many kinds of change. There are many, many possible paths to similar end-states. Which particular path a given team takes is highly “contingent,” that is, it depends upon many details of their experience, and of the context, and often of just sheer randomness. As discussed below, teaching, coaching, and parenting are ways that we rein in this sort of unpredictability. Importantly, performance can sometimes slow (become worse). It is interesting to try to understand why this is when it takes place in a given team’s efforts.
• The value and dangers of averaging over individual data to create “laws”: I try to highlight the value of each type of analysis. Average the paths together into “laws” of change can be valuable for prediction, and in design, but such abstractions hide the multiple root causes at the detailed (“microgenetic”) level. Analysis at the microgenetic level is useful in understanding errors, as well as in creating hypotheses about what leads to the overall structure of learning. Both types of analysis have important implications for practice and teaching, some of which are discussed below.
• The limits of learning: There is, of course, a limit to how much one can improve; any action or communicative act takes time, no matter how much you practice, as do cognitive processes. However, in most activities people keep improving, even if in tiny ways, asymptotically heading toward this limit (Crossman, 1959). Although the instructions for the seeker game encourage speed and accuracy, and so explicitly optimize their strategies, people will tend to do so even without instruction (Agre & Shrager, 1990).
• The shape of the average learning curve: Although any setting in which there are diminishing returns will have the sort of sublinear shape approximated by a power function (with a negative exponent), Heathcote et al. (2000) found that an exponential function fits practice effect data better than a power function. However, the observation that many things change from trial to trial suggests the interesting possibility that there are several simultaneous linear functions operating (Côté et al., 2007). The simplest formulation of this hypothesis involves the convolution of two linear functions: a high negative slope “fast” function associated with strategy change, and a less negatively sloped “slow” function, associated with memory improvements. Both of these operate all the way through the task, but early on, strategy change dominates, whereas later on memory phenomena dominate. Sometimes a new discovery at the strategic level can cause a step function later in the task.

Transfer phenomena

An additional phase that I generally add to the basic activity, time allowing, is an exploration of transfer effects. That is, how performance changes when aspects of the task are changed. One could ask the teams to swap the seeker and guide roles, and/or have the teams swap patterns. These manipulations will have a greater or lesser effect of making the activity temporarily more difficult, increasing the rate of mistakes, and the time it takes to complete the task.

I usually do only two or three additional trials with the new configuration, and the right part of Figure 3 depicts actual results in one class. Roughly, the task completion times increase, but not all the way back up to where they were at the start, and rapidly return to their asymptotic march toward the physical limit. This happens because the new task shares much in common with the former task. The advantage deriving from reuse of learnings supported by common elements between tasks is variously called “savings,” “learnings,” or “positive transfer,” and the extra load, and thus lag, resulting from changed, uncommon elements is variously called “loss” or “negative transfer” (Singley & Anderson, 1989).

A typical savings across changes is that the team has already figured out a good communicative strategy, although the particular players might not have practiced these in their new roles. Also, even if the roles are swapped, if the patterns are not swapped between teams, the former guide (who now becomes seeker), will have some memory of the pattern having guided the previous seeker through it several times, although they will not have the “motor memory” that the original seeker had. Notice that if the pattern is changed, the seeker’s memory of the pattern is likely to actually interfere with learning the new one, a source of “loss” (sometimes called “interference”). The more the elements of the task are changed, the more error-prone and slower the activity will become, although learning continues to operate, and performance will continue to improve from the change point forward.

The role of teachers, coaches, parents, and designers

Having observed that strategy change constitutes the greatest part of the practice effect, at least early on, it is natural to ask where strategy changes come from. Unfortunately, like the path contingency observed above, strategy change is also fundamentally contingent. Just as there are many paths to similar endpoints, and which path a person or group takes will depend on tiny details of the participants’ experience and of the learning context, how and when these paths are discovered is fundamentally unpredictable.

People might invent new, efficient strategies on their own, as the teams do in this activity. But in the real world, it is much more common to have strategies culturally communicated: Your piano teacher tells you to hold your hands a particular way, and organizes the pieces you play to build up certain skills; a baseball coach tells you to keep your eye on the ball; your flight instructor tells you to scan your instruments in a particular order; coaches tell us not to practice our mistakes; tutors teach us to use spaced practice; your parents teach you to say “please” and “thank you,” and to look both ways before crossing the street; and so on. Indeed, teaching, coaching, and parenting are all about offering us strategies and guiding us through experiences in an order that will make learning safe and efficient.

Topics for advanced discussion

Here are some ideas for more advanced discussion, given more time or a more advanced course:

• As previously mentioned, Heathcote et al. (2000) found that an exponential function fits the data better than a power function. (For a constant, c, c^x is an exponential function, whereas x^c is a power function. Additional parameters control the shape and location of the curve.) How could one design experiments to tease apart the power function versus other mathematical forms, and versus more complex models, such as the multilinear model discussed earlier? What statistics would you use to test these hypotheses? Can such an experiment be done without damaging the fundamental interactive nature of the activity?
• What does the universal nature of learning curves tell us about people, either individually or regarding collaboration, and about teaching, coaching, and parenting? How do these practices differ in regard to how they lead students along the learning curve? Is it necessary to pass through a specific series of interim stages in order to reach a desired final strategy? How does this differ for different domains? How does individual discovery interact with “received” (taught) strategies?
• In the seeker game we observe increased speed and accuracy, whereas in other domains, such as musical performance, the goal is not speed so much as accuracy, fluency, and expression. In domains such as mathematics or art, speed apparently does not play any role at all, yet we still see movement toward improvement, whatever the goal. Do similar processes of change underlie all of these? What differs among them?
• How do the structure and interactivity of the team influence learning? Sun et al. (2020) describe a model of collaborative problem solving that has three major subcomponents: (1) Constructing shared knowledge (expressing one’s own ideas and attempt to understand other ideas); (2) Negotiation/coordination—agree upon and execute shared plans of action; (3) Maintaining team function. How do these roles and functions influence practice effects in these sorts of settings?

Conclusion and prospectus

Humans are incredible; with only a few simple instructions, even young children are able to succeed and improve in novel, complex activities. We often think of learning in simple terms: “Practice Makes Perfect,” or asserted “laws” such as “The Power Law of Practice.” Microgenetic analysis, looking closely at the sources of change, reveals a complex interaction among the cognitive, physical, and interpersonal elements of human activity. The models that arise from these sorts of analyses have important implications for human performance, as well as in the practical design of educational and coaching interventions, and the engineering of any sort of system that involves people.

The phenomena of practice effects are directly applicable to any domain involving humans. Not only do we communicate strategies to one another through teaching, coaching, and parenting, we design them into our engineered world. Every engineered system embodies a preferred mode of use, and it is expected by the designers that either people can immediately use the device efficiently, or, more commonly, that they can efficiently learn to use it. Indeed, the course that uses this activity is called “Interaction Analysis,” and it is a capstone seminar in the Human-Computer Interaction track within the Symbolic Systems Program—approximately the Cognitive Science department—at Stanford. This course is generally attended by senior and early graduate students in symbolic systems, computer science, psychology, engineering design, and a smattering of students from the “hard” sciences.

Microgenetic analysis of The Seeker Game helps us to see how change straddles the psychological, physical, and social context; when something changes in human activity, the change may be in one person’s head, but most changes involve both individuals and the world, and often involve social connections. The fact that psychological phenomena often occupy a middle-ground between the cognitive, social, and physical domains is one of the aspects that makes psychology a particularly difficult science. Indeed, modern statistics was, in large part, invented to deal with the complexity of psychology. I sometimes joke with my colleagues that the richness and complexity of important human phenomena, such as the practice effect, and the dangers of abstracting “laws” over complex behavioral phenomena, is what makes the “soft” sciences the hard ones! ■

Acknowledgments

I thank the students in my Symbolic Systems 245 course at Stanford, over many years, for their patience and good humor in putting up with playing The Seeker Game, and in helping to improve it. I thank two anonymous reviewers, as well as Dr. Matthew Lewis, and Dr. David Wojnowski for guidance that significantly improved this paper.