Research and teaching
By Sarah Beth Wilson and Pratibha Varma-Nelson
Peer-Led Team Learning (PLTL) is a small-group, collaborative problem-solving model in which approximately eight students meet weekly for 90–120 minutes under the guidance of a trained undergraduate facilitator, called a peer leader, to discuss and debate questions aligned with that week’s lecture content (Wilson & Varma-Nelson, 2016). A recent review article reported that the positive effects of PLTL participation on students’ course performance have been well-documented in a variety of undergraduate courses (Wilson & Varma-Nelson, 2016). Several PLTL studies reported improved student performance on nationally normed standardized exams, course retention, and retention in a curriculum sequence (Wilson & Varma-Nelson, 2016). Not only has PLTL been an effective active-learning intervention in a variety of disciplines, the pedagogy has shown promising adaptability to the other settings, including laboratories and online (Wilson & Varma-Nelson, 2016).
The percentage of higher education students taking at least one online course has risen to 31.6% of all higher education enrollments (Seaman et al., 2018). “Blended” or “hybrid” courses, those with an online component that augments face-to-face lectures, have also become more prevalent in recent years (Graham, 2005). A recent meta-analysis reported that students in hybrid courses performed significantly better than their face-to-face counterparts, while purely online students performed comparably to face-to-face students (Means et al., 2013). Likewise, two studies that assessed the effectiveness of blended and face-to-face sections of statistics courses across multiple institutions reported comparable outcomes in terms of course completion (Allen & Seaman, 2017; Bowen et al., 2014), course grades (Allen & Seaman, 2017; Bowen et al., 2014), and performance on a national statistical literacy test (Allen & Seaman, 2017). Tutty & Klein’s (2008) comparison study of face-to-face and online pairs of students in an undergraduate preservice teacher computer literacy course asked each other more questions when they collaboratively solved the assignments in a computer-mediated environment than they did in face-to-face environments. Moreover, Burke and Chidambaram (1996) found comparable performance among face-to-face, synchronous online, and asynchronous online students’ groups who collaboratively wrote a policy manual using the GroupWriter co-authoring program.
When peer leaders are present to facilitate student learning, students in asynchronous online courses participate more regularly (Tagg, 1994) and exhibit more meaningful construction of knowledge (Aviv et al., 2003; Garrison & Cleveland-Innes, 2005). Dorner (2012) reported that e-mentors scaffolded “group-level collaboration” and shared professional insights and suggestions when facilitating large (n = 20) student groups, yet were more directive when facilitating small groups (n = 8). Mauser et al. (2011) and Smith et al. (2014) evaluated the impact of transitioning a peer-led, active-learning pedagogy, PLTL, to an online setting. Cyber Peer-Led Team Learning (cPLTL) is a synchronous online adaptation of PLTL in which the students collaborate via a web conferencing program (Mauser et al., 2011; Wilson & Varma-Nelson, 2016). cPLTL students experience all four factors of media richness—feedback immediacy, cue transmission capacity, natural language use, and personal focus, such as seeing individual’s expressions that convey emotions (Daft & Lengel, 1986)—because web cameras, document cameras, and the chat function enable participants to hear one another’s voices and see one another’s facial expressions and gestures. Such feedback is fundamentally necessary for effective interactions, whether students are face-to-face or online (McIssac & Gunawardena, 1996). Smith et al. (2014) reported that general chemistry PLTL and cPLTL students earned comparable mean course grades and American Chemical Society (ACS) general chemistry exam scores, although there were notable differences in the dynamics of the two learning environments. For instance, the cPLTL students’ conversations tended to focus more on alternative ways to solve problems than PLTL students. Secondly, the peer leaders stressed different levels of student accountability in the two settings: addressing groups of PLTL students to check for understanding, yet querying individual cPLTL students for formative assessment. Hence, the same peer leaders when placed in different environments behaved differently (Smith et al., 2014).
Although a separate study characterized the same population of students’ use and explanation of electron-pushing mechanism arrows (Wilson &Varma-Nelson, 2019), the purpose of this study was to evaluate both the overall course performance impact and participant perceptions of implementing PLTL and cPLTL for students in that first-semester organic chemistry course. This study’s analysis of the impact of implementing a hybrid adaptation of an academic intervention in an organic chemistry course adds to the growing body of literature about the efficacy of blended-learning environments for different academic content (Means et al., 2013; Nguyen, 2015). Likewise, reporting students’ viewpoints regarding setting choice and setting-related experiences is informative as more hybrid and online courses are being developed. Moreover, organic chemistry is a pivotal course in the curriculum of several science majors and preparation for health profession schools (Elam et al., 2002). Therefore, it is important to assess an academic intervention with the possibility of improving course performance and retention of science, technology, engineering, and mathematics (STEM) majors since the U.S. economy needs approximately one million more STEM college graduates (Olson & Riordan, 2012) and approximately 2.4 million new healthcare professionals in the years from 2016 to 2026 (U.S. Bureau of Labor Statistics, 2019). Additionally, since PLTL has been shown to be particularly efficacious for women and underrepresented minority STEM students (Gafney & Varma-Nelson, 2008), this academic intervention could result in greater numbers of female and minority STEM professionals, who offer unique perspectives and experiences to developing scientific innovations (Ong et al., 2011). Hence, the purposes of this study were (1) to determine the effects of PLTL/cPLTL on student performance in a sophomore, undergraduate, first-semester organic chemistry course and (2) to examine the PLTL and cPLTL students’ perceptions of a problem-solving workshop series.
A combination of 110-minute, weekly PLTL and cPLTL workshops was implemented in three sections of first-semester, three-credit organic chemistry lecture courses in the spring and fall 2014 semesters. No non-PLTL sections were employed because it would have been unethical to withhold PLTL from students given the positive impact of the pedagogy on previous students at this institution and reported in the literature (Gosser et al., 1996; Wilson & Varma-Nelson, 2016). No additional course credit was granted for participation in the PLTL or cPLTL sessions because PLTL was instituted instead of recitations, which are customary chemistry course features that are not affiliated with course credit (Tien et al., 2002). By fall 2013, the PLTL workshop program was considered a stable environment for both implementing and evaluating the implementation of cPLTL in the organic chemistry course. Institutional support, including increased funding for peer leaders and longer classroom reservations, allowed the extension of PLTL workshop duration to 1 hour 50 minutes to align with the recommended PLTL model (Gafney & Varma-Nelson, 2008, p. 12) beginning in spring 2014.
The authors referred to the PLTL literature (Gafney, 2001; Gafney & Varma-Nelson, 2008; Gosser et al., 2010; Wilson & Varma-Nelson, 2016) and articles about implementation of cPLTL in general chemistry courses (Mauser et al., 2011; Smith et al., 2014) to develop the organic chemistry PLTL/cPLTL program. The multiple lecture sections in the fall semester were treated as a single course, having common workshop problem sets, lecture slides, and final exams. The PLTL sections consisted of three groups of 8−10 students, while the three cPLTL sections consisted of approximately eight students each. Document cameras were purchased by the department to enable the implementation and evaluation of organic chemistry cPLTL without incurring additional costs for students, as long as the document cameras were returned at the end of the semester.
Peer leaders were trained in both content and pedagogy (Gosser et al., 2001) in weekly 90-minute training sessions before they facilitated PLTL and cPLTL workshops. In the workshops, students worked together to solve problem sets that were designed to challenge the students’ conceptual understanding of the course content, while developing problem-solving skills. At this institution, the peer leaders were paid a stipend to facilitate two PLTL and/or cPLTL workshops per week, although other insitutions either grant service credit or use volunteer peer leaders (Gaston, 2004; Schray et al., 2009). The funding for the peer leader stipends at this institution were generated from $80 PLTL fees.
The two key differences in the implementation of cPLTL compared to PLTL were: (1) cPLTL students and their peer leaders collaborated in an Adobe Connect web conference synchronous online setting and (2) cPLTL students received training, called Workshop Zero, about how to use the document camera and the web conferencing interface prior to the first content workshop (Smith et al., 2014).
Students from both semesters were pooled as a single sample population because Chi Square analyses of spring and fall 2014 students’ gender, ethnicity, and previous chemistry grade point average (GPA) were comparable, although significantly more students over age 23 were enrolled in the spring semester (36% versus 25%). The subjects of this study included a subset of the population who enrolled in “comparison group” workshop sections, which were pairs of PLTL and cPLTL sections led by the same peer leaders (one pair of comparison groups for spring and three pairs of comparison groups for fall). The subjects included in this study of course performance and student perceptions are the same subjects whose use and explanation of electron-pushing arrows was characterized in an earlier study (Wilson & Varma-Nelson, 2019). Subjects self-selected into either a PLTL or a cPLTL section. Because the PLTL/cPLTL emphasis was on problem-solving, no answer keys were provided to either PLTL or cPLTL students (Gosser et al., 2001).
At the end of each semester, students were administered a questionnaire to assess their perceptions of their learning in the workshops and understand their rationale for setting choice (Table 1). After submission of final grades, the questionnaire data were collated and analyzed. After Institutional Review Board approval, the course instructors and the Information Management and Institutional Research Office provided student course grades, identifying data (gender, ethnicity, and previous chemistry GPA), and ACS organic chemistry first-semester exam (version 2010) scores. The exam scores were converted to Z-scores in order to compare results across different versions of the exam (Chan & Bauer, 2015).
Descriptive statistics were calculated from students’ questionnaire responses, ethnicity, gender, previous chemistry course GPAs, and ACS organic chemistry first-semester exam (version 2010) scores. Mann-Whitney U tests, the nonparametric equivalent of a t-test, were performed to compare PLTL and cPLTL students’ survey responses and course grade distributions because the sample sizes were not large enough to be normally distributed (Rosner & Grove, 1999). Then, an analysis of covariance (ANCOVA) was performed to determine if there was a significant difference in PLTL and cPLTL students’ ACS organic chemistry first-semester exam (version 2010) scores, depending on their ethnicity, gender, and setting with ACS organic chemistry first-semester exam (version 2010) Z-scores, controlling for previous chemistry GPA.
Mann-Whitney U tests indicated that there were no significant difference in the distribution of course grades for PLTL and cPLTL students (Figure 1). ANCOVA analysis of PLTL and cPLTL students’ ACS organic chemistry first-semester exam (version 2010) scores were comparable, with no interaction effect based on gender or ethnicity. Lastly, a Chi Square analysis indicated there was no significant difference in student attendance in PLTL or cPLTL workshops.
Fifty-two comparison group students (33 PLTL students; 19 cPLTL students) completed the Workshop Perception Questionnaire, for an overall response rate of 76%. Cronbach’s alpha, calculated in SPSS (version 22) as an assessment of the reliability of the student perception survey instrument, was computed to be 0.71, which is the appropriate level for a low-stakes testing situation (Cronbach, 1984; Cronbach, 1951). Although both PLTL and cPLTL students believed that each of the activities involved in the workshop learning environment benefited their learning (Table 1), PLTL students’ perceptions of learning impact were significantly higher than those reported by cPLTL students in three categories: one-on-one discussion with the peer leader; a small group member explaining a concept to them; and collaborating to solve problems. Interestingly, the PLTL students perceived the workshop problems as being significantly more challenging than the cPLTL reported. PLTL students reported a significant preference for learning face-to-face and taking courses on campus, while cPLTL students preferred learning online and avoiding the commute to campus.
The objective of this study was to evaluate the effect of implementing cPLTL at an institution that had an established organic chemistry PLTL program. We hypothesized that cPLTL students’ performance would be comparable to the performance of their PLTL classmates in the first-
semester organic chemistry course and that both PLTL and cPLTL students would perceive positive benefits from participation in the workshop series. Statistical evaluation of both course grades and ACS first-semester organic chemistry exam scores indicated comparable PLTL and cPLTL students’ performance, which is consistent with results of an earlier general chemistry cPLTL implementation study (Smith et al., 2014). Therefore, the results of our study support the literature that media-rich hybrid courses, such as cPLTL supplementing a traditional lecture, provide sufficient interactivity and feedback for effective collaborative learning to occur. Students from both settings perceived that interacting with each other and their peer leader benefited their learning, which is consistent with Finn and Campisi’s (2015) findings. The PLTL students in this study reported significantly higher perceived value from one-on-one conversations with the peer leader, collaborating with classmates in their small group, and having concepts explained by a small group member than the cPLTL students conveyed.
In conclusion, our study indicates that there is no significant difference in undergraduate organic chemistry PLTL and cPLTL students’ course performance or achievement on a standardized, nationally normed content exam. This organic chemistry PLTL/cPLTL study, the Smith et al. (2014) general chemistry PLTL/cPLTL study, and the Burke and Chidambaram (1996) management study all indicate that there are comparable positive learning gains for face-to-face and synchronous online students in media-rich collaborative settings, which suggests that our findings are likely to generalize to other STEM postsecondary courses.
This study evaluated the effect of implementing cPLTL compared to PLTL by using course grades and a standardized content exam. Because there was no significant difference between PLTL and cPLTL students’ performance in this study and PLTL students outperformed nonPLTL students at this institution historically, we have inferred that cPLTL students would outperform nonPLTL students even though we did not conduct a direct comparison between nonPLTL and cPLTL students.
|Table 1. Workshop Perception Questionnaire results.|
Implications for research and practice Comparable organic chemistry student achievement in face-to-face and online PLTL settings corroborate the Smith et al. (2014) findings that social constructivism can occur in an online environment. Therefore, as more higher education institutions expand offerings for hybrid and online courses to accommodate commuting students and adult learners, the cPLTL adaptation is a viable option to provide a collaborative online learning environment. Furthermore, the PLTL academic intervention, whether implemented face-to-face or in online settings, may be an effective means to bolster the performance and retention of diverse STEM majors, including female and minority students (Horwitz & Rodger, 2009; Preszler, 2009; Quitadamo et al., 2009), which is a key step in increasing the number of diverse STEM college graduates and healthcare professionals. The particular facilitation techniques employed by the peer leaders can be influenced by perceived environmental limitations, so PLTL/cPLTL workshop coordinators and instructors should address possible collaborative learning activities and facilitation methods with peer leaders during weekly training. Future research should compare the quality of PLTL and cPLTL students’ performance on specific content and skills, similar to the Finn and Campisi (2015) and Wilson and Varma-Nelson (2019) studies, to determine if particular topics or techniques are better developed by students in a specific setting. ■
We thank the students who participated in the study, the peer leaders who dedicated themselves to improving the learning of others, and the IUPUI Department of Chemistry and Chemical Biology for ongoing support of pedagogical innovation and evaluation. We also thank Virginia Rhodes of the STEM Education Innovation and Research Institute at IUPUI for her helpful comments during the preparation of this manuscript.
Sarah Beth Wilson (firstname.lastname@example.org) is an associate professor in the Department of Chemistry at the Oakland City University in Oakland City, Indiana. Pratibha Varma-Nelson (email@example.com) is professor of chemistry and founding executive director of the STEM Education Innovation and Research Institute at Indiana University–Purdue University Indianapolis in Indianapolis, Indiana.
Allen, I. E., & Seaman, J. (2017). Digital learning compass: Distance education enrollment report 2017. https://onlinelearningsurvey.com/reports/digtiallearningcompassenrollment2017.pdf
Aviv, R., Erlich, Z., Ravid, G., & Geva, A. (2003). Network analysis of knowledge construction in asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1–23.
Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2014). Interactive learning online at public universities: Evidence from randomized trials. Journal of Policy Analysis and Management, 33(1), 94–111.
Burke, K., & Chidambaram, L. (1996). Do mediated contexts differ in information richness? A comparison of collocated and dispersed meetings. In 29th Hawaii International Conference on System Sciences (HICSS) (pp. 92–101). Maui, Hawaii.
Chan, J. Y. K., & Bauer, C. F. (2015). Effect of peer-led team learning (PLTL) on student achievement, attitude, and self-concept in college general chemistry in randomized and quasi experimental designs. Journal of Research in Science Teaching, 52(3), 319–346.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
Cronbach, L. J. (1984). Essentials of psychological testing (4th ed.). Harper & Row.
Daft, R. L., & Lengel, R. H. (1986). Organizational information requirements, media richness, and structural design. Management Science, 32(5), 554–571.
Dorner, H. (2012). Effects of online mentoring in computer-supported collaborative learning environments: Mentor presence and cognitive engagement. American Journal of Distance Education, 26(3), 157–171.
Elam, C., Seaver, D. C., Berres, P. N., & Brandt, B. F. (2002). Preparation for medical, dental, pharmacy, physical therapy, and physician assistant careers: Helping students gain a competitive edge. Journal of College Admission, 176, 16–21.
Finn, K., & Campisi, J. (2015). Implementing and evaluating a Peer-Led Team Learning approach in undergraduate anatomy and physiology. Journal of College Science Teaching, 44(06), 38–44. https://doi.org/10.2505/4/jcst15_044_06_38
Gafney, L. (2001). Workshop evaluation. In D. K. Gosser, M. S. Cracolice, J. A. Kampmeier, V. Roth, V. S. Strozak, & P. Varma-Nelson (Eds.), Peer-Led Team Learning: A guidebook (pp. 75–93). Prentice Hall.
Gafney, L., & Varma-Nelson, P. (2008). Peer-Led Team Learning: Evaluation, dissemination and institutionalization of a college level initiative. Springer.
Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: Interaction is not enough. American Journal of Distance Education, 19(3), 133–148.
Gaston, J. L. (2004). An analysis of a special peer-led team learning mathematics. Assessment Essentials, 1(1), 2–4.
Gosser, D. K., Cracolice, M. S., Kampmeier, J. A., Roth, V., Strozak, V., & Varma-Nelson, P. (2001). Peer-Led Team Learning: A guidebook. Prentice Hall.
Gosser, D. K., Kampmeier, J. A., & Varma-Nelson, P. (2010). Peer-Led Team Learning: 2008 James Flack Norris Award address. Journal of Chemical Education, 87(4), 374–380.
Gosser, D., Roth, V., Gafney, L., Kampmeier, J., Strozak, V., Varma-Nelson, P., Radel, S., & Weiner, M. (1996). Workshop chemistry: Overcoming the barriers to student success. The Chemical Educator, 1(1), 1–17.
Graham, C. R. (2005). Blended learning systems: Definition, current trends, and future directions. In Handbook of blended learning: Global perspectives, local designs (pp. 3–21). Pfeiffer.
Horwitz, S., & Rodger, S. H. (2009). Using peer-led team learning to increase participation and success of under-represented groups in introductory computer science. ACM SIGCSE Bulletin, 41(1), 163–167.
Mauser, K., Sours, J., Banks, J., Newbrough, J. R., Janke, T., Shuck, L., & Zhu, L. (2011). Cyber Peer-Led Team Learning (cPLTL): Development and implementation. EDUCAUSE Review Online, 1–17.
McIssac, M. S., & Gunawardena, C. (1996). Distance education. In Handbook of research for educational communications and technology (pp. 403–437). Macmillan.
Means, B., Yukie, T., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record, 115, 1–47.
Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. Journal of Online Learning and Teaching, 11(2), 309–320.
Olson, S., & Riordan, D. G. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Report to the President. Executive Office of the President.
Ong, M., Wright, C., Espinosa, L. L., & Orfield, G. (2011). Inside the double bind: A synthesis of empirical research on undergraduate and graduate women of color in science, technology, engineering, and mathematics. Harvard Educational Review, 81(2), 172–209.
Preszler, R. W. (2009). Replacing lecture with peer-led workshops improves student learning. CBE-Life Sciences Education, 8, 182–192.
Quitadamo, I. J., Brahler, C. J., & Crouch, G. J. (2009). Peer-Led Team Learning: A prospective method for increasing critical thinking in undergraduate science courses. Science Educator, 18(1), 29–39.
Rosner, B., & Grove, D. (1999). Use of the Mann-Whitney U-test for clustered data. Statistics in Medicine, 18(11), 1387–1400.
Schray, K., Russo, M. J., Egolf, R., Lademan, W., & Gelormo, D. (2009). Are in-class peer leaders effective in the peer-led team-learning approach? Journal of College Science Teaching, 38(4), 62–67.
Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase: Tracking distance education in the United States. Babson Survey Research Group.
Smith, J., Wilson, S. B., Banks, J., Zhu, L., & Varma-Nelson, P. (2014). Replicating Peer-Led Team Learning in cyberspace: Research, opportunities, and challenges. Journal of Research in Science Teaching, 51(6), 714–740.
Tagg, A. C. (1994). Leadership from within: Student moderation of computer conferences. American Journal of Distance Education, 8(3), 40–50.
Tien, L. T., Roth, V., & Kampmeier, J. A. (2002). Implementation of a peer-led team learning instructional approach in an undergraduate organic chemistry course. Journal of Research in Science Teaching, 39(7), 606–632.
Tutty, J., & Klein, J. (2008). Computer-mediated instruction: A comparison of online and face-to-face collaboration. Educational Technology Research and Development, 56(2), 101–124.
U.S. Bureau of Labor Statistics. (2019). Healthcare occupations. http://www.bls.gov/ooh/healthcare/home.htm
Wilson, S. B. & Varma-Nelson, P. (2016). Small groups, significant impact: A review of Peer-Led Team Learning research with implications for STEM education researchers and faculty. Journal of Chemical Education, 93(10), 1686–1702.
Wilson, S. B., & Varma-Nelson, P. (2019). Characterization of first-semester organic chemistry peer-led team learning and cyber peer-led team learning students’ use and explanation of electron-pushing formalism. Journal of Chemical Education, 96(1), 25–34. https://doi.org/10.1021/acs.jchemed.8b00387
NSTA Press BookUncovering Student Ideas in Science, Volume 2, Second Edition: 25 More Formative Assessment Probes
COMING IN APRIL 2021; NOW AVAILABLE FOR PREORDER “Leave no alternative science idea unchallenged!” could be the slogan of this second edition o...