The study

This study addresses two research questions about iCons pedagogy during the program’s first two courses:

1. Did iCons instructors implement pedagogical approaches as intended? If not, then how did implemented approaches differ from intended ones?
2. How did students perceive iCons pedagogical approaches, particularly the emphasis on collaboration?

To answer these questions, we utilized a convergent parallel mixed-methods design (Creswell & Clark, 2017), collecting quantitative data (student surveys for Research Question 2) and qualitative data (observations, interviews, and open-response survey questions for Research Questions 1 and 2) in parallel.

Data collection

To answer Research Question 1, instructors of both courses (N = 5) were interviewed at the start of the course to determine their intended pedagogical approaches; observations were also conducted in two iCons 1 sessions, and all iCons 2 sessions. Because we were unable to observe all iCons 1 sessions, we interviewed three iCons 1 students to assess how the pedagogy intended by the instructors was actually implemented. The students were the only volunteers available at the time of the scheduled interview.

Each class session was 75 minutes. Observations focused on activities that took place in each class session and the types of interactions faculty had with students, especially noting the questions students asked.

To answer Research Question 2, students from both courses (iCons 1: N = 34; iCons 2: N = 33) were surveyed in the first and last class meetings, and six students (three from each year) were interviewed at the end of the semester. The i1 students were those we had interviewed to answer Research Question 1, and the i2 students were randomly selected from those who volunteered to a request for interviews. The survey, adapted from So and Brush (2008), included seven learning and seven collaboration items [α(Learning) = .85 and α(Collaboration) = .72] on a 5-point Likert scale (from strongly disagree to strongly agree), as well as four open-response questions asking about perceived learning outcomes and expectations (Figure 2).

Data analysis

Data were analyzed both statistically and qualitatively. For the qualitative data generated from the open-response questions, open coding was used to reflect how answers addressed the research questions (Strauss, 1990). Codes were organized into categories that allowed finding commonalities among responses. Interview responses were coded using predetermined codes focusing on pedagogy and collaboration.

For the Likert-scale survey items, a confirmatory factor analysis determined that items from each scale were well correlated and aligned with So and Brush’s (2008) findings [α(Learning) = .855; α (Collaboration) = .826]. Two-sample and paired-sample t-tests were used to compare the Learning and Collaboration data from iCons 1 and iCons 2 and the pre- and post-course responses.

Findings

Planned pedagogical approaches (Research Question 1)

The interviews revealed that all three iCons 1 instructors saw their role as introducing students to the structure and philosophy of the program and guiding students to ask questions and learn ways to find answers themselves. They saw this as different from the normal lecture format the students encountered in other courses. One iCons 1 instructor stated, “They have to come up with the ideas and go find out for themselves. What they learn will diverge and will differ by what they choose to research.” Similarly, both iCons 2 instructors stressed that they do not provide students with problems to solve “because once you provide examples, it’s virtually impossible for students to scrub them from their minds and all of their questions will just be small variations on the examples.” Instructors emphasized providing an external support structure as needed (e.g., asking questions, pointing to contradictions, or sharing strong and weak examples of professional writing genres), but one i2 instructor noted that he did not want to start by “building students a ladder,” as they would then never have the chance to “learn how to jump” to reach goals: “If you never ask a student to do what they cannot do, they never learn what they can do.”

Implemented pedagogy (Research Questions 1 and 2)

According to the three students interviewed at the end of iCons 1, the instructors implemented student-directed learning. Interview and observation data confirmed that class time was generally dedicated to class discussions, teamwork, and student presentations. Only occasionally did instructors include short lectures on methodology-specific topics such as a Zika-modeling spreadsheet or certain statistical analyses. As intended, instructors provided some background information to the whole class on a problem, and the teams chose which aspect of the problem to address and how to address it. Observation data showed that this teaching approach, minimizing lecture and maximizing student-to-student interaction, was also implemented in iCons 2 classes, where students proposed and selected topics and then worked on them in teams.

All interviewed iCons 1 students agreed that the course had a student-directed approach. While the students reported that it sometimes left them “floundering,” they felt that they and their peers rose to the occasion. As one student noted, the “floundering” was “where a lot of the learning took place.” The interviewed students reported patterns that confirmed what instructors intended. For example, students noted that when they asked for help, the instructors did not give them an answer but instead asked questions that helped them refine or shift their thinking. Direct answers were occasionally provided around a more technical question, such as in the modeling project.

Along with the positive feedback about and appreciation of the student-centered teaching approach, iCons 1 students mentioned a couple of areas in which they would have liked more support. For example, they wished there had been more scheduled check-ins, where faculty helped the team evaluate their plan and progress, at the beginning of the semester, since they were working on their first problem. Speaking for the class, the interviewees pointed out that some groups were reluctant to ask for such meetings, though they felt they needed more guidance. While students understood that instructors did not want to give overly specific criteria, students wanted more explicit directions and criteria for the project presentations and products to avoid feeling that there were hidden expectations that might impact their grades in the course.

Observations showed that iCons 2 instructors made expectations for the products clear and made their strategies explicit to students. For example, instructors stressed that they purposefully selected sample documents or presentations that demonstrated both strong and weak aspects of the genre to spark discussions and help students identify and distinguish good products from weak ones. To increase students’ ability to evaluate their own and each other’s work, students and instructors worked together to develop assessment rubrics for each written assignment or presentation type. Students first brainstormed possible evaluation criteria for each section of a document (e.g., abstract, introduction, etc.), and then, with the instructor’s guidance, refined those criteria and created appropriate rubrics. Both instructors and students used these rubrics to assess their work (e.g., abstracts, project proposals, presentations). Because students assessed each other’s work at first more positively than the instructors felt was merited, instructors discussed with students the need for honest critique and reflection, which resulted in improved student feedback over the semester; however, students’ feedback never fully aligned with instructor feedback.

Observations confirmed the student-directed nature of planned activities. Throughout the semester, iCons 2 students were given many opportunities to decide on topics and activities. For example, students in the renewable energy course chose a debate for their final activity and decided on the debate question and rules. While students in the biomedicine course were assigned a poster presentation for their final product, they developed criteria for evaluating each other’s posters and conducted peer review and feedback sessions before the final presentations.

In summary, observations and interviews indicated that the iCons instructors in both levels of the course implemented the curriculum in the ways they had intended. They offered suggestions and questions instead of direct instruction, and i2 instructors provided examples for discussions about the structure of research projects and written work rather than exemplars to be followed for assignments.

How the students understood and perceived the planned and implemented curriculum was the focus of Research Question 2. To explore that question, we compared the students’ learning goals with their self-reported outcomes and asked students how satisfied they expected to be and actually were with their learning and collaboration in iCons.

Student learning goals (Research Question 2)

In the open-response section of the surveys, students were asked to describe what they wanted to learn in the iCons course. The most popular learning goals for iCons 1 students (N = 34) were working on real-world global issues (44%), collaboration and teamwork (41%), problem-solving (29%), and learning to work with multiple perspectives (26%), followed by critical and creative thinking (20%) and expanding their knowledge (15%), as seen in feedback from one i1 student: “I wish to learn how to address real-world problems and utilize scientific solutions to achieve impactful change. I wish to learn from engaged and enthusiastic peers whose unique and varied perspectives and specialties will help me to attack problems from as many angles as possible.”

For iCons 2 students (N = 33), communication (72%) and expanding their knowledge (48%) were the top learning goals, along with collaboration (18%) and problem-solving (18%), as reflected in a comment from one i2b student: “I want to learn how to articulate my ideas and plans in a clear manner. I want to learn how to continue collaborating with others even if we have opposing views. I want to learn more about the field of biomedicine and the careers it impacts. I want to learn how to make a difference.”

Students from both years gave similar answers to the question of how they planned to meet their own learning goals: More than half of all the respondents named participation and engagement, especially in discussion (52%); using peers as resources through collaboration (51%); and putting in the time to do the required work (52%). One-third of iCons 1 students also identified listening to others’ points of view as a path to meeting their learning goals: “[L]istening to my peers, helping them, and letting them help me will take me far.”

When asked what kind of support they expected from their iCons instructors to help them meet their learning goals, the responses were largely similar for both courses, with a few points of divergence. iCons 1 students (N = 34) acknowledged the planned student-directed nature of the course, and most embraced it. They wanted their instructors to provide guidance and advice (67%) but not “hand-holding.” They also identified independence (32%) as something they wanted the instructors to give them, along with providing resources (11%) and facilitating discussion (17%). One i1student stated, “I expect my iCons instructor to help point me in the right direction and help with any questions I may have, but for the most part I expect to work independent of the iCons instructors.”

iCons 2 students (N = 33) also identified expectations that were coded under the category of guidance as a primary type of instructor support (48%), along with constructive feedback (30%) and being understanding about workload (21%). Direct instruction and delivery of information were also selected by some of the iCons 2 students (24%), likely in response to the skills-oriented communication focus of the course. One of the i2e students wrote, “iCons is a course known for perhaps a ‘lack’ of instructor support. However, a guiding hand and an instructor that encourages ideas would be beneficial in this course.”

Learning experience and collaboration (Research Question 2)

Students completed Likert-scale survey items on their expectations for the course during their first meeting, as well as items to measure their overall satisfaction with their learning and the collaborative aspects of the class during their last meeting (Table 1). The post-survey results of the learning and collaboration scales show that students in both courses were satisfied with their learning and their collaborative experiences (Table 1).

Though students generally were satisfied with their learning and collaboration experiences, a paired-sample t-test indicated there was a small but significant decrease in the Learning scale mean from the pre-course (4.36) to the post-course (4.15) survey (p < .01). There was no significant difference in the Collaboration scale mean (pre-course = 4.38, post-course = 4.33, p = .388). Comparing the two levels of the course using a two-sample t-test, iCons 1 students had higher pre-course ratings for both Learning (i1 = 4.54, i2 = 4.17, p < .01) and Collaboration (i1 = 4.50, i2 = 4.27, p = .02) but post-course, only the Learning items showed a significant difference (i1 = 4.29, i2 = 4.00, p = .03).

When asked what they had learned in the course (end-of-course open-response item; Figure 2), students from both courses identified working effectively in teams as a primary outcome (iCons 1 = 74%; iCons 2 = 36%), followed by communication (18%) and problem-solving (15%). iCons 2 students also identified presentation skills as a notable area of learning (73%).

End-of-year surveys provided evidence that instructors had taught in the manner expected and that students were satisfied with the student-directed nature of the courses. When asked about the support they had received to meet their learning goals, most responses (68%) from all three sections identified feedback, critique, and guidance. Some students (9%) mentioned that information or help was given if a student or group was “stuck,” but that independence to pursue projects was the norm (13%), as reflected in an iCons 1 student comment: “It seems to me that the goal of the course is for the students to be as independent as possible without being totally unstructured and I think the instructors do a good job of this by making us set our own goals and standards at the beginning of each case study. This allows us to be realistic in our expectations for ourselves but also lets us pursue ideas that perhaps would not have fit if we had concrete guidelines laid out for us in advance.”

Talking about the feedback offered by instructors, one i2e student said: “The iCons instructors provided a lot of helpful feedback and tips for ways of improvement. I really like constructive criticism. I’d rather have someone nitpick at my project than say, ‘Yeah, that looks fine.’ There are always ways to improve, and the iCons instructors are very good at pointing those out in a constructive manner.” From these responses, it seems that the iCons instructors have met their goal of providing scaffolded support (Research Question 1).

When asked if they had wanted or needed other support, 33% of students answered that no further support was needed. Those who did want more support fell into three categories: those who wanted clearer expectations from the instructors about assignments or the flow of the class (17%), those who wanted to have more personal input about the scope or timing of assignments (9%), and those who would have liked financial support for laboratory activities (11%). Only a handful of students (5%) wished there had been more direct instruction or focused feedback: “Personally, I like more strict instruction, but I also feel like I learned better having to do my own thinking on how things should be done,” one iCons 1 student said.

With respect to changes in their collaboration skills throughout the semester, students’ responses were generally split among four categories: improved communication or sharing ideas (30%); better at taking into account other points of view and appreciating the skills and knowledge of others (26%); speaking up or taking charge more often (19%); and improvements in adapting to changing roles or dividing up tasks depending on the needs of the group (24%). One i2e student commented, “I have learned better ways to clearly communicate. I have come to regard communication with increased importance as the semester has progressed. With a shared purpose and clear communication, collaboration happens organically in a small group such as the one I am a part of in iCons 2.” Another student responded, “I think I have lost any fear I previously had to speak freely. If I feel a certain way or have any idea, I don’t hesitate to share. I think this has helped my collaboration and teamwork skills this semester, as I have found team communication to be a large component of being effective and successful. I have found my collaboration skills have improved this semester.”

In summary, results show that, in general, iCons 1 and 2 students perceived the pedagogy as intended by the instructors, worked collaboratively in teams, communicated their research to peers and the campus community, and developed collaboration skills.

Discussion

Findings indicate that the instructors did implement the curriculum in the student-directed way that they intended (Research Question 1) and that students expected and were satisfied with the implementation (Research Question 2). iCons 1 students grasped the course goals early in the semester and aligned their learning expectations accordingly, which speaks to the clear and explicit messages that iCons 1 instructors communicated throughout the course. Since iCons 2 students had successfully completed iCons 1 and, thus, were familiar with the nature of the program, it seems less surprising that their responses around expectations also aligned with the iCons 2 course goals centered around scientific communication.

Students from both years identified collaboration, communication, and problem-solving as course goals and learning outcomes. Given the push from industry to equip students with these soft skills (National Academies of Sciences, Engineering, and Medicine, 2016; Society for Human Resource Management, 2019), the results of this study indicate that a program such as iCons can provide a model for learning these skills within an application-based context. However, further research is necessary to explore whether iCons students transfer these skills to the work environment and demonstrate an appropriate combination of technical and soft skills needed to succeed in the workplace.