The ability to read, understand, and interpret research literature is a key skill for students across scientific disciplines (Brownell et al., 2013; Bruehl et al. 2015; Fang, 2005; Hoskins et al., 2011; Hoskins et al., 2007; Round & Campbell, 2013). Introducing students to reading primary scientific literature during their undergraduate science studies is considered an important part of scientific education because doing so promotes students’ scientific knowledge and practice and prepares them to participate in the scientific community (Gallo & Rinaldo, 2012; Kozeracki et al., 2006; Muench, 2000; Sato et al., 2014). 

Despite these advantages, this skill is known to be a difficult one that takes novices considerable practice before they can reach a professional level (Fang, 2005; Gillen, 2006; van Lacum et al., 2012; Wu, 2009). Most students in the tertiary setting have difficulty comprehending research articles (Hoskins et al., 2007; Round & Campbell, 2013). Scientific papers are written for an audience of other professional scientists (Gillen, 2006), and they use specialized scientific language (Fang, 2005) that has characteristics such as “informational density” (p. 338), “abstraction” (p. 339), and “technicality” (p. 341), which are challenging and complicated for students to read. Many science educators contend that undergraduate students do not master enough skills to be able to successfully read and analyze the literature to gain the most from it (Gillen, 2006; van Lacum et al., 2012; Wu, 2009). This has been partially attributed to a lack of appropriately targeted guided exploration of literature by teachers in most current science curricula, such that students may take a poor strategic approach to reading the papers (Coil et al., 2010; Flaspohler et al., 2007; Hoskins et al., 2007; Kozeracki et al., 2006; Round & Campbell, 2013; Sato et al., 2014; Shannon & Winterman, 2012; Smith, 2001).

Few courses directly target the skills of interpretation and analysis of literature in higher education (Coil et al., 2010; Fang, 2005; Jackson et al., 2006; Kozeracki et al., 2006; Sato et al., 2014; Spiegelberg, 2014; Willard & Brasier, 2014; Willmott et al., 2003). Although some educators advocate curriculum modules to help first-year undergraduate students with reading the literature, because of the long-term benefits of early introduction to the literature (Bruehl et al., 2015; Willard & Brasier, 2014), others argue that the brain of a novice might not be “well structured” and it would be too difficult to acquire the ability to interpret the literature at this early stage (Flaspohler et al., 2007; Shannon & Winterman, 2012). Some educators choose to simply leave the literature out of the undergraduate science curriculum due to its difficulty (Shannon & Winterman, 2012).

Many approaches, in the setting of a formal class or as a supplementary course, have been designed to incorporate literature into science courses and help students develop the skills to gain the most from reading. Studies reveal that most undergraduate students have an outstanding improvement in their reading skill as well as understanding the scientific literature when they participate in targeted training courses (Flaspohler et al., 2007; Hoskins et al., 2007; Sato et al., 2014; Segura-Totten & Dalman, 2013; Shannon & Winterman, 2012; Willmott et al., 2003). These approaches involve journal clubs, laboratory courses, short and long training activities, primary literature modules, and journal article–focused writing tasks.

Beliefs about knowledge, confidence, and ability influence reading strategy

Students’ skills in understanding and interpreting scientific literature depend on their approaches to reading, and their skills and approaches are affected by their beliefs and attitudes toward knowledge, as well as the source of their motivation. Students’ epistemological beliefs about learning and knowledge have been found to be a significant factor in determining their ability to read and understand the scientific ideas presented in primary literature (Gottesman & Hoskins, 2013; Hoskins et al., 2011). Their beliefs are also key factors in helping students decide which kind of strategy to apply to their learning (Hofer, 2004; Hoskins et al., 2011). According to Rodríguez and Cano (2006, p. 630), epistemological beliefs and learning approaches are “generally consistent and logically interrelated constructs” among students in the tertiary setting. In other words, a simple epistemological belief about learning may lead to superficial reading and understanding, and a deep epistemological belief is generally believed to motivate students to look in more depth at the meaning of the knowledge (Rodríguez & Cano, 2006). Research shows that college students who have more sophisticated epistemological beliefs tend to employ more self-regulated motivational, cognitive, and behavioral strategies for learning and understanding (Paulsen & Feldman, 2005). How students think about the role of the knowledge has an influence on their use of strategy and their academic achievement (Coil et al., 2010; Gillen, 2006). Even so, as the task is a difficult one, even positively motivated students are likely to need additional support or instruction from the educators.

Aims of this study

This study aimed to identify how much the current second- and third-year biochemistry and molecular biology undergraduate students at the University of Melbourne are required to interact with primary scientific literature based on the assessment expectations. It compares this level of interaction with the students’ experiences, confidence, beliefs, attitudes, and views about the scientific literature. This study is a precursor to deciding what additional scaffolding of scientific literature reading and interpreting skills are needed in this major.

Methods

Document analysis

A qualitative approach of document analysis was used to map the assessment requirements of all the subjects in the major of biochemistry and molecular biology (BCMB) at the University of Melbourne, totaling nine subjects: two core and one elective in the second year and three core and three elective in the third year. The assessment task instructions and marking rubrics were analyzed to determine how the teachers require students to interact with the literature. The test and exam papers, which complement the assessment tasks, from previous years were also examined, but as they did not require any interaction with the literature, they were excluded from further analysis. To categorize the degree to which students are required to interact with scientific papers in the assessment tasks, a table of criteria (Table 1) was generated based on Bloom’s taxonomy (Krathwohl, 2002). The higher the number on the criteria, the more complex the skill in the engagement with the literature. The categories are hierarchical, as assessment tasks with higher levels usually include skills from lower numbers, too.

Quantitative online survey

A quantitative online questionnaire was designed to investigate the perceptions of the second- and third-year undergraduate students about reading scientific papers, using similar questions to previous papers that examined the self-reported beliefs, attitudes, and views of students about the importance of reading the literature and their engagement with it (Brownell et al., 2013; Coil et al., 2010; Gottesman & Hoskins, 2013; Hoskins et al., 2011; Round & Campbell, 2013; Smith, 2001; van Lacum et al., 2012; Willard & Brasier, 2014). The questionnaire had two demographic questions, then 24 closed-ended questions, mostly with a 5-point Likert scale (Online Appendix A). The survey was hosted by an online survey program (Qualtrics) and distributed to all second- and third-year students in the BCMB program in 2018 by sharing a link that the subject coordinators provided for students through the university’s learning management system. The survey was voluntary and anonymous, and a chance to win a small voucher was used as an incentive. During the 2 weeks that the survey was available, a total of 137 students completed the whole survey (or a majority of it). Of these students, 104 were from a second-year BCMB subject (18% of the 587 enrolled), and 33 were from a third-year BCMB subject (30% of the 112 enrolled).

The data were analyzed using SPSS version 22. Independent samples t-tests were used to identify any differences between gender, whether or not English was a student’s first language, and year level. Pearson’s chi-square tests were used to identify any differences between students’ answers to the same question, and the Spearman’s rank-order correlation test was used to compare their answers to different questions. A two-step cluster analysis was used to identify whether specific groups of students could be distinguished on the basis of their answer patterns. This study was approved by the School of Biomedical Sciences HEAG Application 1851625.1.

Results

Analysis of the use of literature in assessment tasks

As Figure 1 shows, in the assessments of the second-year BCMB subjects, there was little expected interaction with the scientific literature. That is, only 10% of Level 2 interaction (see Table 1 in the “understand” category of Bloom’s revised taxonomy) is required for assessment tasks in one BCMB elective subject for the second-year students. By contrast, the third-year subjects assessed the students’ ability to use the scientific literature up to Category 4, which involves explaining and interpreting. The third-year core practical subject had the highest proportion of assessment using the scientific literature, as it contains many assessment tasks geared toward scientific experimentation.

Analysis of students’ responses

Of the 137 survey participants, 71.5% were female, 27% were male, and 1.5% preferred not to say (and were not included from this analysis). The t-test showed a statistically significant difference between the means of female and male students for only five of the questions (Figure 2; see also Online Appendix B). A significant difference between genders was found in Q17 (negatively phrased, p = 0.02*), with the mean of female students higher, and Q19 (p = 0.01*), with the mean of female students slightly lower. There was a highly statistically significant difference between female and male students for Q23 (negatively phrased, p = 0.001**) and Q24 (negatively phrased, p = 0.006**), which asked about students’ stress and frustration when reading research papers. Compared with only 30% of males, 56.1% of females agreed or strongly agreed with Q23 (“Reading primary research papers is often a stressful experience for me”; Figure 2). Although many male and female students agreed with Q24 (“I tend to get frustrated when trying to understand primary research papers”), the proportion of female students who agreed or strongly agreed was 62.3%, far more than male students (38%; Figure 2).

Of the students who contributed to the survey, 113 (82.5%) had English as their first language (EFL) and 24 (17.5%) did not (ENFL). Interestingly, there was little difference between the survey responses for these sets of students according to an independent samples t-test, with just one marginally statistically significant difference in Q20 (Online Appendix B), in which ENFL students agreed more that reading the literature helps them better understand the lecture content (EFL = 3.46 ± 1.12; ENFL = 3.92 ± 0.78; p = 0.02*).

The third-year students indicated they read the scientific literature significantly more frequently than the second-year students, by a Pearson chi-square test, with a medium effect size, Χ2 (d.f. = 4, n = 137) = 14.4, p = 0.006**, Phi = 0.32 (Figure 3). Frequent reading of the literature correlated with students who reported higher confidence and internal motivation (Q5, Q8, Q9, Q12, Q13, Q16, Q17, Q18, Q19, Q20, Q22, Q23, Q24, Q26) for reading in the other parts of the questionnaire (Online Appendix E). The second- and third-year students did not report any significant difference in the amount of a paper they would read when they read the primary scientific literature (Online Appendix C), chi-square test (d.f. = 3, n = 137, p = 0.229).

There was a highly significant difference in the answers of students in second year and third year for Q5, 8, 9, 17, 18, and 19 using an independent samples t-test (Figure 4; see also Online Appendix C). In general, most second- and third-year students agreed that reading the literature is important and helpful for improving critical thinking, but they also thought it is a time-consuming and challenging activity (with no significant difference between year levels; Online Appendix D). Notably, 97% of the third-year students agreed or strongly agreed that reading the literature is important in science, even more than the second-year students, at 82% (p < 0.001***). The students in the third year felt much more confident compared with when they were in their first year, indicating that there is a strong improvement in perceived ability between the first, second, and third years. In addition, 90.9% of the third-year students agreed or strongly agreed that they learn from reading the papers, and only 65.4% of the second-year students agreed or strongly agreed with that statement (Figure 4).

Responses to Q20 and Q26 (Figure 5; Online Appendix D) suggested that students have similar motivations for reading the literature in both the second and third years, with some additional internal motivation in third-year students. In both year levels, the majority of students indicated that they read the papers only because they have to: 85.5% of second-year students and 63.6% of third-year students agreed or strongly agreed with Q10, and 74% of second-year students and 67.6% of third-year students agreed or strongly agreed with Q11. Interestingly, 58% of the third-year students said they do additional reading besides the papers recommended by the educators, whereas less than 30% of the second-year students said they do so.

Both the second- and third-year students considered that they had reasonable skills in understanding the components of the research papers (Figure 6; Online Appendix D). It is heartening that 87.9% of all students agreed or strongly agreed that they had these skills, including 77.8% of the second-year students.

Most second- and third-year students felt confident in communicating information in the primary literature and reading and evaluating primary literature, although for the latter skills, confidence levels were higher for third-year students, who are also expected to do more than this difficult skill (Figure 7; Online Appendix D).

The students’ answers to different questions were compared using the Spearman’s rank-order correlation test (Online Appendix E). There were several groups of questions that correlated together very well. One strong group was Q8, Q9, Q10, Q11, and Q12, which asked about the importance of reading papers and motivation for reading papers (Q10 and Q11 showed negative correlations). These questions also correlated with Q17 (negatively) and Q20, which asked about the broader advantages of reading papers. A second block of correlations tied together Q23, Q24, Q25, and Q26 (Q23 and Q24 negatively), which asked about confidence and frustration around reading papers. As would be expected, both these blocks of questions showed a correlation, with students reporting a higher frequency for reading of papers (Q4).

To understand whether students can be grouped consistently based on their answers to the questions, a cluster analysis was performed. The analysis identified two clusters, with fair-cluster quality, indicating that the clusters were not strongly separate but still fairly close in characteristics.

The differences between the means of the answers from Cluster 1 (64 students) and Cluster 2 (72 students) were found to be highly statistically significant (almost all p values < 0.001***; see Table 2) except for Q21 (“I have difficulty understanding why a particular experiment was performed in a primary scientific paper”; NEG). In general, Cluster 1 showed more frustration and lower confidence, motivation, and valuing with regard to reading primary journal articles, whereas Cluster 2 students scored the opposite.

The two-step cluster analysis reflected that more of the second-year students were in Cluster 1, yet more of the third-year students were in Cluster 2 (Figure 8).

Discussion

In this study, female students generally reported feeling a bit more frustrated and stressed when reading the literature compared with male students. However, there was no significant difference in the proportions of female and male students in the clusters (confident and not-confident) by a chi-square test, so the overall results of the survey did not show a strong gender difference. There are known to be differences in male and female university students’ beliefs about knowledge, and confidence in science (Karabenick & Moosa, 2005; Schommer, 1993; Yang et al., 2016). In general, male students are more likely to believe in quick learning (Schommer, 1993), and female students spend more time on understanding data and graphics (Yang et al., 2016).

Previous research indicated that if students have limited proficiency in English as a foreign language, this can constrain their science learning when the instruction and assessment of scientific content knowledge are undertaken exclusively or predominantly in English (Curtis & Millar, 1988; Lee, 2005; Tobin & McRobbie, 1996; Torres, 2002). Nevertheless, in our study, almost no difference was found between students whose first language was English and those for whom it was not, concerning their opinions on literature reading. Interestingly, more students whose first language is not English considered that reading the literature helped them gain a better understanding of the lecture content. This may show that written tasks can help students whose second language is English keep up with scientific terminology, as this knowledge can be hard for them to obtain through only listening to the lecturers in class due to language challenges.

In this study, students who were intrinsically motivated to read and saw the value in reading reported that they read the research papers more carefully and were more likely to do additional reading. It has been previously reported that college students who perceive more importance in reading the literature appear to apply more self-regulated motivational, cognitive, and behavioral strategies to learning and understanding (Hoskins et al., 2011; Paulsen & Feldman, 2005; Rodríguez & Cano, 2006).

Compared with students in their second year, the third-year students reported reading more frequently and reading more articles overall. Indeed, during this major, all of the third-year subjects required that students engage with scientific papers, including Level 4 and 5, synthesis, evaluation, and critique. In particular, the third-year practical (laboratory-based) subject had the highest emphasis on students engaging with, and analyzing, the scientific literature, with an assessment specifically requiring the synthesis, evaluation, and critique of a paper in a group presentation and an individual written report. In comparison, second-year students did not have to read the literature in the two core subjects. This may be because many academics consider the literature to be quite challenging for novices to read and interpret, and they assume that the students lack the relevant skills (Fang, 2005; Gillen, 2006; van Lacum et al., 2012; Wu, 2009). However, it could be beneficial to include a scaffolded approach in which guided exploration during the second year is followed by more independent reading in depth in the third year.

The higher level and frequency of reading primary literature expected of the third-year students were reflected in their higher confidence and motivation. The third-year students showed more advanced attitudes, perceptions, and beliefs about reading the literature compared with the second-year students. Many students in both years considered reading the research papers to be important for their study and for fostering their critical thinking, but the third-year students felt it was much more important than the second-year students. In addition, the third-year students tended to think that they had more scientific knowledge acquisition through reading and were more likely to show more confidence in interpreting the literature than the second-year students. In addition, a higher proportion of third-year students fell into Cluster 2, the high confidence and motivation cluster, compared with second-year students. It is important to note that these are two different cohorts from the same calendar year, so it is not clear from these data whether the students change between the second and third years or whether a self-selection process caused more motivated students to enter these third-year subjects.

Interestingly, there was no difference in the second- and third-year students’ reported ability in reading and understanding the literature. There was also no difference in the amount of a paper that a student would choose to read, when they did read the papers. The amount of a paper that a student reads is a strategic decision based on what an assessment requires of the student, so it is interesting that the students made a similar range of decisions in the second and third years despite very different requirements from their assessment tasks.

The majority of students in this study considered that reading the literature was hard, with 83% who agreed or strongly agreed that reading the primary literature is challenging. The primary scientific literature can be difficult for anyone to read, particularly novices. It is positive to find that second-year students considered the literature to be so important. This suggests that it is worth increasing the use of literature in second-year subjects, though some guided exploration or scaffolding seems necessary.

It was promising to note that the majority of second-year students stated they are less intimidated by reading the literature and more comfortable interpreting the scientific elements of the papers compared with those in their first year. Almost all third-year students agreed or strongly agreed with those two statements as well. There is not much scaffolded assistance to help students gain confidence and skills as they move from first to second year in terms of reading and interpreting the scientific literature, so it seems likely that this self-reported improvement is a result of the students’ increased scientific content knowledge and understanding of the processes and language of science from other areas of the curriculum. Their skills could be improved further by adding a guided exploration of the literature at an earlier time in the first or second year. Scientific literature introductory programs, such as CREATE and journal clubs, which do not have any laboratory component and are relatively inexpensive to run, have been found to significantly improve students’ ability to think scientifically and understand the primary literature (Gottesman & Hoskins, 2013; Hoskins et al., 2011; Hoskins et al., 2007; Krontiris-Litowitz, 2013; Marusic & Marusic, 2003; Segura-Totten & Dalman, 2013; Smith, 2001; Stevens & Hoskins, 2014).

In fact, it is possible that the students feel that their skill in reading the literature has improved from the second year to the third year simply because they have been involved in more active learning. In particular, they have learned to consider the scientific process and how to work with data and think about how data are represented in figures and tables. This sort of data-driven approach to science can improve students’ understanding of the scientific method and the content that they are learning, leading to an increased confidence and ability to read scientific literature (Auerbach & Schussler, 2017; Round & Campbell, 2013). Literacy can be taught using briefer assessments such as an annotated bibliography, which is used frequently in the humanities but not commonly in science (Flaspohler et al., 2007). The use of any scientific literature introductory programs for college students at an early stage would also be of great help for enhancing their knowledge of science, as discussed earlier.

Conclusion

The ability to interpret scientific literature is a key skill for research students in science. However, most students in the tertiary educational setting are initially not very confident in their ability to read the literature. This study analyzed the experiences of second- and third-year students with the literature in detail in a biomedical science undergraduate program at the University of Melbourne in Australia. The study showed that students’ perceptions, attitudes, and beliefs largely and positively influence the frequency with which they do the readings, the amount of readings they do, their acquisition of knowledge, and their reading comprehension. Based on the data analyzed, this study implies that the third-year educators tend to expect their students to engage more deeply with the literature than second-year educators expect. In line with the higher requirement, third- year students showed a higher level and frequency of reading primary literature. Our data also reflected that third-year students expressed more advanced attitudes toward reading the literature, higher confidence with interpreting the literature, and more positive perceptions and beliefs about reading the literature compared with the second-year students. To develop students’ engagement with primary scientific literature and their reading experiences across the program, we recommend that educators consider providing guided exploration for undergraduate science students and that these experiences be scaffolded across the different levels to accommodate students’ varying skills and understanding.

Yi Hou (yi.hou@monash.edu) is currently a doctoral student in the Faculty of Education at Monash University in Melbourne, Australia. Heather Verkade (heather.verkade@unimelb.edu.au) is an associate professor in biomedical education in the Department of Biochemisry and Pharmacology, School of Biomedical Sciences, Faculty of Medicine Dentistry and Health Sciences, and Jan van Driel (j.vandriel@unimelb.edu.au) is a professor in science education, Melbourne Graduate School of Education, Faculty of Education, both at the University of Melbourne in Australia.