Engaging Undergraduate Students in Course-based Research Improved Student Learning of Course Material

INTRODUCTION

Research experiences benefit undergraduate students by offering opportunities to engage in critical thinking and problem solving beyond the textbook and known experimental outcomes (Kardash, 2000; Russell et al., 2007; D’Arcy et al., 2019). AAAS Vision and Change: A Call to Action suggested incorporating research into undergraduate education for students to appreciate the setbacks and unexpected outcomes of scientific research and to apply analytical skills and critical thinking to understanding their results (Bauerle et al., 2011). Skills developed during undergraduate research experiences (UREs), such as teamwork, critical thinking, and oral and written communication, help prepare students for the workforce independent of whether they stay in a Science, Technology, Engineering, and Mathematics (STEM) field (McClure-Brenchley et al., 2020). Participating in undergraduate research is also associated with increased retention and likelihood of pursuing scientific research as a career (Mastronardi et al., 2021). Course-based undergraduate research experiences (CURE) provide students the opportunities to obtain scientific process skills while also having a greater outreach compared with one-on-one undergraduate research experiences (Bangera and Brownell, 2014; Burmeister et al., 2021). CURE classes are defined as integrating scientific practices, discovery, collaboration, and iteration with broadly relevant work. All five criteria must be met for a class to be a CURE, although every CURE may cover each criterion in varying degrees (Auchincloss et al., 2014).

Various surveys for undergraduate research experiences are available for measuring psychological and knowledge-based gains of participating in a CURE class. Using these tools, research has shown several benefits to transitioning to a CURE class. Biology Intensive Orientation Summer (BIOS) is a CURE that originated in China to help undergraduate students gain research skills and graduate students gain mentor skills. The CURE gave students the confidence and skills to pursue mentor based UREs, which was a graduation requirement (Fendos et al., 2022). Another study found that, after participating in a CURE for a year, students perceived gains in scientific literacy, data collection, presenting results via oral and written communication, and maintaining a lab notebook (Peteroy-Kelly et al., 2017). Other CUREs, such as the biochemistry authentic scientific inquiry laboratory (BASIL) CURE, measured student reported gains in lab skills, aka anticipated learning outcomes or course-based undergraduate research abilities. Students were asked about their knowledge, experience, and confidence in the seven anticipated learning outcomes in pretests and posttests and reported gains in all seven areas (Irby et al., 2020). Measuring gains in content knowledge is rarer, but a study by Wolkow et al. followed up with students 1 and 3 years after taking an introductory biology class to which students were randomly assigned to either a CURE or traditional lab class (Wolkow et al., 2014; Wolkow et al., 2019). One year after the course, students in the CURE lab reported greater psychological gains, such as enjoying the class and considering a research career, and performed better on an assessment used to measure gains on topics covered in the CURE lab. The knowledge gains for general introductory biology were comparable between groups (Wolkow et al., 2014). By senior year, perceived gains were no longer different between those who were in the CURE and traditional lab classes as freshmen, and the knowledge gains for general introductory biology were comparable, too. However, the targeted knowledge gains of what was covered in the lab classes remained significantly higher in the CURE group (Wolkow et al., 2019).

In many CUREs, students develop their own questions and experiments to fulfill the five criteria of integrating scientific practices, discovery, collaboration, and iteration with broadly relevant work. Although students report positive outcomes when asked to compare CUREs with previous traditional labs they have taken, obtaining empirical, measurable benefits for students to engage in undergraduate research is difficult when questions and experiments vary by semester or even by lab group (Linn et al., 2015). In collaboration with Dr. Brownell, we agreed members from her lab could interview our students to determine whether any differences in cognitive and emotional ownership existed between the two class formats and, if so, whether that impacted student perceptions on collaboration, iteration, or discovery/relevance. The interviews were performed in 2016 (traditional), 2017 (CURE), and 2018 (CURE). Based on that collaboration, we learned that changing our Experimental Immunology class to a CURE format did not significantly impact collaboration or iteration, but the CURE format students perceived their data were novel and relevant outside of class. CURE format students also expressed increased cognitive and emotional ownership compared with traditional format students (Cooper et al., 2019). To ease the transition from a traditional format to the CURE format, the only change made between the formats was that the CURE students researched how a genetic change impacted the immune response alongside the control experiments by comparing the known immune responses of wild-type (WT) mice to previously uncharacterized genetically modified mice. The experiments and assessments were unchanged between class formats. We realized retrospectively that we were in a unique position to empirically measure whether and how incorporating research impacted students and therefore fulfill the gap in knowledge detailed by Linn et al. (2015). We knew our students had increased cognitive and emotional ownership when research was incorporated into the course (Cooper et al., 2019), and ownership has been linked to improved student performance (Martínez et al., 2019). Therefore, the first question we asked was whether incorporating research into the course resulted in improved overall performance? This question was examined using three main variables: overall course performance; sets of quizzes, reports, and exams; and individual assessment items. While we could not quantitate the amount of literature read or scientific skills acquired, we were able to compare the scores of the direct summative assessments intended to measure student learning. In this lab course, we had direct summative assessments in the form of lab reports, quizzes, and two exams. During the semester, we also assessed participation and lab notebooks to encourage students to come prepared to perform experiments and to understand the material beforehand, but we did not use participation or notebook grades to measure how well students learned the course material since those assessments were tools to ensure students came to class knowing the procedures and actively participated. Therefore, we compared the total results from direct summative assessments (lab report, quiz, and exam grades) to determine whether incorporating research into a lab class resulted in any impact on student learning. The quizzes and exams remained identical between the two class formats, which provided another control variable when comparing class formats. Because the Teaching Assistants (TAs) grading the assessments did not know scores would be compared by the professor after the transition to a CURE format, we argue the assessments were graded without bias for CURE or traditional formats. Because students were not told beforehand whether the class was traditional or CURE format, the “volunteer effect” also did not impact our findings (Brownell et al., 2013). In a second experimental question, we asked whether additional factors influenced course performance or were distinct between traditional versus CURE formats, including grade point average (GPA), academic major, academic year, grading trends, and racial/ethnic or gender diversity.

MATERIALS AND METHODS

This study was conducted with an approved Institutional Review Board protocol (#4249) from Arizona State University.

Traditional Class Format

The class was divided into five sections each consisting of two or three laboratory exercises. The five sections were anatomy and cells of the immune system (labs 2–3), innate immunity (labs 4–6), adaptive immune system development (labs 7–8), acute adaptive immune response (labs 10–12), and immune memory and protection (labs 13–15). All experiments and protocols followed the lab manual “The Immune System: An Experimental Approach” (Blattman et al., 2016). The purpose of each lab could be copied from the lab manual. In other words, the traditional lab class format was prescriptive or “cookbook.” While students wrote individual hypotheses, the findings were not novel and no outside literature was needed to support the hypothesis; all necessary information to form a hypothesis was in the lab manual. All lab experiments were performed using immune cells from recently killed WT Bl6 mice (IACUC 19-1684T). The class structure was to start with a quiz, review the quiz, answer students’ questions regarding immunology and the day’s lab exercises, and then let the students perform the experiments and, when applicable, gather data the same day. Notebooks were signed at the end of class. Students were encouraged to have everything written in their notebooks and review class material before class started.

CURE Class Format

To transition to a CURE format, students studied mice with a genetic mutation that had not been studied in immunology thereby generating novel data. By collaborating with other laboratories within Arizona State University, students studied what effect knocking out Mohawk (2017, Alan Rawls), having a Raf1L613V mutation (2018, Jason Newborn), or knocking out Z-DNA-binding protein 1 (2019, Bertram Jacobs) had on the immune response. The five areas of immunology studied, the lab manual, and protocols remained unchanged between traditional and CURE formats. The purpose of each CURE lab shifted to understanding how the immune response of a genetically modified mouse differed from the WT mouse. CURE students were required to read outside literature before class started to generate a novel hypothesis. They were instructed to hypothesize how the immune response would differ (better, worse, or no change) between mice and provide their own reasoning as to why. Other than encouraging students to find publications on pubmed, instructors did not help students generate hypotheses. Before experiments started, students discussed their hypotheses in small groups before sharing their different hypotheses with the class. Instructors encouraged students to share their different hypotheses by asking, “Did anyone think the immune response would be better in the knock out mouse? Why? Who thought there’d be no change? Why?” and finally saying, “All these hypotheses are valid. We don’t know the answer yet because the experiment has never been done before.” The class structure was to start with the quiz, review the quiz, answer immunology questions, discuss hypotheses and reasoning, answer questions related to lab exercise, let the students perform the experiments and, when applicable, gather data the same day. Notebooks were signed at the end of class. Because there was no time to generate a hypothesis during class, students were required to read the course material and apply it to the genetic mutation before coming to class.

Quizzes

At the beginning of every lab, students took a quiz on the immunology on which the lab was based and the experiment itself. The first and last quizzes were omitted because the first quiz was used to show students how the class would flow throughout the semester and therefore did not count toward the final grade and the last quiz was a practical to determine student ability to analyze flow cytometry data.

Before class, both teaching methods required students to read the lab material, write the purpose, question, hypothesis, and procedures for the day’s lab in their notebooks, and take a quiz at the beginning of class. Students used the same book with the same optional practice questions and took the same quizzes. Although both class formats required students to come to class prepared, the CURE teaching method enforced that requirement because incorporating real scientific research into the class required CURE students to develop a novel hypothesis on how altering the gene of interest would impact the immune response. Due to time constraints, all reading for generating their novel hypothesis needed to happen before the class started. Immediately after reviewing the quiz, CURE students discussed their hypotheses in their groups for 2 minutes prior to sharing with the class via random call. To receive a notebook signature for the day’s lab, CURE students needed to have citations for their hypotheses. Students were expected to have hypotheses with citations before the start of class beginning with the second lab quiz.

Reports

Students wrote lab reports after the first four class sections. The fifth lab report was not included in this analysis since students taking the class 2013–2015 were not told to write a fifth report, and students in 2016–2019 could write the fifth report to replace the lowest report grade. Therefore, the significant difference between report grades was calculated based on reports 1 through 4 without replacing any scores since report 5 was omitted. Regardless of class format, students followed the same report rubric. Each report was worth 50 points, and the points were Introduction-7, Methods-5, Results-12, Discussion-15, References-5, Grammar-2, Legends/captions-2, and Formatting-2. In the introduction, students were expected to provide relevant information, purpose of the experiments, questions answered, and hypotheses. The CURE students not only provided the relevant immunology background for the report but also read additional literature for relevant background information regarding the gene of interest. This background reading (which took place before class and therefore before the quiz) then needed to provide a clear link to the hypothesis. Students were told the hypothesis should answer whether they expected the immune response would be greater, the same, or less than the WT mouse and why. The methods remained unchanged other than the CURE students had one additional sample to run due to also analyzing the immune response from the genetically modified mouse. For results, students in both class formats analyzed immune organ cell counts, flow cytometry data, ELISA results, and cytotoxicity data for their reports. However, students taking the CURE format had additional samples and needed to compare WT results with the genetically altered mouse. While the analysis itself was similar given the rubric (figures, description/summary, how data were obtained, and identifying controls in the experiments), CURE students analyzed two sets of data, learned how to prevent bias between samples, and then compared/contrasted the data in the discussion. In the discussion, both class formats read outside literature and discussed the impact of the data. Both formats analyzed WT data and determined whether the data obtained fit within expected values. For the CURE students, the WT data served as a control that then told them whether they correctly performed the experiment. Therefore, if the WT values fit the expected norm, then the data from the genetically altered mouse, which used the same methods and reagents, could be believed. Students then read further literature to try to understand the reasoning behind the results from the genetically altered mouse and then showed how their research regarding the gene of interest had impact outside of class. Both class formats discussed the impact of the established immunology and why the immunology was important to study (Supplemental Table S1).

Exams

The class had two exams: the open book take-home midterm was given as a hard copy before spring break and due when classes resumed and the closed book in-class final.

Controls

The students followed the same rubric for lab reports (Supplemental Table S1) and had the same quizzes and exams. Quiz and exam questions consisted of multiple choice, fill in the blank, drawing, short answer, identify cells or organs, and math. Short answer and drawing questions could have resulted in variance in TA grading. However, any variance was mitigated by reviewing all quiz answers in class and exam answers when requested. The professor, who did not change between formats, was present for classes and answered questions regarding which short answers were or were not acceptable and whether partial credit would be granted. Drawings were also reviewed in class using either a whiteboard or TV screens depending on class size.

If the increase in grades in the CURE format were due to students obtaining copies of previous quizzes, then we would expect quiz grades to have started rising during the 4 years the traditional format was taught. The midterm exam was always an open book take home exam given before students left for spring break. The final exam was a closed book, in class exam for both class formats and had the same questions and available number of points.

For both formats, TAs encouraged study practices for the final exam. In the traditional class, students were allowed a notecard during the final. In the CURE format, students had an in-class quiz-like review session 2 days before the final. While the TAs provided different study aids, students still studied on their own. In other words, the students in the traditional class could have still quizzed themselves while students in the CURE class were observed taking notes during the review session.

Regarding lab reports, different TAs graded the lab reports depending on the year. However, the same rubric was followed for grading, and the available number of points for each report remained consistent within the class format.

Finally, the quality of education remained consistent across the different class formats. The same professor was responsible for the class even though the TA teaching the class changed. In both formats, the class TAs were recognized for quality teaching. The traditional format was taught by a TA who was student-nominated and awarded ASU’s Teacher of the Year. The CURE format was taught by a TA who was self-nominated and awarded GPSA’s Teaching Excellence Award.

Student Demographics

GPA, degree program, and academic level were all analyzed in the results section as described below. The class did not have any prerequisites to enroll and was not required by any degree program at Arizona State University. In other words, the likelihood of students enrolling in Experimental Immunology remained consistent between class formats. A lecture class (MIC 420: Basic Immunology) was offered all the years that Experimental Immunology was taught. However, the lecture class was not required, and both the traditional and CURE class formats had a mixture of students who had and had not taken the lecture. Students did not elect to enroll in a CURE or traditional class and were not told prior to enrollment that the class format had changed to a CURE. Other demographics, including prior research experience, were assessed previously and not found to be significantly different between class formats (Cooper et al., 2019).

Statistical Analysis

Scores from quizzes, reports, and exams were pulled from 2013 to 2019 and analyzed for any differences via GraphPad Prism unpaired t tests. Correction from multiple t tests was done using false discovery rate determined by two-stage step-up (Benjamini, Krieger, and Yekutieli). GPA was also analyzed via unpaired t tests to determine significance.

The Shannon–Wiener diversity index, Chi-squared, and Fisher t test were used to determine level of diversity for degree program, academic level, race, and gender. The Shannon–Wiener diversity index is a way to measure diversity within a population (Shannon, 1948). A t test was used to compare results from the Shannon–Wiener diversity index (Hutcheson, 1970). Further analysis for degree program, academic level, and race used Chi-squared. Significant differences in gender were determined using the Fisher t test.

GraphPad Prism’s multiple regression analysis was used to determine the impact of the predictor variables GPA, class format (0-Traditional, 1-Cure), and academic level on the outcome variable (overall points earned). Further analysis studied the predictor variables on points earned on quizzes, reports, and exams separately.

Scores from quizzes 2013–2016 were analyzed via one-way ANOVA in GraphPad Prism.

RESULTS

Students in CURE Class Averaged ∼5% Higher than Students Taught via Traditional Method

To determine whether students learned more course material in the CURE format, we compared overall course grades and found that students in the CURE class performed better overall with a class average of 80% compared with students in the traditional format course who averaged 75% (p < 0.0001) (Figure 1A). The aggregate semester grade was calculated from student quizzes (79% vs. 71%, p < 0.0001) (Figure 1B), lab reports (84% vs. 78%, p < 0.0001) (Figure 1C), and exams (82% vs. 77%, p < 0.01) (Figure 1D). On all three assessments, CURE format students had significantly improved performance, which suggests incorporating research into the immunology laboratory class resulted in improved understanding and application of the course material.

Changing Class Format to a CURE Improved Majority of Quiz Scores

We used quizzes to test student preparedness for class and understanding of important background information each laboratory period. CURE students achieved significantly higher scores on seven out of 13 quizzes (Figure 2). Early in the semester, the CURE teaching method resulted in students performing significantly better on the second quiz (88% vs. 80%, p < 0.01). The CURE teaching method resulted in continued improved performance when viral infection was mimicked in quiz 5 by studying the innate immune response to poly(I:C) (61% vs. 51%, p < 0.01) and when lymphocyte development was studied in quiz 7 (88% vs. 78%, p < 0.0001) and quiz 8 (88% vs. 81%. p < 0.01). Later in the semester, lymphocyte response to virus was studied, specifically how T cells respond to viral infections. The difference in quiz scores between teaching methods then often exceeded 15% such as in quizzes 10 (76% vs. 58%, p < 0.0001), 11 (82% vs. 61%, p < 0.0001), and 14 (83% vs. 66%, p < 0.0001). Scores for quizzes 12 and 13 were not significantly different between teaching methods (70% vs. 62% and 80% vs. 75%, respectively). When the focus was on learning a new technique instead of forming a new hypothesis, such as quiz 6 (79% vs. 80%) and quiz 9 (77% vs. 75%), no significant difference in scores was noticed. Scores for quizzes 3 and 4 scores were also not significantly different between teaching methods (87% vs. 82% and 67% vs. 71%, respectively). Lab 3 studied cells of the immune system and reviewed fundamentals for flow cytometry, which the class used to analyze data. Lab 4 studied oxidative burst, which occurs when leukocytes encounter a pathogen. Overall, seven of the 13 quizzes were significantly improved for CURE format versus traditional format. Of the six quizzes that were not significantly improved, two involved learning a technique instead of generating a hypothesis before obtaining novel results.

Incorporating Scientific Research Resulted in Improved Performance on Reports

While quizzes demonstrated student preparedness for class, we used laboratory reports to assess student analytical skills for interpreting data, as well as critical thinking about how their work applied to current research outside the class. The rubric for grading reports was unchanged between class formats. We found significantly improved scores for all four analyzed laboratory reports from CURE format students compared with scores from traditional format students. As with quizzes, incorporating research into the class benefitted students from the beginning. The CURE format resulted in students earning 6% higher on the first report (74% vs. 68%, p < 0.01). Students appeared to incorporate feedback from the first report regardless of class format given the second report was one letter grade higher for both sets of students. However, the benefit of incorporating research early resulted in the CURE class still scoring 8% higher (87% vs. 79%, p < 0.0001). The third report (89% vs. 83%, p < 0.0001) and fourth report (87% vs. 84%, p < 0.05) also demonstrated improved scientific writing when research was incorporated (Figure 3).

Midterm, but not Final, Exam Scores Improved in CURE Format

Two exams were given to assess student mastery of the course material. A midterm exam assessed student understanding of labs 1–9, and a final exam covered material from labs 10 to 15. The questions and format were the same for exams for the traditional and CURE format courses. Again, students in the CURE format class performed significantly better on the midterm exam compared with students from the traditional format course (88% vs. 83%, p < 0.0001). However, student performance on the final exam did not differ significantly between traditional and CURE format courses (75% vs. 73%) (Figure 4).

Student Demographics Remained Consistent Between Formats

The improved performance on quizzes, exams, and lab reports in the CURE format course compared with the traditional format course, despite no other differences in format or assessments, suggests incorporating research into a laboratory course increases student mastery. However, other factors including student demographics could also result in this change. To determine whether the student population changed, data on students’ overall academic GPAs, degree programs, and academic levels were analyzed. No significant difference was observed across GPA (p = 0.07) (Figure 5C), degree program (p = 0.6) (Figure 5A), or academic level (p = 0.4) (Figure 5B). Therefore, the students who took the traditional class format were equally capable of mastering the course material as students who took the CURE class format. The improved performance observed in the CURE format was due to incorporating research into the teaching method.

Multiple Linear Regression Analysis Indicates the Teaching Intervention Improved Student Scores

Multiple linear regression (MLR) controls for other variables that impact student performance and is therefore a reliable method for determining whether a teaching intervention, such as incorporating research, impacted student performance and learning or whether the change in performance was due to student-intrinsic factors (Theobald and Freeman, 2014). In setting up the MLR analysis, we chose student GPA, class format, and academic level as the predictor, or control, variables. The outcome variable was the total number of points earned in the class. GPA represented overall academic performance. Academic level helped measure preparedness of previous coursework since more senior students would likely have taken more life science classes to prepare them for an upper division immunology course. Class format represented the teaching intervention, which was incorporating research. MLR analysis showed how well incorporating research impacted student performance (p = 0.0004) in the class when the predictor variables were controlled (Table 1). GPA also served as a good indicator of well a student would do in the class (p < 0.0001), but academic level had no impact on how well students performed in class. Individual analyses were performed for quizzes, reports, and exams with similar findings (Supplemental Tables S2–S4).

Improved Scores were Due to Changing the Teaching Format without Changing Assessments

If the increase in grades in the CURE format were due to students obtaining copies of previous assessments, especially quizzes, then we hypothesized that we would see significant increases in quiz grades prior to changing class formats. We analyzed quiz averages across the 4 years the traditional lab was taught, 2013–2016. While there was an improvement between 2013 and 2014 (Supplemental Figure S1), no further improvement was noted across the 4 years. 2013 was the first year this course was taught. Nonetheless, we reanalyzed the overall scores for quizzes, reports, and exams to determine whether 2013 falsely lowered the scores from the traditional class format. All three assessment areas remained significant between class formats when the scores from the first ever class were omitted (Supplemental Figure S2).

Assessing Differences in TA Grading Showed No Significance Difference between Formats

The traditional format had four graders over the course of 4 years (two TAs and two assistant TAs). The CURE format had two graders over the course of 3 years (two TAs). The class professor, who was consistent across formats, also graded occasionally. Although unlikely that the clear divide in grades across class formats was due to grading variances since there were seven total graders for the class, we sought to determine whether differences in scores were due to grading. We therefore analyzed the coefficient of variance (%CV) within samples and compared the two class formats. Any %CV due to student-intrinsic factors would be similar between teaching methods because student demographics were comparable between class formats. If %CV were significantly different between formats, then other factors, including differences in TA grading, would likely be responsible. The %CV for quizzes, reports, and exams were not significant between class formats (p = 0.0671, 0.3162, and 0.8858, respectively) (Figure 6), which suggests that the grading rigor was comparable between class formats.

Experimental Immunology Taught a Diverse Student Population

CUREs are known to include more students in research and have a broader outreach than the traditional one-on-one mentoring method (Bangera and Brownell, 2014; Burmeister et al., 2021). Both the traditional and CURE class formats rated high on the Shannon–Wiener diversity index with richness scores of 7 and 11, respectively. There was no significant difference between class diversity as calculated via Shannon–Wiener diversity index (p = 0.2) or Chi-squared test (p = 0.3255) (Figure 7A). Both the traditional and CURE class formats had over 50% female students, and the ratio of female to male students was not significantly different between class formats as calculated via Shannon–Wiener diversity index (p = 0.2) or Fisher’s exact test (p = 0.1298) (Figure 7B). Overall, the Experimental Immunology class serves a diverse group of students. By changing the class format to incorporate research, we included the diverse student population we serve in critical thinking and problem solving.

DISCUSSION

While incorporating research into existing laboratory courses benefits students by encouraging critical thinking, problem solving, and reading current literature to show how their work is novel and applicable outside class, quantitating the impact of integrating research on student mastery of the course material has been difficult (Linn et al., 2015). We changed the format of an upper division immunology lab course into a CURE class by having students study the immune response of previously uncharacterized genetically altered mice compared with the known response of WT mice. The class structure, such as the experiments performed, lab manual used, and assessments, remained unchanged between formats thereby allowing us to compare the effect incorporating research has on student performance in the class. We realized retroactively that we were therefore in a unique position to determine whether incorporating research improved student learning of the original course material as evidenced by improved scores.

Overall, we found incorporating research into the class resulted in students performing significantly better in all assessment areas. The overall difference in student performance was not surprising since every assessment showed that incorporating research improved student performance. The difference in quiz scores could be due to the nature of the hypotheses required for both classes. Because students studied known outcomes in the traditional class format, the lab manual often provided enough information for students to know what to expect and why. However, the lab manual did not detail any genetic mutations. While CURE students would have read the same immunology background from the lab manual, they had the additional responsibility to apply what they learned to whether a genetic mutation would impact the immune system and, if so, how. Encouraging students to apply their knowledge before taking any assessment likely resulted in improved learning of the course material and therefore higher quiz scores (Freeman et al., 2014).

Incorporating research improved students’ scientific writing as evidenced by improved lab report scores. Writing about real scientific results in the results and discussion sections, which are responsible for over half the lab report grade, likely made scientific writing more approachable. For example, the discussion section required students to compare their results with outside literature and explain why their results did or did not agree with current literature. The CURE teaching method required students to start reading outside literature before writing the report. Therefore, CURE students had an advantage regarding which literature sources to cite for the discussion because they already read multiple sources to formulate a hypothesis. The increased ownership and engagement in the class (Cooper et al., 2019) may have also resulted in increased scores as they had to write why their novel results were important outside of class (Conley and French, 2014; Cannata et al., 2019; Martínez et al., 2019). The traditional teaching format encouraged outside reading before writing the report but did not require it. The results were not novel for the traditional teaching method, and therefore the impact of what was studied focused less on their results and more on how the experiments studied are still used in current research.

The significant difference in exam scores resulted from the CURE teaching method improving student scores by 5% on the midterm exam. Because the midterm exam was an open book take home exam completed over spring break, students in both class formats had equal access to the lab manual and previous quizzes to do equally well on the exam. Therefore, the difference in scores was likely due to student motivation to devote the time to do well on the exam (Dweck, 1986) possibly due to increased project ownership (Cannata et al., 2017). The midterm also correlated with the increased peak in quiz performance suggesting that students in the CURE format exhibited higher levels of engagement immediately after spring break.

The difference in scores began to decline when quizzes and assignments for the class overlapped with projects necessary for students to graduate, such as capstone and honors thesis projects. If students experienced equal levels of burnout or had multiple assignments due around the time the final was taken, then student engagement in the class would be comparable between class formats and therefore explain why the scores on the final exam were not significantly different.

Active learning encourages students to take ownership of their education by actively participating in what they learn. By changing the lab format to a CURE class, students received guidance on how to look up and interpret journal articles. Once empowered in ways to educate themselves and look up information beyond what was provided in the course materials, the improved scores suggest students truly engaged in the class and took ownership of their projects and their education. Teaching the students scientific processes, such as graphing, data analysis, experimental design, scientific writing, and science communication, before students enrolled in introductory science classes improved content learning when students later enrolled in introductory biology classes despite minimal differences in student GPA/SAT scores (Dirks and Cunningham, 2006). CUREs teach scientific processes alongside class material (Auchincloss et al., 2014). Our work supports that CUREs are an effective way to improve undergraduate education by engaging more students in scientific research. We showed transitioning to a CURE format resulted in a similar improvement in scores compared with teaching scientific processes separately. This further shows the CURE format enhanced student learning of course material rather than distracted from it, which is a concern raised when educators express reasons for not integrating more active learning in their courses (Kim et al., 2018; Shadle et al., 2017; Ul-Huda et al., 2018).

One limitation of this study is that different TAs were present and responsibilities, such as grading, shifted as TAs shifted. To mitigate variation, answers were reviewed in class whenever possible (always for quizzes, upon request for exams). Regarding quizzes, since all those answers were approved by the professor who was present for both class formats, CURE students performed better than traditional students in seven of the 13 quizzes, or seven of the 11 quizzes that involved applying their knowledge to a genetic mutation. A subset of assessments was unavailable to regrade to verify the impact of adding research because assessments were handed back to students. Nonetheless, most questions did not have multiple correct answers. The only questions that could have different points awarded based on TA leniency were drawings and short answers. To understand whether there were any significant differences, we compared %CV between formats. We showed through multiple analyses (MLR, Chi-squared, diversity index, and previous work from Cooper et al. (2019)) that student demographics were not responsible for the difference in scores. Any %CV related to student ability would remain consistent between formats. We reasoned that a change in %CV would therefore be due to other variables, such as TA grading leniency. Our data showed that %CV was not different between class formats. Therefore, we do not believe having different graders significantly impacted the scores because no variance was noted within the class formats themselves.

We also analyzed whether academic dishonesty in the form of obtaining quizzes from prior years could have resulted in improved scores. While the first year the class was taught did have lower scores compared with subsequent semesters, no further improvement occurred. The improvement from 2013 to 2014 likely resulted from having an experienced professor and experienced TAs.

Another limitation of this study is we had no information on differences in family support and/or responsibilities (parents, spouse, children), socioeconomics such as whether the students were working to support themselves or were supported by family, and other personal factors that could affect student performance in the class (Mushtaq and Khan, 2012). Nonetheless, previous studies showed prior test scores, course knowledge, and experience had the highest correlations to student performance when compared with other factors such as learning/teaching styles, gender, and family stress (Van Lanen et al., 2000; Clark and Latshaw, 2012; Mushtaq and Khan, 2012). Because no significant difference was noted in GPA, degree program, or academic level between the two formats, the improved scores likely resulted from the teaching method and not the students. This was further supported via MLR analysis in which class format was found to contribute to student performance when all other variables, including GPA, were controlled. GPA was accessed when most of the students had graduated Arizona State University, which means the GPA analyzed was likely the students’ final GPAs or close to their final GPAs. We recognize that our students had both visible and invisible diversities that were not disclosed in interviews or through demographic information. Therefore, measuring to what extent transitioning the class format to a CURE class impacted each demographic is outside the scope of this study. Nonetheless, we did note that students in the CURE class performed better overall. We hope this information encourages educators to include active learning, particularly course-based research, in their classes and universities to offer rewards and incentives for educators to update courses as needed to improve student engagement and thereby improve student mastery of course material.

Overall, we showed incorporating research into an upper division lab improved student learning and mastery of course material. Changing the class to a CURE format resulted in students experiencing increased project ownership, which was likely associated with increased engagement in the course and ownership of their education which then translated to improved scores on assessments. We were able to show this because the overall class structure and assessments remained unaltered between the traditional and CURE class formats; the only change was students studied how a previously uncharacterized gene impacted immune system development, response, and memory. Over the course of 3 years, 139 students from diverse backgrounds, some of which are underrepresented in science, participated in scientific research through this class, which supports that CUREs can engage large numbers of diverse students in science (Auchincloss et al., 2014).

ACKNOWLEDGMENTS

REFERENCES